docs: 📝 add start of the creating resources guide #810

signekb · 2024-10-24T08:32:32Z

Description

This adds the guide on creating and managing data resources. It is not complete as there are some things I don't know how they may work from an implementation perspective, but this is a good starting place for us.

Closes #759

This PR needs an in-depth review.

…prout into docs/guide-for-managing-resources

…m/seedcase-project/seedcase-sprout into docs/guide-for-managing-resources

…prout into docs/guide-for-managing-resources

signekb

Nice 👍 Just some suggestions.
(bc I made the PR, I can't request changes or approve, only comment - so to show that I actually request changes, I have manually moved this PR to in progress on the board)

docs/guide/resources.qmd

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>

…prout into docs/guide-for-managing-resources

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>

…m/seedcase-project/seedcase-sprout into docs/guide-for-managing-resources

docs/guide/resources.qmd

signekb · 2025-02-27T08:58:18Z

@lwjohnst86 Is this ready for review or there more changes on the way?

lwjohnst86 · 2025-02-27T10:15:32Z

@signekb yup!

signekb · 2025-02-28T13:40:23Z

I’ll start reviewing this now 🚀

signekb

Very nice seeing this coming together. I have some suggestions and thoughts :)
(since I created this PR, I can only “Comment”)

docs/guide/resources.qmd

signekb · 2025-02-28T15:27:53Z

docs/guide/resources.qmd

+In this case, we don't want to add anything else, so we'll write the
+text to the `README.md` file:


Couldn’t this be a nice to show how this would be done, actually?

I'm not sure what you mean. Could you expand?

docs/guide/resources.qmd

…1099) ## Description To fit the #810 and the current [naming scheme](https://sprout.seedcase-project.org/docs/design/architecture/naming#actions).  This PR needs a quick review. ## Checklist - [X] Added or updated tests - [X] Ran `just run-all`

## Description This PR adds a `path_readme()` which was referred to in a TODO item in #810 This function will probably have to be rewritten a bit when we fully switch to the “local-first” approach, but I thought it made sense to add it now, so we don’t forget about it.  This PR needs a quick review. ## Checklist - [X] Added or updated tests - [X] Ran `just run-all` --------- Co-authored-by: Luke W. Johnston <lwjohnst86@users.noreply.github.com>

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>

…prout into docs/guide-for-managing-resources

lwjohnst86 · 2025-03-06T20:00:11Z

@signekb annoyingly, I wasn't notified of your comments (or maybe I missed them, not sure). I can't re-assign you to review, just letting you know I've updated things!

signekb

@lwjohnst86 Nice! Just some more comments (i.e., requested changes) :)

signekb · 2025-03-11T15:26:50Z

docs/guide/resources.qmd

+1.  Create the properties for the resource, using the original raw data
+    as a starting point and edit as needed.
+2.  Create a folder to store the (processed) data resource in your
+    package, as well as having a folder for the (tidy) raw data.


Suggested change

package, as well as having a folder for the (tidy) raw data.

package, as well as having a folder for the (tidy) batch data.

Remove mentions of “raw data” within a resource.

signekb · 2025-03-11T15:28:12Z

docs/guide/resources.qmd

@@ -22,6 +22,22 @@ that you have a record of the steps taken to clean and transform the
 data.
 :::

+Putting your raw data into a data package makes it easier for yourself
+and others to use later one. So the steps you'll take to get this raw


Suggested change

and others to use later one. So the steps you'll take to get this raw

and others to use later one. So the steps you'll take to get your

Removing mentions of “raw data” within a resource.

signekb · 2025-03-11T15:32:23Z

docs/guide/resources.qmd

-Let's start with extracting the resource properties from the raw data.
-While this function tries to infer the data types in the raw data, it might not get it right. So, be sure to check the properties after using this function. It can also not infer things that are not in the data itself, like a description of what the data contains or the unit of the data.
+You'll start by creating the resource's properties. Before you can have
+data stored in a data package, it needs metadata (called properties) on


Suggested change

data stored in a data package, it needs metadata (called properties) on

data stored in a data package, it needs properties (i.e., metadata) on

I think it makes more sense to have them this way around so we consistently refer to properties and not metadata.

signekb · 2025-03-11T15:34:45Z

docs/guide/resources.qmd


-We've already create a package (using the steps from the [package
-guide](packages.qmd)), with the path set as the variable `package_path`:
+We assume you've already create a package (either by using the steps


Suggested change

We assume you've already create a package (either by using the steps

We assume you've already created a package (either by using the steps

signekb · 2025-03-11T15:35:16Z

docs/guide/resources.qmd

-guide](packages.qmd)), with the path set as the variable `package_path`:
+We assume you've already create a package (either by using the steps
+from the [package guide](packages.qmd) or started making one for your
+own data), with the path set as the variable `package_path`:


Suggested change

own data), with the path set as the variable `package_path`:

own data), with the path to the data package set as the variable `package_path`:

signekb · 2025-03-11T15:37:55Z

docs/guide/resources.qmd

+this package, using the helper `path_resources()` function to give the
+correct path to the resources folder. The default behaviour of
+`path_resources()` is to use the current working directory, but for this
+guide you'll have to use the `path` argument to point to where the


Is this actually “we’ll” here? Bc if they’re following along locally, they should be able to use the cwd, right?

signekb · 2025-03-11T15:40:23Z

docs/guide/resources.qmd

+Next step is to set up the resource properties so that it gets checked
+and saved into the `datapackage.json` file. You can use the


Suggested change

Next step is to set up the resource properties so that it gets checked

and saved into the `datapackage.json` file. You can use the

The next step is to add the resource properties to the `datapackage.json` file. Before they are added, they will be checked to confirm that they are in the correct shape and that no required fields are missing. You can use the

signekb · 2025-03-11T15:41:32Z

docs/guide/resources.qmd

-the first one in the package, so we can use `path_resource(1)`.
+Next step is to set up the resource properties so that it gets checked
+and saved into the `datapackage.json` file. You can use the
+`path_properties()` helper function to always give you the correct


Suggested change

`path_properties()` helper function to always give you the correct

path_properties()` helper function to give you the

I feel like “always” is promising too much — what if they give it the wrong path, for instance.

signekb · 2025-03-11T15:42:31Z

docs/guide/resources.qmd

-resource_properties = sp.create_resource_properties(
-    properties=resource_properties,
-    path=package_path / sp.path_resource(1)
+# TODO: This function needs to be updated to write to data package.


Suggested change

# TODO: This function needs to be updated to write to data package.

# TODO: This function needs to be updated to write to datapackage.json

signekb · 2025-03-11T15:43:07Z

docs/guide/resources.qmd

+pprint(sp.read_properties(package_path / sp.path_properties()))
+```
+
+## Storing a backup of the raw data


raw —> batch

lwjohnst86 and others added 2 commits October 9, 2024 20:59

docs: 🚧 draft of creating and managing resources

e1f2209

Merge branch 'main' into docs/guide-for-managing-resources

8a309c3

github-actions bot assigned signekb Oct 24, 2024

lwjohnst86 assigned lwjohnst86 and unassigned signekb Nov 3, 2024

lwjohnst86 added 6 commits November 4, 2024 19:24

Merge branch 'main' of https://github.com/seedcase-project/seedcase-s…

b1fe1c8

…prout into docs/guide-for-managing-resources

Merge branch 'main' of https://github.com/seedcase-project/seedcase-s…

f748490

…prout into docs/guide-for-managing-resources

Merge branch 'docs/guide-for-managing-resources' of https://github.co…

54e61be

…m/seedcase-project/seedcase-sprout into docs/guide-for-managing-resources

Merge branch 'docs/guide-for-managing-resources' of https://github.co…

f2cab3e

…m/seedcase-project/seedcase-sprout into docs/guide-for-managing-resources

Merge branch 'main' of https://github.com/seedcase-project/seedcase-s…

3562449

…prout into docs/guide-for-managing-resources

docs: 📝 draft of resource guide

43cc868

lwjohnst86 changed the title ~~docs: 🚧 draft of creating and managing resources~~ docs: 📝 added start of the creating resources guide Feb 18, 2025

lwjohnst86 marked this pull request as ready for review February 18, 2025 09:08

lwjohnst86 requested a review from a team as a code owner February 18, 2025 09:08

signekb changed the title ~~docs: 📝 added start of the creating resources guide~~ docs: 📝 add start of the creating resources guide Feb 19, 2025

signekb commented Feb 19, 2025

View reviewed changes

lwjohnst86 and others added 6 commits February 20, 2025 09:19

docs: ✏️ clarifications from review

a945000

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>

chore(pre-commit): ✏️ automatic fixes

426a19a

Merge branch 'main' of https://github.com/seedcase-project/seedcase-s…

1f0bceb

…prout into docs/guide-for-managing-resources

docs: ✏️ clarifications from review

e34527a

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>

Merge branch 'docs/guide-for-managing-resources' of https://github.co…

397796a

…m/seedcase-project/seedcase-sprout into docs/guide-for-managing-resources

docs: 📝 small updates to resource guide

741edcc

lwjohnst86 reviewed Feb 24, 2025

View reviewed changes

docs/guide/resources.qmd Outdated Show resolved Hide resolved

docs: ✏️ need to use mkdir to make the folder

b5f487d

Merge branch 'main' into docs/guide-for-managing-resources

1fb3537

signekb commented Feb 28, 2025

View reviewed changes

signekb mentioned this pull request Feb 28, 2025

refactor: 🚚 rename create_readme_text() to build_readme_text() #1099

Merged

2 tasks

signekb mentioned this pull request Feb 28, 2025

feat: ✨ add path_readme() #1100

Merged

2 tasks

lwjohnst86 and others added 4 commits March 6, 2025 19:59

docs: ✏️ suggestions from review

6a6df8b

Co-authored-by: Signe Kirk Brødbæk <40836345+signekb@users.noreply.github.com>

chore(pre-commit): ✏️ automatic fixes

3aa3bdd

Merge branch 'main' of https://github.com/seedcase-project/seedcase-s…

dd79dd7

…prout into docs/guide-for-managing-resources

docs: 📝 address comments from review

e9a4d9e

signekb commented Mar 11, 2025

View reviewed changes

signekb requested a review from lwjohnst86 March 11, 2025 19:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: 📝 add start of the creating resources guide #810

docs: 📝 add start of the creating resources guide #810

signekb commented Oct 24, 2024 •

edited by lwjohnst86

Loading

signekb left a comment •

edited

Loading

signekb commented Feb 27, 2025

lwjohnst86 commented Feb 27, 2025

signekb commented Feb 28, 2025

signekb left a comment

signekb Feb 28, 2025

lwjohnst86 Mar 6, 2025

lwjohnst86 commented Mar 6, 2025

signekb left a comment

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

signekb Mar 11, 2025

		In this case, we don't want to add anything else, so we'll write the
		text to the `README.md` file:

	package, as well as having a folder for the (tidy) raw data.
	package, as well as having a folder for the (tidy) batch data.

	and others to use later one. So the steps you'll take to get this raw
	and others to use later one. So the steps you'll take to get your

	data stored in a data package, it needs metadata (called properties) on
	data stored in a data package, it needs properties (i.e., metadata) on

	We assume you've already create a package (either by using the steps
	We assume you've already created a package (either by using the steps

	own data), with the path set as the variable `package_path`:
	own data), with the path to the data package set as the variable `package_path`:

		Next step is to set up the resource properties so that it gets checked
		and saved into the `datapackage.json` file. You can use the

	Next step is to set up the resource properties so that it gets checked
	and saved into the `datapackage.json` file. You can use the
	The next step is to add the resource properties to the `datapackage.json` file. Before they are added, they will be checked to confirm that they are in the correct shape and that no required fields are missing. You can use the

	`path_properties()` helper function to always give you the correct
	path_properties()` helper function to give you the

	# TODO: This function needs to be updated to write to data package.
	# TODO: This function needs to be updated to write to datapackage.json

docs: 📝 add start of the creating resources guide #810

Are you sure you want to change the base?

docs: 📝 add start of the creating resources guide #810

Conversation

signekb commented Oct 24, 2024 • edited by lwjohnst86 Loading

Description

signekb left a comment • edited Loading

Choose a reason for hiding this comment

signekb commented Feb 27, 2025

lwjohnst86 commented Feb 27, 2025

signekb commented Feb 28, 2025

signekb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lwjohnst86 commented Mar 6, 2025

signekb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

signekb commented Oct 24, 2024 •

edited by lwjohnst86

Loading

signekb left a comment •

edited

Loading