Store implementing Open - Edit - Save workflow #1453
Unanswered
marcel-goldschen-ohm
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
A common workflow for files is to read the file into memory, edit the contents, and then if you are satisfied save the results back to file and if not discard your changes without saving.
For a zarr hierarchy you can open whatever subtree with
root = zarr.open(path/to/root)
and then edit it as you please. However, every edit is immediately applied to the store. But what if you decide that you don't like your edits and you would like to discard them?You could use
zarr.copy
to copy a persistent group into an in-memory group, then edit that in-memory group, and when ready copy (overwrite) it back to the persistent group. For serial edits of this type this should suffice. However, when working with multiple parts distributed throughout a large heirarchy simultaneously (i.e., multiple traces/images meeting some condition in a dataset) this would require one to manually keep track of where all of these copied groups came from so they could be written back to the appropriate paths in the original persistent heirarchy. This seems like the sort of thing that would be nice if zarr could handle all that tracking for you.Thus, I feel that it would be useful to have a store that acts like a directory store (or similar) in that it only loads into memory the bits that are explicitly asked for, but all writes to the store are stored in memory and only "flushed" to the persistent storage system (e.g., file system) when a save() command is issued. Actually, as this behavior could be implemented for every non-memory store, it should probably be a feature of a store (i.e., toggle on-off this behavior). The writes should be limited to only those parts of the store that were edited.
The more that I think about this the more I feel that the tracking wouldn't be that hard for me to code manually. But then it might not be that hard to add as a feature in zarr either ;)
Beta Was this translation helpful? Give feedback.
All reactions