Skip to content

Commit

Permalink
Merge main
Browse files Browse the repository at this point in the history
  • Loading branch information
abarciauskas-bgse committed Feb 21, 2025
2 parents e9a4cce + f4bff96 commit daf8ad5
Show file tree
Hide file tree
Showing 10 changed files with 1,381 additions and 137 deletions.
4 changes: 2 additions & 2 deletions ci/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ dependencies:
- packaging
- universal_pathlib
- hdf5plugin
- numcodecs>=0.15.1
- icechunk>=0.1.1
- numcodecs
- icechunk>=0.2.0
# Testing
- codecov[toml]
- pre-commit
Expand Down
1 change: 0 additions & 1 deletion ci/upstream.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,5 @@ dependencies:
- pip
- zarr>=3.0.2
- pip:
- git+https://github.com/earth-mover/icechunk.git@main#subdirectory=icechunk-python # Installs zarr-python v3.0.0 as dependency
- git+https://github.com/fsspec/kerchunk.git@main
- imagecodecs-numcodecs==2024.6.1
2 changes: 1 addition & 1 deletion docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,5 +71,5 @@ Anyone with commit privileges to the repository can issue a release, and you sho
8. Edit the draft release notes for consistency.
9. Select 'Publish' to publish the release. This should automatically upload the new release to [PyPI](https://pypi.org/project/virtualizarr/) and [conda-forge](https://anaconda.org/conda-forge/virtualizarr).
10. Check that this has run successfully (PyPI should show the new version number very quickly, but conda-forge might take several hours).
11. Create and merge a PR to add a new empty section to the `docs/releases.rst` for the next release in the future.
11. Create and merge a PR to add a new empty section to the `docs/releases.rst` for the next release in the future. See [this commit](https://github.com/zarr-developers/VirtualiZarr/commit/e3912f08e22f2e3230af6eb1a2aacb5728822fa1) for an example (you can assume the next release will be numbered `vX.Y.Z+1`, but the number doesn't actually matter).
12. (Optional) Advertise the release on social media 📣
26 changes: 24 additions & 2 deletions docs/releases.rst
Original file line number Diff line number Diff line change
@@ -1,14 +1,36 @@
Release notes
=============
.. _v1.3.2:

v1.3.2 (unreleased)
-------------------

New Features
~~~~~~~~~~~~

Breaking changes
~~~~~~~~~~~~~~~~

Deprecations
~~~~~~~~~~~~

Bug fixes
~~~~~~~~~

Documentation
~~~~~~~~~~~~~

Internal Changes
~~~~~~~~~~~~~~~~

.. _v1.3.1:

v1.3.1 (unreleased)
v1.3.1 (18th Feb 2025)
-------------------

New Features
~~~~~~~~~~~~

- Examples use new Icechunk syntax
Breaking changes
~~~~~~~~~~~~~~~~

Expand Down
7 changes: 4 additions & 3 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -471,15 +471,15 @@ By default references are placed in separate parquet file when the total number

### Writing to an Icechunk Store

We can also write these references out as an [IcechunkStore](https://icechunk.io/). `Icechunk` is a Open-source, cloud-native transactional tensor storage engine that is compatible with zarr version 3. To export our virtual dataset to an `Icechunk` Store, we simply use the {py:meth}`vds.virtualize.to_icechunk <virtualizarr.VirtualiZarrDatasetAccessor.to_icechunk>` accessor method.
We can also write these references out as an [IcechunkStore](https://icechunk.io/). `Icechunk` is an open-source, cloud-native transactional tensor storage engine that is compatible with Zarr version 3. To export our virtual dataset to an `Icechunk` Store, we simply use the {py:meth}`vds.virtualize.to_icechunk <virtualizarr.VirtualiZarrDatasetAccessor.to_icechunk>` accessor method.

```python
# create an icechunk repository, session and write the virtual dataset to the session
from icechunk import Repository, Storage, VirtualChunkContainer, local_filesystem_storage
import icechunk
storage = local_filesystem_storage("./local/icechunk/store")

# By default, local virtual references and public remote virtual references can be read wihtout extra configuration.
repo = Repository.create(storage=storage)
repo = icechunk.Repository.create(storage)
session = repo.writeable_session("main")

# write the virtual dataset to the session with the IcechunkStore
Expand All @@ -500,6 +500,7 @@ session.commit("Appended second dataset")
```

See the [Icechunk documentation](https://icechunk.io/icechunk-python/virtual/#creating-a-virtual-dataset-with-virtualizarr) for more details.
icechunk-python/virtual/#creating-a-virtual-dataset-with-virtualizarr

## Opening Kerchunk references as virtual datasets

Expand Down
Loading

0 comments on commit daf8ad5

Please sign in to comment.