From 22ebded93aa88ae1e5f87f6711fa7057ca2e8478 Mon Sep 17 00:00:00 2001 From: David Stansby Date: Wed, 8 Jan 2025 17:38:43 +0000 Subject: [PATCH] Clean up release notes in preparation for v3 (#2634) --- docs/conf.py | 4 +- docs/developers/contributing.rst | 2 +- docs/developers/index.rst | 1 - docs/developers/release.rst | 2334 ------------------------------ docs/index.rst | 2 +- docs/release-notes.rst | 16 + docs/user-guide/v3_migration.rst | 2 + 7 files changed, 22 insertions(+), 2339 deletions(-) delete mode 100644 docs/developers/release.rst create mode 100644 docs/release-notes.rst diff --git a/docs/conf.py b/docs/conf.py index 8410b9b0b3..22d24c3515 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -105,10 +105,10 @@ def skip_submodules( "license": "https://github.com/zarr-developers/zarr-python/blob/main/LICENSE.txt", "tutorial": "user-guide", "getting-started": "quickstart", - "release": "developers/release.html", "roadmap": "developers/roadmap.html", "installation": "user-guide/installation.html", - "api": "api/zarr/index" + "api": "api/zarr/index", + "release": "release-notes" } # The language for content autogenerated by Sphinx. Refer to documentation diff --git a/docs/developers/contributing.rst b/docs/developers/contributing.rst index 4358230eff..31cf80bed6 100644 --- a/docs/developers/contributing.rst +++ b/docs/developers/contributing.rst @@ -213,7 +213,7 @@ and functions are included in the API documentation, under the ``docs/api`` fold using the `autodoc `_ extension to sphinx. Any new features or important usage information should be included in the user-guide (``docs/user-guide``). Any changes should also be included in the release -notes (``docs/developers/release.rst``). +notes (``docs/release-notes.rst``). The documentation can be built locally by running:: diff --git a/docs/developers/index.rst b/docs/developers/index.rst index 3feb0aff71..4bccb3a469 100644 --- a/docs/developers/index.rst +++ b/docs/developers/index.rst @@ -6,5 +6,4 @@ Developer's Guide :maxdepth: 1 contributing - release roadmap diff --git a/docs/developers/release.rst b/docs/developers/release.rst deleted file mode 100644 index ce15c68f4a..0000000000 --- a/docs/developers/release.rst +++ /dev/null @@ -1,2334 +0,0 @@ -Release notes -============= - -.. - # Copy the warning statement _under_ the latest release version - # and unindent for pre-releases. - - .. warning:: - Pre-release! Use :command:`pip install --pre zarr` to evaluate this release. - -.. - # Unindent the section between releases in order - # to document your changes. On releases it will be - # re-indented so that it does not show up in the notes. - -.. note:: - Zarr-Python 2.18.* is expected be the final release in the 2.* series. Work on Zarr-Python 3.0 is underway. - See `GH1777 `_ for more details on the upcoming - 3.0 release. - -.. release_3.0.0-beta: - -3.0.0-beta series ------------------ - -.. warning:: - Zarr-Python 3.0.0-beta is a pre-release of the upcoming 3.0 release. This release is not feature complete or - expected to be ready for production applications. - -.. note:: - The complete release notes for 3.0 have not been added to this document yet. See the - `3.0.0-beta `_ release on GitHub - for a record of changes included in this release. - -Dependency Changes -~~~~~~~~~~~~~~~~~~ - -* fsspec was moved from a required dependency to an optional one. Users should install - fsspec and any relevant implementations (e.g. s3fs) before using the ``RemoteStore``. - By :user:`Joe Hamman ` :issue:`2391`. - -* ``RemoteStore`` was renamed to ``FsspecStore``. - By :user:`Joe Hamman ` :issue:`2557`. - -.. release_3.0.0-alpha: - -3.0.0-alpha series ------------------- - -.. warning:: - Zarr-Python 3.0.0-alpha is a pre-release of the upcoming 3.0 release. This release is not feature complete or - expected to be ready for production applications. - -.. note:: - The complete release notes for 3.0 have not been added to this document yet. See the - `3.0.0-alpha `_ release on GitHub - for a record of changes included in this release. - -Enhancements -~~~~~~~~~~~~ - -* Implement listing of the sub-arrays and sub-groups for a V3 ``Group``. - By :user:`Davis Bennett ` :issue:`1726`. - -* Bootstrap v3 branch with zarrita. - By :user:`Joe Hamman ` :issue:`1584`. - -* Extensible codecs for V3. - By :user:`Norman Rzepka ` :issue:`1588`. - -* Don't import from tests. - By :user:`Davis Bennett ` :issue:`1601`. - -* Listable V3 Stores. - By :user:`Joe Hamman ` :issue:`1634`. - -* Codecs without array metadata. - By :user:`Norman Rzepka ` :issue:`1632`. - -* fix sync group class methods. - By :user:`Joe Hamman ` :issue:`1652`. - -* implement eq for LocalStore. - By :user:`Charoula Kyriakides ` :issue:`1792`. - -* V3 reorg. - By :user:`Joe Hamman ` :issue:`1809`. - -* [v3] Sync with futures. - By :user:`Davis Bennett ` :issue:`1804`. - -* implement group.members. - By :user:`Davis Bennett ` :issue:`1726`. - -* Remove implicit groups. - By :user:`Joe Hamman ` :issue:`1827`. - -* feature(store): ``list_*`` -> AsyncGenerators. - By :user:`Joe Hamman ` :issue:`1844`. - -* Test codec entrypoints. - By :user:`Norman Rzepka ` :issue:`1835`. - -* Remove extra v3 sync module. - By :user:`Max Jones ` :issue:`1856`. - -* Use donfig for V3 configuration. - By :user:`Max Jones ` :issue:`1655`. - -* groundwork for V3 group tests. - By :user:`Davis Bennett ` :issue:`1743`. - -* [v3] First step to generalizes ndarray and bytes. - By :user:`Mads R. B. Kristensen ` :issue:`1826`. - -* Reworked codec pipelines. - By :user:`Norman Rzepka ` :issue:`1670`. - -* Followup on codecs. - By :user:`Norman Rzepka ` :issue:`1889`. - -* Protocols for Buffer and NDBuffer. - By :user:`Mads R. B. Kristensen ` :issue:`1899`. - -* [V3] Expand store tests. - By :user:`Davis Bennett ` :issue:`1900`. - -* [v3] Feature: Store open mode. - By :user:`Joe Hamman ` :issue:`1911`. - -* fix(types): Group.info -> NotImplementedError. - By :user:`Joe Hamman ` :issue:`1936`. - -* feature(typing): add py.typed file to package root. - By :user:`Joe Hamman ` :issue:`1935`. - -* Support all indexing variants. - By :user:`Norman Rzepka ` :issue:`1917`. - -* Feature: group and array name properties. - By :user:`Joe Hamman ` :issue:`1940`. - -* implement .chunks on v3 arrays. - By :user:`Ryan Abernathey ` :issue:`1929`. - -* Fixes bug in transpose. - By :user:`Norman Rzepka ` :issue:`1949`. - -* Buffer Prototype Argument. - By :user:`Mads R. B. Kristensen ` :issue:`1910`. - -* Feature: Top level V3 API. - By :user:`Joe Hamman ` :issue:`1884`. - -* Basic working FsspecStore. - By :user:`Martin Durant `; :issue:`1785`. - -Typing -~~~~~~ - -* Resolve Mypy errors in v3 branch. - By :user:`Daniel Jahn ` :issue:`1692`. - -* Allow dmypy to be run on v3 branch. - By :user:`David Stansby ` :issue:`1780`. - -* Remove unused typing ignore comments. - By :user:`David Stansby ` :issue:`1781`. - -* Check untyped defs on v3. - By :user:`David Stansby ` :issue:`1784`. - -* [v3] Enable some more strict mypy options. - By :user:`David Stansby ` :issue:`1793`. - -* [v3] Disallow generic Any typing. - By :user:`David Stansby ` :issue:`1794`. - -* Disallow incomplete type definitions. - By :user:`David Stansby ` :issue:`1814`. - -* Disallow untyped calls. - By :user:`David Stansby ` :issue:`1811`. - -* Fix some untyped calls. - By :user:`David Stansby ` :issue:`1865`. - -* Disallow untyped defs. - By :user:`David Stansby ` :issue:`1834`. - -* Add more typing to zarr.group. - By :user:`David Stansby ` :issue:`1870`. - -* Fix any generics in zarr.array. - By :user:`David Stansby ` :issue:`1861`. - -* Remove some unused mypy overrides. - By :user:`David Stansby ` :issue:`1894`. - -* Finish typing zarr.metadata. - By :user:`David Stansby ` :issue:`1880`. - -* Disallow implicit re-exports. - By :user:`David Stansby ` :issue:`1908`. - -* Make typing strict. - By :user:`David Stansby ` :issue:`1879`. - -* Enable extra mypy error codes. - By :user:`David Stansby ` :issue:`1909`. - -* Enable warn_unreachable for mypy. - By :user:`David Stansby ` :issue:`1937`. - -* Fix final typing errors. - By :user:`David Stansby ` :issue:`1939`. - -Maintenance -~~~~~~~~~~~ - -* Remedy a situation where ``zarr-python`` was importing ``DummyStorageTransformer`` from the test suite. - The dependency relationship is now reversed: the test suite imports this class from ``zarr-python``. - By :user:`Davis Bennett ` :issue:`1601`. - -* [V3] Update minimum supported Python and Numpy versions. - By :user:`Joe Hamman ` :issue:`1638` - -* use src layout and use hatch for packaging. - By :user:`Davis Bennett ` :issue:`1592`. - -* temporarily disable mypy in v3 directory. - By :user:`Joe Hamman ` :issue:`1649`. - -* create hatch test env. - By :user:`Ryan Abernathey ` :issue:`1650`. - -* removed unused environments and workflows. - By :user:`Ryan Abernathey ` :issue:`1651`. - -* Add env variables to sprint setup instructions. - By :user:`Max Jones ` :issue:`1654`. - -* Add test matrix for V3. - By :user:`Max Jones ` :issue:`1656`. - -* Remove attrs. - By :user:`Davis Bennett ` :issue:`1660`. - -* Specify hatch envs using GitHub actions matrix for v3 tests. - By :user:`Max Jones ` :issue:`1728`. - -* black -> ruff format + cleanup. - By :user:`Saransh Chopra ` :issue:`1639`. - -* Remove old v3. - By :user:`Davis Bennett ` :issue:`1742`. - -* V3 update pre commit. - By :user:`Joe Hamman ` :issue:`1808`. - -* remove windows testing on v3 branch. - By :user:`Joe Hamman ` :issue:`1817`. - -* fix: add mypy to test dependencies. - By :user:`Davis Bennett ` :issue:`1789`. - -* chore(ci): add numpy 2 release candidate to test matrix. - By :user:`Joe Hamman ` :issue:`1828`. - -* fix dependencies. - By :user:`Norman Rzepka ` :issue:`1840`. - -* Add pytest to mypy dependencies. - By :user:`David Stansby ` :issue:`1846`. - -* chore(pre-commit): update pre-commit versions and remove attrs dep mypy section. - By :user:`Joe Hamman ` :issue:`1848`. - -* Enable some ruff rules (RUF) and fix issues. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1869`. - -* Configure Ruff to apply flake8-bugbear/isort/pyupgrade. - By :user:`Norman Rzepka ` :issue:`1890`. - -* chore(ci): remove mypy from test action in favor of pre-commit action. - By :user:`Joe Hamman ` :issue:`1887`. - -* Enable ruff/flake8-raise rules (RSE) and fix issues. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1872`. - -* Apply assorted ruff/refurb rules (FURB). - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1873`. - -* Enable ruff/flake8-implicit-str-concat rules (ISC) and fix issues. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1868`. - -* Add numpy to mypy pre-commit check env. - By :user:`David Stansby ` :issue:`1893`. - -* remove fixture files from src. - By :user:`Davis Bennett ` :issue:`1897`. - -* Fix list of packages in mypy pre-commit environment. - By :user:`David Stansby ` :issue:`1907`. - -* Run sphinx directly on readthedocs. - By :user:`David Stansby ` :issue:`1919`. - -* Apply preview ruff rules. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1942`. - -* Enable and apply ruff rule RUF009. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1941`. - -Documentation -~~~~~~~~~~~~~ - -* Specify docs hatch env for v3 branch. - By :user:`Max Jones ` :issue:`1655`. - -* Development installation/contributing docs updates. - By :user:`Alden Keefe Sampson ` :issue:`1643`. - -* chore: update project settings per scientific python repo-review. - By :user:`Joe Hamman ` :issue:`1863`. - -* doc: update release notes for 3.0.0.alpha. - By :user:`Joe Hamman ` :issue:`1959`. - -.. _release_2.18.3: - -2.18.3 ------- - -Enhancements -~~~~~~~~~~~~ -* Added support for creating a copy of data when converting a `zarr.Array` - to a numpy array. - By :user:`David Stansby ` (:issue:`2106`) and - :user:`Joe Hamman ` (:issue:`2123`). - -Maintenance -~~~~~~~~~~~ -* Removed support for Python 3.9. - By :user:`David Stansby ` (:issue:`2074`). - -* Fix a regression when using orthogonal indexing with a scalar. - By :user:`Deepak Cherian ` :issue:`1931` - -* Added compatibility with NumPy 2.1. - By :user:`David Stansby ` - -* Bump minimum NumPy version to 1.24. - :user:`Joe Hamman ` (:issue:`2127`). - -Deprecations -~~~~~~~~~~~~ - -* Deprecate :class:`zarr.n5.N5Store` and :class:`zarr.n5.N5FSStore`. These - stores are slated to be removed in Zarr Python 3.0. - By :user:`Joe Hamman ` :issue:`2085`. - -.. _release_2.18.2: - -2.18.2 ------- - -Enhancements -~~~~~~~~~~~~ - -* Add Zstd codec to old V3 code path. - By :user:`Ryan Abernathey ` - -.. _release_2.18.1: - -2.18.1 ------- - -Maintenance -~~~~~~~~~~~ -* Fix a regression when getting or setting a single value from arrays with size-1 chunks. - By :user:`Deepak Cherian ` :issue:`1874` - -.. _release_2.18.0: - -2.18.0 ------- - -Enhancements -~~~~~~~~~~~~ -* Performance improvement for reading and writing chunks if any of the dimensions is size 1. - By :user:`Deepak Cherian ` :issue:`1730`. - -Maintenance -~~~~~~~~~~~ -* Enable ruff/bugbear rules (B) and fix issues. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1702`. - -* Minor updates to use `np.inf` instead of `np.PINF` / `np.NINF` in preparation for NumPy 2.0.0 release. - By :user:`Joe Hamman ` :issue:`1842`. - -Deprecations -~~~~~~~~~~~~ - -* Deprecate experimental v3 support by issuing a `FutureWarning`. - Also updated docs to warn about using the experimental v3 version. - By :user:`Joe Hamman ` :issue:`1802` and :issue:`1807`. - -* Deprecate the following stores: :class:`zarr.storage.DBMStore`, :class:`zarr.storage.LMDBStore`, - :class:`zarr.storage.SQLiteStore`, :class:`zarr.storage.MongoDBStore`, :class:`zarr.storage.RedisStore`, - and :class:`zarr.storage.ABSStore`. These stores are slated to be removed from Zarr-Python in version 3.0. - By :user:`Joe Hamman ` :issue:`1801`. - -.. _release_2.17.2: - -2.17.2 ------- - -Enhancements -~~~~~~~~~~~~ - -* [v3] Dramatically reduce number of ``__contains__`` requests in favor of optimistically calling `__getitem__` - and handling any error that may arise. - By :user:`Deepak Cherian ` :issue:`1741`. - -* [v3] Reuse the downloaded array metadata when creating an ``Array``. - By :user:`Deepak Cherian ` :issue:`1734`. - -* Optimize ``Array.info`` so that it calls `getsize` only once. - By :user:`Deepak Cherian ` :issue:`1733`. - -* Override IPython ``_repr_*_`` methods to avoid expensive lookups against object stores. - By :user:`Deepak Cherian ` :issue:`1716`. - -* FSStore now raises rather than return bad data. - By :user:`Martin Durant ` and :user:`Ian Carroll ` :issue:`1604`. - -* Avoid redundant ``__contains__``. - By :user:`Deepak Cherian ` :issue:`1739`. - -Docs -~~~~ - -* Fix link to GCSMap in ``tutorial.rst``. - By :user:`Daniel Jahn ` :issue:`1689`. - -* Endorse `SPEC0000 `_ and state version support policy in ``installation.rst``. - By :user:`Sanket Verma ` :issue:`1665`. - -* Migrate v1 and v2 specification to `Zarr-Specs `_. - By :user:`Sanket Verma ` :issue:`1582`. - -Maintenance -~~~~~~~~~~~ - -* Add CI test environment for Python 3.12 - By :user:`Joe Hamman ` :issue:`1719`. - -* Bump minimum supported NumPy version to 1.23 (per spec 0000) - By :user:`Joe Hamman ` :issue:`1719`. - -* Minor fixes: Using ``is`` instead of ``type`` and removing unnecessary ``None``. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1737`. - -* Fix tests failure related to Pytest 8. - By :user:`David Stansby ` :issue:`1714`. - -.. _release_2.17.1: - -2.17.1 ------- - -Enhancements -~~~~~~~~~~~~ - -* Change occurrences of % and format() to f-strings. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1423`. - -* Proper argument for numpy.reshape. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1425`. - -* Add typing to dimension separator arguments. - By :user:`David Stansby ` :issue:`1620`. - -Docs -~~~~ - -* ZIP related tweaks. - By :user:`Davis Bennett ` :issue:`1641`. - -Maintenance -~~~~~~~~~~~ - -* Update config.yml with Zulip. - By :user:`Josh Moore `. - -* Replace Gitter with the new Zulip Chat link. - By :user:`Sanket Verma ` :issue:`1685`. - -* Fix RTD build. - By :user:`Sanket Verma ` :issue:`1694`. - -.. _release_2.17.0: - -2.17.0 ------- - -Enhancements -~~~~~~~~~~~~ - -* Added type hints to ``zarr.creation.create()``. - By :user:`David Stansby ` :issue:`1536`. - -* Pyodide support: Don't require fasteners on Emscripten. - By :user:`Hood Chatham ` :issue:`1663`. - -Docs -~~~~ - -* Minor correction and changes in documentation. - By :user:`Sanket Verma ` :issue:`1509`. - -* Fix typo in documentation. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1554` - -* The documentation build now fails if there are any warnings. - By :user:`David Stansby ` :issue:`1548`. - -* Add links to ``numcodecs`` docs in the tutorial. - By :user:`David Stansby ` :issue:`1535`. - -* Enable offline formats for documentation builds. - By :user:`Sanket Verma ` :issue:`1551`. - -* Minor tweak to advanced indexing tutorial examples. - By :user:`Ross Barnowski ` :issue:`1550`. - -* Automatically document array members using sphinx-automodapi. - By :user:`David Stansby ` :issue:`1547`. - -* Add a markdown file documenting the current and former core-developer team. - By :user:`Joe Hamman ` :issue:`1628`. - -* Add Norman Rzepka to core-dev team. - By :user:`Joe Hamman ` :issue:`1630`. - -* Added section about accessing ZIP archives on s3. - By :user:`Jeff Peck ` :issue:`1613`, :issue:`1615`, and :user:`Davis Bennett ` :issue:`1641`. - -* Add V3 roadmap and design document. - By :user:`Joe Hamman ` :issue:`1583`. - -Maintenance -~~~~~~~~~~~ - -* Drop Python 3.8 and NumPy 1.20 - By :user:`Josh Moore `; :issue:`1557`. - -* Cache result of ``FSStore._fsspec_installed()``. - By :user:`Janick Martinez Esturo ` :issue:`1581`. - -* Extend copyright notice to 2023. - By :user:`Jack Kelly ` :issue:`1528`. - -* Change occurrence of ``io.open()`` into ``open()``. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1421`. - -* Preserve ``dimension_separator`` when resizing arrays. - By :user:`Ziwen Liu ` :issue:`1533`. - -* Initialise some sets in tests with set literals instead of list literals. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1534`. - -* Allow ``black`` code formatter to be run with any Python version. - By :user:`David Stansby ` :issue:`1549`. - -* Remove ``sphinx-rtd-theme`` dependency from ``pyproject.toml``. - By :user:`Sanket Verma ` :issue:`1563`. - -* Remove ``CODE_OF_CONDUCT.md`` file from the Zarr-Python repository. - By :user:`Sanket Verma ` :issue:`1572`. - -* Bump version of black in pre-commit. - By :user:`David Stansby ` :issue:`1559`. - -* Use list comprehension where applicable. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1555`. - -* Use format specification mini-language to format string. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1558`. - -* Single startswith() call instead of multiple ones. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1556`. - -* Move codespell options around. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1196`. - -* Remove unused mypy ignore comments. - By :user:`David Stansby ` :issue:`1602`. - -.. _release_2.16.1: - -2.16.1 ------- - -Maintenance -~~~~~~~~~~~ - -* Require ``setuptools_scm`` version ``1.5.4``\+ - By :user:`John A. Kirkham ` :issue:`1477`. - -* Add ``docs`` requirements to ``pyproject.toml`` - By :user:`John A. Kirkham ` :issue:`1494`. - -* Fixed caching issue in ``LRUStoreCache``. - By :user:`Mads R. B. Kristensen ` :issue:`1499`. - -.. _release_2.16.0: - -2.16.0 ------- - -Enhancements -~~~~~~~~~~~~ - -* Allow for partial codec specification in V3 array metadata. - By :user:`Joe Hamman ` :issue:`1443`. - -* Add ``__contains__`` method to ``KVStore``. - By :user:`Christoph Gohlke ` :issue:`1454`. - -* **Block Indexing**: Implemented blockwise (chunk blocks) indexing to ``zarr.Array``. - By :user:`Altay Sansal ` :issue:`1428` - -Maintenance -~~~~~~~~~~~ - -* Refactor the core array tests to reduce code duplication. - By :user:`Davis Bennett ` :issue:`1462`. - -* Style the codebase with ``ruff`` and ``black``. - By :user:`Davis Bennett ` :issue:`1459` - -* Ensure that chunks is tuple of ints upon array creation. - By :user:`Philipp Hanslovsky ` :issue:`1461` - -.. _release_2.15.0: - -2.15.0 ------- - -Enhancements -~~~~~~~~~~~~ - -* Implement more extensive fallback of getitem/setitem for orthogonal indexing. - By :user:`Andreas Albert ` :issue:`1029`. - -* Getitems supports ``meta_array``. - By :user:`Mads R. B. Kristensen ` :issue:`1131`. - -* ``open_array()`` now takes the ``meta_array`` argument. - By :user:`Mads R. B. Kristensen ` :issue:`1396`. - -Maintenance -~~~~~~~~~~~ - -* Remove ``codecov`` from GitHub actions. - By :user:`John A. Kirkham ` :issue:`1391`. - -* Replace ``np.product`` with ``np.prod`` due to deprecation. - By :user:`James Bourbeau ` :issue:`1405`. - -* Activate Py 3.11 builds. - By :user:`Joe Hamman ` :issue:`1415`. - -Documentation -~~~~~~~~~~~~~ - -* Add API reference for V3 Implementation in the docs. - By :user:`Sanket Verma ` :issue:`1345`. - -Bug fixes -~~~~~~~~~ - -* Fix the conda-forge error. Read :issue:`1347` for detailed info. - By :user:`Josh Moore ` :issue:`1364` and :issue:`1367`. - -* Fix ``ReadOnlyError`` when opening V3 store via fsspec reference file system. - By :user:`Joe Hamman ` :issue:`1383`. - -* Fix ``normalize_fill_value`` for structured arrays. - By :user:`Alan Du ` :issue:`1397`. - -.. _release_2.14.2: - -2.14.2 ------- - -Bug fixes -~~~~~~~~~ - -* Ensure ``zarr.group`` uses writeable mode to fix issue with :issue:`1304`. - By :user:`Brandur Thorgrimsson ` :issue:`1354`. - -.. _release_2.14.1: - -2.14.1 ------- - -Documentation -~~~~~~~~~~~~~ - -* Fix API links. - By :user:`Josh Moore ` :issue:`1346`. - -* Fix unit tests which prevented the conda-forge release. - By :user:`Josh Moore ` :issue:`1348`. - -.. _release_2.14.0: - -2.14.0 ------- - -Major changes -~~~~~~~~~~~~~ - -* Improve Zarr V3 support, adding partial store read/write and storage transformers. - Add new features from the `v3 spec `_: - - * storage transformers - * `get_partial_values` and `set_partial_values` - * efficient `get_partial_values` implementation for `FSStoreV3` - * sharding storage transformer - - By :user:`Jonathan Striebel `; :issue:`1096`, :issue:`1111`. - -* N5 nows supports Blosc. - Remove warnings emitted when using N5Store or N5FSStore with a blosc-compressed array. - By :user:`Davis Bennett `; :issue:`1331`. - -Bug fixes -~~~~~~~~~ - -* Allow reading utf-8 encoded json files - By :user:`Nathan Zimmerberg ` :issue:`1308`. - -* Ensure contiguous data is give to ``FSStore``. Only copying if needed. - By :user:`Mads R. B. Kristensen ` :issue:`1285`. - -* NestedDirectoryStore.listdir now returns chunk keys with the correct '/' dimension_separator. - By :user:`Brett Graham ` :issue:`1334`. - -* N5Store/N5FSStore dtype returns zarr Stores readable dtype. - By :user:`Marwan Zouinkhi ` :issue:`1339`. - -.. _release_2.13.6: - -2.13.6 ------- - -Maintenance -~~~~~~~~~~~ - -* Bump gh-action-pypi-publish to 1.6.4. - By :user:`Josh Moore ` :issue:`1320`. - -.. _release_2.13.5: - -2.13.5 ------- - -Bug fixes -~~~~~~~~~ - -* Ensure ``zarr.create`` uses writeable mode to fix issue with :issue:`1304`. - By :user:`James Bourbeau ` :issue:`1309`. - -.. _release_2.13.4: - -2.13.4 ------- - -Appreciation -~~~~~~~~~~~~~ - -Special thanks to Outreachy participants for contributing to most of the -maintenance PRs. Please read the blog post summarising the contribution phase -and welcoming new Outreachy interns: -https://zarr.dev/blog/welcoming-outreachy-2022-interns/ - - -Enhancements -~~~~~~~~~~~~ - -* Handle fsspec.FSMap using FSStore store. - By :user:`Rafal Wojdyla ` :issue:`1304`. - -Bug fixes -~~~~~~~~~ - -* Fix bug that caused double counting of groups in ``groups()`` and ``group_keys()`` methods with V3 stores. - By :user:`Ryan Abernathey ` :issue:`1228`. - -* Remove unnecessary calling of `contains_array` for key that ended in `.array.json`. - By :user:`Joe Hamman ` :issue:`1149`. - -* Fix bug that caused double counting of groups in ``groups()`` and ``group_keys()`` - methods with V3 stores. - By :user:`Ryan Abernathey ` :issue:`1228`. - -Documentation -~~~~~~~~~~~~~ - -* Fix minor indexing errors in tutorial and specification examples of documentation. - By :user:`Kola Babalola ` :issue:`1277`. - -* Add `requirements_rtfd.txt` in `contributing.rst`. - By :user:`AWA BRANDON AWA ` :issue:`1243`. - -* Add documentation for find/findall using visit. - By :user:`Weddy Gikunda ` :issue:`1241`. - -* Refresh of the main landing page. - By :user:`Josh Moore ` :issue:`1173`. - -Maintenance -~~~~~~~~~~~ - -* Migrate to ``pyproject.toml`` and remove redundant infrastructure. - By :user:`Saransh Chopra ` :issue:`1158`. - -* Require ``setuptools`` 64.0.0+ - By :user:`Saransh Chopra ` :issue:`1193`. - -* Pin action versions (pypi-publish, setup-miniconda) for dependabot - By :user:`Saransh Chopra ` :issue:`1205`. - -* Remove ``tox`` support - By :user:`Saransh Chopra ` :issue:`1219`. - -* Add workflow to label PRs with "needs release notes". - By :user:`Saransh Chopra ` :issue:`1239`. - -* Simplify if/else statement. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1227`. - -* Get coverage up to 100%. - By :user:`John Kirkham ` :issue:`1264`. - -* Migrate coverage to ``pyproject.toml``. - By :user:`John Kirkham ` :issue:`1250`. - -* Use ``conda-incubator/setup-miniconda@v2.2.0``. - By :user:`John Kirkham ` :issue:`1263`. - -* Delete unused files. - By :user:`John Kirkham ` :issue:`1251`. - -* Skip labeller for bot PRs. - By :user:`Saransh Chopra ` :issue:`1271`. - -* Restore Flake8 configuration. - By :user:`John Kirkham ` :issue:`1249`. - -* Add missing newline at EOF. - By :user:`Dimitri Papadopoulos` :issue:`1253`. - -* Add `license_files` to `pyproject.toml`. - By :user:`John Kirkham ` :issue:`1247`. - -* Adding `pyupgrade` suggestions. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1225`. - -* Fixed some linting errors. - By :user:`Weddy Gikunda ` :issue:`1226`. - -* Added the link to main website in readthedocs sidebar. - By :user:`Stephanie_nkwatoh ` :issue:`1216`. - -* Remove redundant wheel dependency in `pyproject.toml`. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1233`. - -* Turned on `isloated_build` in `tox.ini` file. - By :user:`AWA BRANDON AWA ` :issue:`1210`. - -* Fixed `flake8` alert and avoid duplication of `Zarr Developers`. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1203`. - -* Bump to NumPy 1.20+ in `environment.yml`. - By :user:`John Kirkham ` :issue:`1201`. - -* Bump to NumPy 1.20 in `pyproject.toml`. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1192`. - -* Remove LGTM (`.lgtm.yml`) configuration file. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1191`. - -* Codespell will skip `fixture` in pre-commit. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1197`. - -* Add msgpack in `requirements_rtfd.txt`. - By :user:`Emmanuel Bolarinwa ` :issue:`1188`. - -* Added license to docs fixed a typo from `_spec_v2` to `_spec_v3`. - By :user:`AWA BRANDON AWA ` :issue:`1182`. - -* Fixed installation link in `README.md`. - By :user:`AWA BRANDON AWA ` :issue:`1177`. - -* Fixed typos in `installation.rst` and `release.rst`. - By :user:`Chizoba Nweke ` :issue:`1178`. - -* Set `docs/conf.py` language to `en`. - By :user:`AWA BRANDON AWA ` :issue:`1174`. - -* Added `installation.rst` to the docs. - By :user:`AWA BRANDON AWA ` :issue:`1170`. - -* Adjustment of year to `2015-2018` to `2015-2022` in the docs. - By :user:`Emmanuel Bolarinwa ` :issue:`1165`. - -* Updated `Forking the repository` section in `contributing.rst`. - By :user:`AWA BRANDON AWA ` :issue:`1171`. - -* Updated GitHub actions. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1134`. - -* Update web links: `http:// → https://`. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1313`. - -.. _release_2.13.3: - -2.13.3 ------- - -* Improve performance of slice selections with steps by omitting chunks with no relevant - data. - By :user:`Richard Shaw ` :issue:`843`. - -.. _release_2.13.2: - -2.13.2 ------- - -* Fix test failure on conda-forge builds (again). - By :user:`Josh Moore `; see - `zarr-feedstock#65 `_. - -.. _release_2.13.1: - -2.13.1 ------- - -* Fix test failure on conda-forge builds. - By :user:`Josh Moore `; see - `zarr-feedstock#65 `_. - -.. _release_2.13.0: - -2.13.0 ------- - -Major changes -~~~~~~~~~~~~~ - -* **Support of alternative array classes** by introducing a new argument, - meta_array, that specifies the type/class of the underlying array. The - meta_array argument can be any class instance that can be used as the like - argument in NumPy (see `NEP 35 - `_). - enabling support for CuPy through, for example, the creation of a CuPy CPU - compressor. - By :user:`Mads R. B. Kristensen ` :issue:`934`. - -* **Remove support for Python 3.7** in concert with NumPy dependency. - By :user:`Davis Bennett ` :issue:`1067`. - -* **Zarr v3: add support for the default root path** rather than requiring - that all API users pass an explicit path. - By :user:`Gregory R. Lee ` :issue:`1085`, :issue:`1142`. - - -Bug fixes -~~~~~~~~~ - -* Remove/relax erroneous "meta" path check (**regression**). - By :user:`Gregory R. Lee ` :issue:`1123`. - -* Cast all attribute keys to strings (and issue deprecation warning). - By :user:`Mattia Almansi ` :issue:`1066`. - -* Fix bug in N5 storage that prevented arrays located in the root of the hierarchy from - bearing the `n5` keyword. Along with fixing this bug, new tests were added for N5 routines - that had previously been excluded from testing, and type annotations were added to the N5 codebase. - By :user:`Davis Bennett ` :issue:`1092`. - -* Fix bug in LRUEStoreCache in which the current size wasn't reset on invalidation. - By :user:`BGCMHou ` and :user:`Josh Moore ` :issue:`1076`, :issue:`1077`. - -* Remove erroneous check that disallowed array keys starting with "meta". - By :user:`Gregory R. Lee ` :issue:`1105`. - -Documentation -~~~~~~~~~~~~~ - -* Typo fixes to close quotes. By :user:`Pavithra Eswaramoorthy ` - -* Added copy button to documentation. - By :user:`Altay Sansal ` :issue:`1124`. - -Maintenance -~~~~~~~~~~~ - -* Simplify release docs. - By :user:`Josh Moore ` :issue:`1119`. - -* Pin werkzeug to prevent test hangs. - By :user:`Davis Bennett ` :issue:`1098`. - -* Fix a few DeepSource.io alerts - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`1080`. - -* Fix URLs. - By :user:`Dimitri Papadopoulos Orfanos `, :issue:`1074`. - -* Fix spelling. - By :user:`Dimitri Papadopoulos Orfanos `, :issue:`1073`. - -* Update GitHub issue templates with `YAML` format. - By :user:`Saransh Chopra ` :issue:`1079`. - -* Remove option to return None from _ensure_store. - By :user:`Gregory Lee ` :issue:`1068`. - -* Fix a typo of "integers". - By :user:`Richard Scott ` :issue:`1056`. - -.. _release_2.12.0: - -2.12.0 ------- - -Enhancements -~~~~~~~~~~~~ - -* **Add support for reading and writing Zarr V3.** The new `zarr._store.v3` - package has the necessary classes and functions for evaluating Zarr V3. - Since the format is not yet finalized, the classes and functions are not - automatically imported into the regular `zarr` name space. Setting the - `ZARR_V3_EXPERIMENTAL_API` environment variable will activate them. - By :user:`Gregory Lee `; :issue:`898`, :issue:`1006`, and :issue:`1007` - as well as by :user:`Josh Moore ` :issue:`1032`. - -* **Create FSStore from an existing fsspec filesystem**. If you have created - an fsspec filesystem outside of Zarr, you can now pass it as a keyword - argument to ``FSStore``. - By :user:`Ryan Abernathey `; :issue:`911`. - -* Add numpy encoder class for json.dumps - By :user:`Eric Prestat `; :issue:`933`. - -* Appending performance improvement to Zarr arrays, e.g., when writing to S3. - By :user:`hailiangzhang `; :issue:`1014`. - -* Add number encoder for ``json.dumps`` to support numpy integers in - ``chunks`` arguments. By :user:`Eric Prestat ` :issue:`697`. - -Bug fixes -~~~~~~~~~ - -* Fix bug that made it impossible to create an ``FSStore`` on unlistable filesystems - (e.g. some HTTP servers). - By :user:`Ryan Abernathey `; :issue:`993`. - - -Documentation -~~~~~~~~~~~~~ - -* Update resize doc to clarify surprising behavior. - By :user:`hailiangzhang `; :issue:`1022`. - -Maintenance -~~~~~~~~~~~ - -* Added Pre-commit configuration, incl. Yaml Check. - By :user:`Shivank Chaudhary `; :issue:`1015`, :issue:`1016`. - -* Fix URL to renamed file in Blosc repo. - By :user:`Andrew Thomas ` :issue:`1028`. - -* Activate Py 3.10 builds. - By :user:`Josh Moore ` :issue:`1027`. - -* Make all unignored zarr warnings errors. - By :user:`Josh Moore ` :issue:`1021`. - - -.. _release_2.11.3: - -2.11.3 ------- - -Bug fixes -~~~~~~~~~ - -* Fix missing case to fully revert change to default write_empty_chunks. - By :user:`Tom White `; :issue:`1005`. - - -.. _release_2.11.2: - -2.11.2 ------- - -Bug fixes -~~~~~~~~~ - -* Changes the default value of ``write_empty_chunks`` to ``True`` to prevent - unanticipated data losses when the data types do not have a proper default - value when empty chunks are read back in. - By :user:`Vyas Ramasubramani `; :issue:`965`, :issue:`1001`. - -.. _release_2.11.1: - -2.11.1 ------- - -Bug fixes -~~~~~~~~~ - -* Fix bug where indexing with a scalar numpy value returned a single-value array. - By :user:`Ben Jeffery ` :issue:`967`. - -* Removed `clobber` argument from `normalize_store_arg`. This enables to change - data within an opened consolidated group using mode `"r+"` (i.e region write). - By :user:`Tobias Kölling ` :issue:`975`. - -.. _release_2.11.0: - -2.11.0 ------- - -Enhancements -~~~~~~~~~~~~ - -* **Sparse changes with performance impact!** One of the advantages of the Zarr - format is that it is sparse, which means that chunks with no data (more - precisely, with data equal to the fill value, which is usually 0) don't need - to be written to disk at all. They will simply be assumed to be empty at read - time. However, until this release, the Zarr library would write these empty - chunks to disk anyway. This changes in this version: a small performance - penalty at write time leads to significant speedups at read time and in - filesystem operations in the case of sparse arrays. To revert to the old - behavior, pass the argument ``write_empty_chunks=True`` to the array creation - function. By :user:`Juan Nunez-Iglesias `; :issue:`853` and - :user:`Davis Bennett `; :issue:`738`. - -* **Fancy indexing**. Zarr arrays now support NumPy-style fancy indexing with - arrays of integer coordinates. This is equivalent to using zarr.Array.vindex. - Mixing slices and integer arrays is not supported. - By :user:`Juan Nunez-Iglesias `; :issue:`725`. - -* **New base class**. This release of Zarr Python introduces a new - ``BaseStore`` class that all provided store classes implemented in Zarr - Python now inherit from. This is done as part of refactoring to enable future - support of the Zarr version 3 spec. Existing third-party stores that are a - MutableMapping (e.g. dict) can be converted to a new-style key/value store - inheriting from ``BaseStore`` by passing them as the argument to the new - ``zarr.storage.KVStore`` class. For backwards compatibility, various - higher-level array creation and convenience functions still accept plain - Python dicts or other mutable mappings for the ``store`` argument, but will - internally convert these to a ``KVStore``. - By :user:`Gregory Lee `; :issue:`839`, :issue:`789`, and :issue:`950`. - -* Allow to assign array ``fill_values`` and update metadata accordingly. - By :user:`Ryan Abernathey `, :issue:`662`. - -* Allow to update array fill_values - By :user:`Matthias Bussonnier ` :issue:`665`. - -Bug fixes -~~~~~~~~~ - -* Fix bug where the checksum of zipfiles is wrong - By :user:`Oren Watson ` :issue:`930`. - -* Fix consolidate_metadata with FSStore. - By :user:`Joe Hamman ` :issue:`916`. - -* Unguarded next inside generator. - By :user:`Dimitri Papadopoulos Orfanos ` :issue:`889`. - -Documentation -~~~~~~~~~~~~~ - -* Update docs creation of dev env. - By :user:`Ray Bell ` :issue:`921`. - -* Update docs to use ``python -m pytest``. - By :user:`Ray Bell ` :issue:`923`. - -* Fix versionadded tag in zarr.Array docstring. - By :user:`Juan Nunez-Iglesias ` :issue:`852`. - -* Doctest seem to be stricter now, updating tostring() to tobytes(). - By :user:`John Kirkham ` :issue:`907`. - -* Minor doc fix. - By :user:`Mads R. B. Kristensen ` :issue:`937`. - -Maintenance -~~~~~~~~~~~ - -* Upgrade MongoDB in test env. - By :user:`Joe Hamman ` :issue:`939`. - -* Pass dimension_separator on fixture generation. - By :user:`Josh Moore ` :issue:`858`. - -* Activate Python 3.9 in GitHub Actions. - By :user:`Josh Moore ` :issue:`859`. - -* Drop shortcut ``fsspec[s3]`` for dependency. - By :user:`Josh Moore ` :issue:`920`. - -* and a swath of code-linting improvements by :user:`Dimitri Papadopoulos Orfanos `: - - - Unnecessary comprehension (:issue:`899`) - - - Unnecessary ``None`` provided as default (:issue:`900`) - - - use an if ``expression`` instead of `and`/`or` (:issue:`888`) - - - Remove unnecessary literal (:issue:`891`) - - - Decorate a few method with `@staticmethod` (:issue:`885`) - - - Drop unneeded ``return`` (:issue:`884`) - - - Drop explicit ``object`` inheritance from ``class``-es (:issue:`886`) - - - Unnecessary comprehension (:issue:`883`) - - - Codespell configuration (:issue:`882`) - - - Fix typos found by codespell (:issue:`880`) - - - Proper C-style formatting for integer (:issue:`913`) - - - Add LGTM.com / DeepSource.io configuration files (:issue:`909`) - -.. _release_2.10.3: - -2.10.3 ------- - -Bug fixes -~~~~~~~~~ - -* N5 keywords now emit UserWarning instead of raising a ValueError. - By :user:`Boaz Mohar `; :issue:`860`. - -* blocks_to_decompress not used in read_part function. - By :user:`Boaz Mohar `; :issue:`861`. - -* defines blocksize for array, updates hexdigest values. - By :user:`Andrew Fulton `; :issue:`867`. - -* Fix test failure on Debian and conda-forge builds. - By :user:`Josh Moore `; :issue:`871`. - -.. _release_2.10.2: - -2.10.2 ------- - -Bug fixes -~~~~~~~~~ - -* Fix NestedDirectoryStore datasets without dimension_separator metadata. - By :user:`Josh Moore `; :issue:`850`. - -.. _release_2.10.1: - -2.10.1 ------- - -Bug fixes -~~~~~~~~~ - -* Fix regression by setting normalize_keys=False in fsstore constructor. - By :user:`Davis Bennett `; :issue:`842`. - -.. _release_2.10.0: - -2.10.0 ------- - -Enhancements -~~~~~~~~~~~~ - -* Add N5FSStore. - By :user:`Davis Bennett `; :issue:`793`. - -Bug fixes -~~~~~~~~~ - -* Ignore None dim_separators in save_array. - By :user:`Josh Moore `; :issue:`831`. - -.. _release_2.9.5: - -2.9.5 ------ - -Bug fixes -~~~~~~~~~ - -* Fix FSStore.listdir behavior for nested directories. - By :user:`Gregory Lee `; :issue:`802`. - -.. _release_2.9.4: - -2.9.4 ------ - -Bug fixes -~~~~~~~~~ - -* Fix structured arrays that contain objects - By :user: `Attila Bergou `; :issue: `806` - -.. _release_2.9.3: - -2.9.3 ------ - -Maintenance -~~~~~~~~~~~ - -* Mark the fact that some tests that require ``fsspec``, without compromising the code coverage score. - By :user:`Ben Williams `; :issue:`823`. - -* Only inspect alternate node type if desired isn't present. - By :user:`Trevor Manz `; :issue:`696`. - -.. _release_2.9.2: - -2.9.2 ------ - -Maintenance -~~~~~~~~~~~ - -* Correct conda-forge deployment of Zarr by fixing some Zarr tests. - By :user:`Ben Williams `; :issue:`821`. - -.. _release_2.9.1: - -2.9.1 ------ - -Maintenance -~~~~~~~~~~~ - -* Correct conda-forge deployment of Zarr. - By :user:`Josh Moore `; :issue:`819`. - -.. _release_2.9.0: - -2.9.0 ------ - -This release of Zarr Python is the first release of Zarr to not support Python 3.6. - -Enhancements -~~~~~~~~~~~~ - -* Update ABSStore for compatibility with newer `azure.storage.blob`. - By :user:`Tom Augspurger `; :issue:`759`. - -* Pathlib support. - By :user:`Chris Barnes `; :issue:`768`. - -Documentation -~~~~~~~~~~~~~ - -* Clarify that arbitrary key/value pairs are OK for attributes. - By :user:`Stephan Hoyer `; :issue:`751`. - -* Clarify how to manually convert a DirectoryStore to a ZipStore. - By :user:`pmav99 `; :issue:`763`. - -Bug fixes -~~~~~~~~~ - -* Fix dimension_separator support. - By :user:`Josh Moore `; :issue:`775`. - -* Extract ABSStore to zarr._storage.absstore. - By :user:`Josh Moore `; :issue:`781`. - -* avoid NumPy 1.21.0 due to https://github.com/numpy/numpy/issues/19325 - By :user:`Gregory Lee `; :issue:`791`. - -Maintenance -~~~~~~~~~~~ - -* Drop 3.6 builds. - By :user:`Josh Moore `; :issue:`774`, :issue:`778`. - -* Fix build with Sphinx 4. - By :user:`Elliott Sales de Andrade `; :issue:`799`. - -* TST: add missing assert in test_hexdigest. - By :user:`Gregory Lee `; :issue:`801`. - -.. _release_2.8.3: - -2.8.3 ------ - -Bug fixes -~~~~~~~~~ - -* FSStore: default to normalize_keys=False - By :user:`Josh Moore `; :issue:`755`. -* ABSStore: compatibility with ``azure.storage.python>=12`` - By :user:`Tom Augspurger `; :issue:`618` - - -.. _release_2.8.2: - -2.8.2 ------ - -Documentation -~~~~~~~~~~~~~ - -* Add section on rechunking to tutorial - By :user:`David Baddeley `; :issue:`730`. - -Bug fixes -~~~~~~~~~ - -* Expand FSStore tests and fix implementation issues - By :user:`Davis Bennett `; :issue:`709`. - -Maintenance -~~~~~~~~~~~ - -* Updated ipytree warning for jlab3 - By :user:`Ian Hunt-Isaak `; :issue:`721`. - -* b170a48a - (issue-728, copy-nested) Updated ipytree warning for jlab3 (#721) (3 weeks ago) -* Activate dependabot - By :user:`Josh Moore `; :issue:`734`. - -* Update Python classifiers (Zarr is stable!) - By :user:`Josh Moore `; :issue:`731`. - -.. _release_2.8.1: - -2.8.1 ------ - -Bug fixes -~~~~~~~~~ - -* raise an error if create_dataset's dimension_separator is inconsistent - By :user:`Gregory R. Lee `; :issue:`724`. - -.. _release_2.8.0: - -2.8.0 ------ - -V2 Specification Update -~~~~~~~~~~~~~~~~~~~~~~~ - -* Introduce optional dimension_separator .zarray key for nested chunks. - By :user:`Josh Moore `; :issue:`715`, :issue:`716`. - -.. _release_2.7.1: - -2.7.1 ------ - -Bug fixes -~~~~~~~~~ - -* Update Array to respect FSStore's key_separator (#718) - By :user:`Gregory R. Lee `; :issue:`718`. - -.. _release_2.7.0: - -2.7.0 ------ - -Enhancements -~~~~~~~~~~~~ - -* Start stop for iterator (`islice()`) - By :user:`Sebastian Grill `; :issue:`621`. - -* Add capability to partially read and decompress chunks - By :user:`Andrew Fulton `; :issue:`667`. - -Bug fixes -~~~~~~~~~ - -* Make DirectoryStore __setitem__ resilient against antivirus file locking - By :user:`Eric Younkin `; :issue:`698`. - -* Compare test data's content generally - By :user:`John Kirkham `; :issue:`436`. - -* Fix dtype usage in zarr/meta.py - By :user:`Josh Moore `; :issue:`700`. - -* Fix FSStore key_seperator usage - By :user:`Josh Moore `; :issue:`669`. - -* Simplify text handling in DB Store - By :user:`John Kirkham `; :issue:`670`. - -* GitHub Actions migration - By :user:`Matthias Bussonnier `; - :issue:`641`, :issue:`671`, :issue:`674`, :issue:`676`, :issue:`677`, :issue:`678`, - :issue:`679`, :issue:`680`, :issue:`682`, :issue:`684`, :issue:`685`, :issue:`686`, - :issue:`687`, :issue:`695`, :issue:`706`. - -.. _release_2.6.1: - -2.6.1 ------ - -* Minor build fix - By :user:`Matthias Bussonnier `; :issue:`666`. - -.. _release_2.6.0: - -2.6.0 ------ - -This release of Zarr Python is the first release of Zarr to not support Python 3.5. - -* End Python 3.5 support. - By :user:`Chris Barnes `; :issue:`602`. - -* Fix ``open_group/open_array`` to allow opening of read-only store with - ``mode='r'`` :issue:`269` - -* Add `Array` tests for FSStore. - By :user:`Andrew Fulton `; :issue: `644`. - -* fix a bug in which ``attrs`` would not be copied on the root when using ``copy_all``; :issue:`613` - -* Fix ``FileNotFoundError`` with dask/s3fs :issue:`649` - -* Fix flaky fixture in test_storage.py :issue:`652` - -* Fix FSStore getitems fails with arrays that have a 0 length shape dimension :issue:`644` - -* Use async to fetch/write result concurrently when possible. :issue:`536`, See `this comment - `_ for some performance analysis - showing order of magnitude faster response in some benchmark. - -See `this link `_ -for the full list of closed and merged PR tagged with the 2.6 milestone. - -* Add ability to partially read and decompress arrays, see :issue:`667`. It is - only available to chunks stored using fsspec and using Blosc as a compressor. - - For certain analysis case when only a small portion of chunks is needed it can - be advantageous to only access and decompress part of the chunks. Doing - partial read and decompression add high latency to many of the operation so - should be used only when the subset of the data is small compared to the full - chunks and is stored contiguously (that is to say either last dimensions for C - layout, firsts for F). Pass ``partial_decompress=True`` as argument when - creating an ``Array``, or when using ``open_array``. No option exists yet to - apply partial read and decompress on a per-operation basis. - -.. _release_2.5.0: - -2.5.0 ------ - -This release will be the last to support Python 3.5, next version of Zarr will be Python 3.6+. - -* `DirectoryStore` now uses `os.scandir`, which should make listing large store - faster, :issue:`563` - -* Remove a few remaining Python 2-isms. - By :user:`Poruri Sai Rahul `; :issue:`393`. - -* Fix minor bug in `N5Store`. - By :user:`gsakkis`, :issue:`550`. - -* Improve error message in Jupyter when trying to use the ``ipytree`` widget - without ``ipytree`` installed. - By :user:`Zain Patel `; :issue:`537` - -* Add typing information to many of the core functions :issue:`589` - -* Explicitly close stores during testing. - By :user:`Elliott Sales de Andrade `; :issue:`442` - -* Many of the convenience functions to emit errors (``err_*`` from - ``zarr.errors`` have been replaced by ``ValueError`` subclasses. The corresponding - ``err_*`` function have been removed. :issue:`590`, :issue:`614`) - -* Improve consistency of terminology regarding arrays and datasets in the - documentation. - By :user:`Josh Moore `; :issue:`571`. - -* Added support for generic URL opening by ``fsspec``, where the URLs have the - form "protocol://[server]/path" or can be chained URls with "::" separators. - The additional argument ``storage_options`` is passed to the backend, see - the ``fsspec`` docs. - By :user:`Martin Durant `; :issue:`546` - -* Added support for fetching multiple items via ``getitems`` method of a - store, if it exists. This allows for concurrent fetching of data blocks - from stores that implement this; presently HTTP, S3, GCS. Currently only - applies to reading. - By :user:`Martin Durant `; :issue:`606` - -* Efficient iteration expanded with option to pass start and stop index via - ``array.islice``. - By :user:`Sebastian Grill `, :issue:`615`. - -.. _release_2.4.0: - -2.4.0 ------ - -Enhancements -~~~~~~~~~~~~ - -* Add key normalization option for ``DirectoryStore``, ``NestedDirectoryStore``, - ``TempStore``, and ``N5Store``. - By :user:`James Bourbeau `; :issue:`459`. - -* Add ``recurse`` keyword to ``Group.array_keys`` and ``Group.arrays`` methods. - By :user:`James Bourbeau `; :issue:`458`. - -* Use uniform chunking for all dimensions when specifying ``chunks`` as an integer. - Also adds support for specifying ``-1`` to chunk across an entire dimension. - By :user:`James Bourbeau `; :issue:`456`. - -* Rename ``DictStore`` to ``MemoryStore``. - By :user:`James Bourbeau `; :issue:`455`. - -* Rewrite ``.tree()`` pretty representation to use ``ipytree``. - Allows it to work in both the Jupyter Notebook and JupyterLab. - By :user:`John Kirkham `; :issue:`450`. - -* Do not rename Blosc parameters in n5 backend and add `blocksize` parameter, - compatible with n5-blosc. By :user:`axtimwalde`, :issue:`485`. - -* Update ``DirectoryStore`` to create files with more permissive permissions. - By :user:`Eduardo Gonzalez ` and :user:`James Bourbeau `; :issue:`493` - -* Use ``math.ceil`` for scalars. - By :user:`John Kirkham `; :issue:`500`. - -* Ensure contiguous data using ``astype``. - By :user:`John Kirkham `; :issue:`513`. - -* Refactor out ``_tofile``/``_fromfile`` from ``DirectoryStore``. - By :user:`John Kirkham `; :issue:`503`. - -* Add ``__enter__``/``__exit__`` methods to ``Group`` for ``h5py.File`` compatibility. - By :user:`Chris Barnes `; :issue:`509`. - -Bug fixes -~~~~~~~~~ - -* Fix Sqlite Store Wrong Modification. - By :user:`Tommy Tran `; :issue:`440`. - -* Add intermediate step (using ``zipfile.ZipInfo`` object) to write - inside ``ZipStore`` to solve too restrictive permission issue. - By :user:`Raphael Dussin `; :issue:`505`. - -* Fix '/' prepend bug in ``ABSStore``. - By :user:`Shikhar Goenka `; :issue:`525`. - -Documentation -~~~~~~~~~~~~~ -* Fix hyperlink in ``README.md``. - By :user:`Anderson Banihirwe `; :issue:`531`. - -* Replace "nuimber" with "number". - By :user:`John Kirkham `; :issue:`512`. - -* Fix azure link rendering in tutorial. - By :user:`James Bourbeau `; :issue:`507`. - -* Update ``README`` file to be more detailed. - By :user:`Zain Patel `; :issue:`495`. - -* Import blosc from numcodecs in tutorial. - By :user:`James Bourbeau `; :issue:`491`. - -* Adds logo to docs. - By :user:`James Bourbeau `; :issue:`462`. - -* Fix N5 link in tutorial. - By :user:`James Bourbeau `; :issue:`480`. - -* Fix typo in code snippet. - By :user:`Joe Jevnik `; :issue:`461`. - -* Fix URLs to point to zarr-python - By :user:`John Kirkham `; :issue:`453`. - -Maintenance -~~~~~~~~~~~ - -* Add documentation build to CI. - By :user:`James Bourbeau `; :issue:`516`. - -* Use ``ensure_ndarray`` in a few more places. - By :user:`John Kirkham `; :issue:`506`. - -* Support Python 3.8. - By :user:`John Kirkham `; :issue:`499`. - -* Require Numcodecs 0.6.4+ to use text handling functionality from it. - By :user:`John Kirkham `; :issue:`497`. - -* Updates tests to use ``pytest.importorskip``. - By :user:`James Bourbeau `; :issue:`492` - -* Removed support for Python 2. - By :user:`jhamman`; :issue:`393`, :issue:`470`. - -* Upgrade dependencies in the test matrices and resolve a - compatibility issue with testing against the Azure Storage - Emulator. By :user:`alimanfoo`; :issue:`468`, :issue:`467`. - -* Use ``unittest.mock`` on Python 3. - By :user:`Elliott Sales de Andrade `; :issue:`426`. - -* Drop ``decode`` from ``ConsolidatedMetadataStore``. - By :user:`John Kirkham `; :issue:`452`. - - -.. _release_2.3.2: - -2.3.2 ------ - -Enhancements -~~~~~~~~~~~~ - -* Use ``scandir`` in ``DirectoryStore``'s ``getsize`` method. - By :user:`John Kirkham `; :issue:`431`. - -Bug fixes -~~~~~~~~~ - -* Add and use utility functions to simplify reading and writing JSON. - By :user:`John Kirkham `; :issue:`429`, :issue:`430`. - -* Fix ``collections``'s ``DeprecationWarning``\ s. - By :user:`John Kirkham `; :issue:`432`. - -* Fix tests on big endian machines. - By :user:`Elliott Sales de Andrade `; :issue:`427`. - - -.. _release_2.3.1: - -2.3.1 ------ - -Bug fixes -~~~~~~~~~ - -* Makes ``azure-storage-blob`` optional for testing. - By :user:`John Kirkham `; :issue:`419`, :issue:`420`. - - -.. _release_2.3.0: - -2.3.0 ------ - -Enhancements -~~~~~~~~~~~~ - -* New storage backend, backed by Azure Blob Storage, class :class:`zarr.storage.ABSStore`. - All data is stored as block blobs. By :user:`Shikhar Goenka `, - :user:`Tim Crone ` and :user:`Zain Patel `; :issue:`345`. - -* Add "consolidated" metadata as an experimental feature: use - :func:`zarr.convenience.consolidate_metadata` to copy all metadata from the various - metadata keys within a dataset hierarchy under a single key, and - :func:`zarr.convenience.open_consolidated` to use this single key. This can greatly - cut down the number of calls to the storage backend, and so remove a lot of overhead - for reading remote data. - By :user:`Martin Durant `, :user:`Alistair Miles `, - :user:`Ryan Abernathey `, :issue:`268`, :issue:`332`, :issue:`338`. - -* Support has been added for structured arrays with sub-array shape and/or nested fields. By - :user:`Tarik Onalan `, :issue:`111`, :issue:`296`. - -* Adds the SQLite-backed :class:`zarr.storage.SQLiteStore` class enabling an - SQLite database to be used as the backing store for an array or group. - By :user:`John Kirkham `, :issue:`368`, :issue:`365`. - -* Efficient iteration over arrays by decompressing chunkwise. - By :user:`Jerome Kelleher `, :issue:`398`, :issue:`399`. - -* Adds the Redis-backed :class:`zarr.storage.RedisStore` class enabling a - Redis database to be used as the backing store for an array or group. - By :user:`Joe Hamman `, :issue:`299`, :issue:`372`. - -* Adds the MongoDB-backed :class:`zarr.storage.MongoDBStore` class enabling a - MongoDB database to be used as the backing store for an array or group. - By :user:`Noah D Brenowitz `, :user:`Joe Hamman `, - :issue:`299`, :issue:`372`, :issue:`401`. - -* **New storage class for N5 containers**. The :class:`zarr.n5.N5Store` has been - added, which uses :class:`zarr.storage.NestedDirectoryStore` to support - reading and writing from and to N5 containers. - By :user:`Jan Funke ` and :user:`John Kirkham `. - -Bug fixes -~~~~~~~~~ - -* The implementation of the :class:`zarr.storage.DirectoryStore` class has been modified to - ensure that writes are atomic and there are no race conditions where a chunk might appear - transiently missing during a write operation. By :user:`sbalmer `, :issue:`327`, - :issue:`263`. - -* Avoid raising in :class:`zarr.storage.DirectoryStore`'s ``__setitem__`` when file already exists. - By :user:`Justin Swaney `, :issue:`272`, :issue:`318`. - -* The required version of the `Numcodecs`_ package has been upgraded - to 0.6.2, which has enabled some code simplification and fixes a failing test involving - msgpack encoding. By :user:`John Kirkham `, :issue:`361`, :issue:`360`, :issue:`352`, - :issue:`355`, :issue:`324`. - -* Failing tests related to pickling/unpickling have been fixed. By :user:`Ryan Williams `, - :issue:`273`, :issue:`308`. - -* Corrects handling of ``NaT`` in ``datetime64`` and ``timedelta64`` in various - compressors (by :user:`John Kirkham `; :issue:`344`). - -* Ensure ``DictStore`` contains only ``bytes`` to facilitate comparisons and protect against writes. - By :user:`John Kirkham `, :issue:`350`. - -* Test and fix an issue (w.r.t. fill values) when storing complex data to ``Array``. - By :user:`John Kirkham `, :issue:`363`. - -* Always use a ``tuple`` when indexing a NumPy ``ndarray``. - By :user:`John Kirkham `, :issue:`376`. - -* Ensure when ``Array`` uses a ``dict``-based chunk store that it only contains - ``bytes`` to facilitate comparisons and protect against writes. Drop the copy - for the no filter/compressor case as this handles that case. - By :user:`John Kirkham `, :issue:`359`. - -Maintenance -~~~~~~~~~~~ - -* Simplify directory creation and removal in ``DirectoryStore.rename``. - By :user:`John Kirkham `, :issue:`249`. - -* CI and test environments have been upgraded to include Python 3.7, drop Python 3.4, and - upgrade all pinned package requirements. :user:`Alistair Miles `, :issue:`308`. - -* Start using pyup.io to maintain dependencies. - :user:`Alistair Miles `, :issue:`326`. - -* Configure flake8 line limit generally. - :user:`John Kirkham `, :issue:`335`. - -* Add missing coverage pragmas. - :user:`John Kirkham `, :issue:`343`, :issue:`355`. - -* Fix missing backslash in docs. - :user:`John Kirkham `, :issue:`254`, :issue:`353`. - -* Include tests for stores' ``popitem`` and ``pop`` methods. - By :user:`John Kirkham `, :issue:`378`, :issue:`380`. - -* Include tests for different compressors, endianness, and attributes. - By :user:`John Kirkham `, :issue:`378`, :issue:`380`. - -* Test validity of stores' contents. - By :user:`John Kirkham `, :issue:`359`, :issue:`408`. - - -.. _release_2.2.0: - -2.2.0 ------ - -Enhancements -~~~~~~~~~~~~ - -* **Advanced indexing**. The ``Array`` class has several new methods and - properties that enable a selection of items in an array to be retrieved or - updated. See the :ref:`user-guide-indexing` tutorial section for more - information. There is also a `notebook - `_ - with extended examples and performance benchmarks. :issue:`78`, :issue:`89`, - :issue:`112`, :issue:`172`. - -* **New package for compressor and filter codecs**. The classes previously - defined in the :mod:`zarr.codecs` module have been factored out into a - separate package called `Numcodecs`_. The `Numcodecs`_ package also includes - several new codec classes not previously available in Zarr, including - compressor codecs for Zstd and LZ4. This change is backwards-compatible with - existing code, as all codec classes defined by Numcodecs are imported into the - :mod:`zarr.codecs` namespace. However, it is recommended to import codecs from - the new package, see the tutorial sections on :ref:`user-guide-compress` and - :ref:`user-guide-filters` for examples. With contributions by - :user:`John Kirkham `; :issue:`74`, :issue:`102`, :issue:`120`, - :issue:`123`, :issue:`139`. - -* **New storage class for DBM-style databases**. The - :class:`zarr.storage.DBMStore` class enables any DBM-style database such as gdbm, - ndbm or Berkeley DB, to be used as the backing store for an array or group. See the - tutorial section on :ref:`user-guide-storage` for some examples. :issue:`133`, - :issue:`186`. - -* **New storage class for LMDB databases**. The :class:`zarr.storage.LMDBStore` class - enables an LMDB "Lightning" database to be used as the backing store for an array or - group. :issue:`192`. - -* **New storage class using a nested directory structure for chunk files**. The - :class:`zarr.storage.NestedDirectoryStore` has been added, which is similar to - the existing :class:`zarr.storage.DirectoryStore` class but nests chunk files - for multidimensional arrays into sub-directories. :issue:`155`, :issue:`177`. - -* **New tree() method for printing hierarchies**. The ``Group`` class has a new - :func:`zarr.hierarchy.Group.tree` method which enables a tree representation of - a group hierarchy to be printed. Also provides an interactive tree - representation when used within a Jupyter notebook. See the - :ref:`user-guide-diagnostics` tutorial section for examples. By - :user:`John Kirkham `; :issue:`82`, :issue:`140`, :issue:`184`. - -* **Visitor API**. The ``Group`` class now implements the h5py visitor API, see - docs for the :func:`zarr.hierarchy.Group.visit`, - :func:`zarr.hierarchy.Group.visititems` and - :func:`zarr.hierarchy.Group.visitvalues` methods. By - :user:`John Kirkham `, :issue:`92`, :issue:`122`. - -* **Viewing an array as a different dtype**. The ``Array`` class has a new - :func:`zarr.Array.astype` method, which is a convenience that enables an - array to be viewed as a different dtype. By :user:`John Kirkham `, - :issue:`94`, :issue:`96`. - -* **New open(), save(), load() convenience functions**. The function - :func:`zarr.convenience.open` provides a convenient way to open a persistent - array or group, using either a ``DirectoryStore`` or ``ZipStore`` as the backing - store. The functions :func:`zarr.convenience.save` and - :func:`zarr.convenience.load` are also available and provide a convenient way to - save an entire NumPy array to disk and load back into memory later. See the - tutorial section :ref:`user-guide-persist` for examples. :issue:`104`, - :issue:`105`, :issue:`141`, :issue:`181`. - -* **IPython completions**. The ``Group`` class now implements ``__dir__()`` and - ``_ipython_key_completions_()`` which enables tab-completion for group members - to be used in any IPython interactive environment. :issue:`170`. - -* **New info property; changes to __repr__**. The ``Group`` and - ``Array`` classes have a new ``info`` property which can be used to print - diagnostic information, including compression ratio where available. See the - tutorial section on :ref:`user-guide-diagnostics` for examples. The string - representation (``__repr__``) of these classes has been simplified to ensure - it is cheap and quick to compute in all circumstances. :issue:`83`, - :issue:`115`, :issue:`132`, :issue:`148`. - -* **Chunk options**. When creating an array, ``chunks=False`` can be specified, - which will result in an array with a single chunk only. Alternatively, - ``chunks=True`` will trigger an automatic chunk shape guess. See - :ref:`user-guide-chunks` for more on the ``chunks`` parameter. :issue:`106`, - :issue:`107`, :issue:`183`. - -* **Zero-dimensional arrays** and are now supported; by - :user:`Prakhar Goel `, :issue:`154`, :issue:`161`. - -* **Arrays with one or more zero-length dimensions** are now fully supported; by - :user:`Prakhar Goel `, :issue:`150`, :issue:`154`, :issue:`160`. - -* **The .zattrs key is now optional** and will now only be created when the first - custom attribute is set; :issue:`121`, :issue:`200`. - -* **New Group.move() method** supports moving a sub-group or array to a different - location within the same hierarchy. By :user:`John Kirkham `, - :issue:`191`, :issue:`193`, :issue:`196`. - -* **ZipStore is now thread-safe**; :issue:`194`, :issue:`192`. - -* **New Array.hexdigest() method** computes an ``Array``'s hash with ``hashlib``. - By :user:`John Kirkham `, :issue:`98`, :issue:`203`. - -* **Improved support for object arrays**. In previous versions of Zarr, - creating an array with ``dtype=object`` was possible but could under certain - circumstances lead to unexpected errors and/or segmentation faults. To make it easier - to properly configure an object array, a new ``object_codec`` parameter has been - added to array creation functions. See the tutorial section on :ref:`user-guide-objects` - for more information and examples. Also, runtime checks have been added in both Zarr - and Numcodecs so that segmentation faults are no longer possible, even with a badly - configured array. This API change is backwards compatible and previous code that created - an object array and provided an object codec via the ``filters`` parameter will - continue to work, however a warning will be raised to encourage use of the - ``object_codec`` parameter. :issue:`208`, :issue:`212`. - -* **Added support for datetime64 and timedelta64 data types**; - :issue:`85`, :issue:`215`. - -* **Array and group attributes are now cached by default** to improve performance with - slow stores, e.g., stores accessing data via the network; :issue:`220`, :issue:`218`, - :issue:`204`. - -* **New LRUStoreCache class**. The class :class:`zarr.storage.LRUStoreCache` has been - added and provides a means to locally cache data in memory from a store that may be - slow, e.g., a store that retrieves data from a remote server via the network; - :issue:`223`. - -* **New copy functions**. The new functions :func:`zarr.convenience.copy` and - :func:`zarr.convenience.copy_all` provide a way to copy groups and/or arrays - between HDF5 and Zarr, or between two Zarr groups. The - :func:`zarr.convenience.copy_store` provides a more efficient way to copy - data directly between two Zarr stores. :issue:`87`, :issue:`113`, - :issue:`137`, :issue:`217`. - -Bug fixes -~~~~~~~~~ - -* Fixed bug where ``read_only`` keyword argument was ignored when creating an - array; :issue:`151`, :issue:`179`. - -* Fixed bugs when using a ``ZipStore`` opened in 'w' mode; :issue:`158`, - :issue:`182`. - -* Fill values can now be provided for fixed-length string arrays; :issue:`165`, - :issue:`176`. - -* Fixed a bug where the number of chunks initialized could be counted - incorrectly; :issue:`97`, :issue:`174`. - -* Fixed a bug related to the use of an ellipsis (...) in indexing statements; - :issue:`93`, :issue:`168`, :issue:`172`. - -* Fixed a bug preventing use of other integer types for indexing; :issue:`143`, - :issue:`147`. - -Documentation -~~~~~~~~~~~~~ - -* Some changes have been made to the Zarr Specification v2 document to clarify - ambiguities and add some missing information. These changes do not break compatibility - with any of the material as previously implemented, and so the changes have been made - in-place in the document without incrementing the document version number. See the - section on changes in the specification document for more information. -* A new :ref:`user-guide-indexing` section has been added to the tutorial. -* A new :ref:`user-guide-strings` section has been added to the tutorial - (:issue:`135`, :issue:`175`). -* The :ref:`user-guide-chunks` tutorial section has been reorganised and updated. -* The :ref:`user-guide-persist` and :ref:`user-guide-storage` tutorial sections have - been updated with new examples (:issue:`100`, :issue:`101`, :issue:`103`). -* A new tutorial section on :ref:`user-guide-pickle` has been added (:issue:`91`). -* A new tutorial section on :ref:`user-guide-datetime` has been added. -* A new tutorial section on :ref:`user-guide-diagnostics` has been added. -* The tutorial sections on :ref:`user-guide-sync` and :ref:`user-guide-tips-blosc` have been - updated to provide information about how to avoid program hangs when using the Blosc - compressor with multiple processes (:issue:`199`, :issue:`201`). - -Maintenance -~~~~~~~~~~~ - -* A data fixture has been included in the test suite to ensure data format - compatibility is maintained; :issue:`83`, :issue:`146`. -* The test suite has been migrated from nosetests to pytest; :issue:`189`, :issue:`225`. -* Various continuous integration updates and improvements; :issue:`118`, :issue:`124`, - :issue:`125`, :issue:`126`, :issue:`109`, :issue:`114`, :issue:`171`. -* Bump numcodecs dependency to 0.5.3, completely remove nose dependency, :issue:`237`. -* Fix compatibility issues with NumPy 1.14 regarding fill values for structured arrays, - :issue:`222`, :issue:`238`, :issue:`239`. - -Acknowledgments -~~~~~~~~~~~~~~~ - -Code was contributed to this release by :user:`Alistair Miles `, :user:`John -Kirkham ` and :user:`Prakhar Goel `. - -Documentation was contributed to this release by :user:`Mamy Ratsimbazafy ` -and :user:`Charles Noyes `. - -Thank you to :user:`John Kirkham `, :user:`Stephan Hoyer `, -:user:`Francesc Alted `, and :user:`Matthew Rocklin ` for code -reviews and/or comments on pull requests. - -.. _release_2.1.4: - -2.1.4 ------ - -* Resolved an issue where calling ``hasattr`` on a ``Group`` object erroneously - returned a ``KeyError``. By :user:`Vincent Schut `; :issue:`88`, - :issue:`95`. - -.. _release_2.1.3: - -2.1.3 ------ - -* Resolved an issue with :func:`zarr.creation.array` where dtype was given as - None (:issue:`80`). - -.. _release_2.1.2: - -2.1.2 ------ - -* Resolved an issue when no compression is used and chunks are stored in memory - (:issue:`79`). - -.. _release_2.1.1: - -2.1.1 ------ - -Various minor improvements, including: ``Group`` objects support member access -via dot notation (``__getattr__``); fixed metadata caching for ``Array.shape`` -property and derivatives; added ``Array.ndim`` property; fixed -``Array.__array__`` method arguments; fixed bug in pickling ``Array`` state; -fixed bug in pickling ``ThreadSynchronizer``. - -.. _release_2.1.0: - -2.1.0 ------ - -* Group objects now support member deletion via ``del`` statement - (:issue:`65`). -* Added :class:`zarr.storage.TempStore` class for convenience to provide - storage via a temporary directory - (:issue:`59`). -* Fixed performance issues with :class:`zarr.storage.ZipStore` class - (:issue:`66`). -* The Blosc extension has been modified to return bytes instead of array - objects from compress and decompress function calls. This should - improve compatibility and also provides a small performance increase for - compressing high compression ratio data - (:issue:`55`). -* Added ``overwrite`` keyword argument to array and group creation methods - on the :class:`zarr.hierarchy.Group` class - (:issue:`71`). -* Added ``cache_metadata`` keyword argument to array creation methods. -* The functions :func:`zarr.creation.open_array` and - :func:`zarr.hierarchy.open_group` now accept any store as first argument - (:issue:`56`). - -.. _release_2.0.1: - -2.0.1 ------ - -The bundled Blosc library has been upgraded to version 1.11.1. - -.. _release_2.0.0: - -2.0.0 ------ - -Hierarchies -~~~~~~~~~~~ - -Support has been added for organizing arrays into hierarchies via groups. See -the tutorial section on :ref:`user-guide-groups` and the :mod:`zarr.hierarchy` -API docs for more information. - -Filters -~~~~~~~ - -Support has been added for configuring filters to preprocess chunk data prior -to compression. See the tutorial section on :ref:`user-guide-filters` and the -:mod:`zarr.codecs` API docs for more information. - -Other changes -~~~~~~~~~~~~~ - -To accommodate support for hierarchies and filters, the Zarr metadata format -has been modified. See the ``spec_v2`` for more information. To migrate an -array stored using Zarr version 1.x, use the :func:`zarr.storage.migrate_1to2` -function. - -The bundled Blosc library has been upgraded to version 1.11.0. - -Acknowledgments -~~~~~~~~~~~~~~~ - -Thanks to :user:`Matthew Rocklin `, :user:`Stephan Hoyer ` and -:user:`Francesc Alted ` for contributions and comments. - -.. _release_1.1.0: - -1.1.0 ------ - -* The bundled Blosc library has been upgraded to version 1.10.0. The 'zstd' - internal compression library is now available within Blosc. See the tutorial - section on :ref:`user-guide-compress` for an example. -* When using the Blosc compressor, the default internal compression library - is now 'lz4'. -* The default number of internal threads for the Blosc compressor has been - increased to a maximum of 8 (previously 4). -* Added convenience functions :func:`zarr.blosc.list_compressors` and - :func:`zarr.blosc.get_nthreads`. - -.. _release_1.0.0: - -1.0.0 ------ - -This release includes a complete re-organization of the code base. The -major version number has been bumped to indicate that there have been -backwards-incompatible changes to the API and the on-disk storage -format. However, Zarr is still in an early stage of development, so -please do not take the version number as an indicator of maturity. - -Storage -~~~~~~~ - -The main motivation for re-organizing the code was to create an -abstraction layer between the core array logic and data storage (:issue:`21`). -In this release, any -object that implements the ``MutableMapping`` interface can be used as -an array store. See the tutorial sections on :ref:`user-guide-persist` -and :ref:`user-guide-storage`, the ``spec_v1``, and the -:mod:`zarr.storage` module documentation for more information. - -Please note also that the file organization and file name conventions -used when storing a Zarr array in a directory on the file system have -changed. Persistent Zarr arrays created using previous versions of the -software will not be compatible with this version. See the -:mod:`zarr.storage` API docs and the ``spec_v1`` for more -information. - -Compression -~~~~~~~~~~~ - -An abstraction layer has also been created between the core array -logic and the code for compressing and decompressing array -chunks. This release still bundles the c-blosc library and uses Blosc -as the default compressor, however other compressors including zlib, -BZ2 and LZMA are also now supported via the Python standard -library. New compressors can also be dynamically registered for use -with Zarr. See the tutorial sections on :ref:`user-guide-compress` and -:ref:`user-guide-tips-blosc`, the ``spec_v1``, and the -:mod:`zarr.compressors` module documentation for more information. - -Synchronization -~~~~~~~~~~~~~~~ - -The synchronization code has also been refactored to create a layer of -abstraction, enabling Zarr arrays to be used in parallel computations -with a number of alternative synchronization methods. For more -information see the tutorial section on :ref:`user-guide-sync` and the -:mod:`zarr.sync` module documentation. - -Changes to the Blosc extension -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -NumPy is no longer a build dependency for the :mod:`zarr.blosc` Cython -extension, so setup.py will run even if NumPy is not already -installed, and should automatically install NumPy as a runtime -dependency. Manual installation of NumPy prior to installing Zarr is -still recommended, however, as the automatic installation of NumPy may -fail or be sub-optimal on some platforms. - -Some optimizations have been made within the :mod:`zarr.blosc` -extension to avoid unnecessary memory copies, giving a ~10-20% -performance improvement for multi-threaded compression operations. - -The :mod:`zarr.blosc` extension now automatically detects whether it -is running within a single-threaded or multi-threaded program and -adapts its internal behaviour accordingly (:issue:`27`). There is no need for -the user to make any API calls to switch Blosc between contextual and -non-contextual (global lock) mode. See also the tutorial section on -:ref:`user-guide-tips-blosc`. - -Other changes -~~~~~~~~~~~~~ - -The internal code for managing chunks has been rewritten to be more -efficient. Now no state is maintained for chunks outside of the array -store, meaning that chunks do not carry any extra memory overhead not -accounted for by the store. This negates the need for the "lazy" -option present in the previous release, and this has been removed. - -The memory layout within chunks can now be set as either "C" -(row-major) or "F" (column-major), which can help to provide better -compression for some data (:issue:`7`). See the tutorial -section on :ref:`user-guide-chunks-order` for more information. - -A bug has been fixed within the ``__getitem__`` and ``__setitem__`` -machinery for slicing arrays, to properly handle getting and setting -partial slices. - -Acknowledgments -~~~~~~~~~~~~~~~ - -Thanks to :user:`Matthew Rocklin `, :user:`Stephan Hoyer `, -:user:`Francesc Alted `, :user:`Anthony Scopatz ` and -:user:`Martin Durant ` for contributions and comments. - -.. _release_0.4.0: - -0.4.0 ------ - -See `v0.4.0 release notes on GitHub -`_. - -.. _release_0.3.0: - -0.3.0 ------ - -See `v0.3.0 release notes on GitHub -`_. - -.. _Numcodecs: https://numcodecs.readthedocs.io/ diff --git a/docs/index.rst b/docs/index.rst index 0dcfd7f90f..6ab07b0693 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -11,8 +11,8 @@ Zarr-Python quickstart user-guide/index API reference + release-notes developers/index - developers/release about **Version**: |version| diff --git a/docs/release-notes.rst b/docs/release-notes.rst new file mode 100644 index 0000000000..175bd21aa5 --- /dev/null +++ b/docs/release-notes.rst @@ -0,0 +1,16 @@ +Release notes +============= + +.. _release_3.0.0: + +3.0.0 +----- + +3.0.0 is a new major release of Zarr-Python, with many breaking changes. +See the :ref:`v3 migration guide` for a listing of what's changed. + +Normal release note service will resume with further releases in the 3.0.0 +series. + +Release notes for the zarr-python 2.x and 1.x releases can be found here: +https://zarr.readthedocs.io/en/support-v2/release.html diff --git a/docs/user-guide/v3_migration.rst b/docs/user-guide/v3_migration.rst index 66fcca6d19..bda1ae64ed 100644 --- a/docs/user-guide/v3_migration.rst +++ b/docs/user-guide/v3_migration.rst @@ -1,3 +1,5 @@ +.. _v3 migration guide: + 3.0 Migration Guide ===================