diff --git a/CHANGELOG.md b/CHANGELOG.md index f1d59085..a93c2133 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - Add a global configuration `config::Config` accessible via `config::{get_config,get_config_mut}` - Currently it exposes a single configuration option: `validate_checksums` (default: `true`) + - Document correctness issues with past versions and how to correct errant arrays in crate root ## [0.11.5] - 2024-02-05 diff --git a/README.md b/README.md index b724da2f..42c2611b 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ A rust library for the [Zarr V3](https://zarr.dev) storage format for multidimen Developed at the [Department of Materials Physics](https://physics.anu.edu.au/research/mp/), Australian National University, Canberra, Australia. -**zarrs is experimental and in limited production use. Use at your own risk!** +**zarrs is experimental and in limited production use. Use at your own risk! Correctness issues with past versions are [detailed here](https://docs.rs/zarrs/latest/zarrs/#correctness-issues-with-past-versions).** - [API documentation (`docs.rs`)](https://docs.rs/zarrs/latest/zarrs/) - [Changelog (`CHANGELOG.md`)](./CHANGELOG.md) diff --git a/src/lib.rs b/src/lib.rs index 5033e96c..d8bbdb8b 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -2,7 +2,7 @@ //! //! Developed at the [Department of Materials Physics](https://physics.anu.edu.au/research/mp/), Australian National University, Canberra, Australia. //! -//! **zarrs is experimental and in limited production use. Use at your own risk!** +//! **zarrs is experimental and in limited production use. Use at your own risk! Correctness issues with past versions are [detailed here](#correctness-issues-with-past-versions).** //! //! A changelog can be found [here](https://github.com/LDeakin/zarrs/blob/main/CHANGELOG.md). //! @@ -68,6 +68,21 @@ //! - [zarrs-ffi](https://github.com/LDeakin/zarrs-ffi): A subset of zarrs exposed as a C API. //! - [zarrs_tools](https://github.com/LDeakin/zarrs_tools): Various tools for creating and manipulating Zarr v3 data. //! +//! ## Correctness Issues with Past Versions +//! - Prior to zarrs [v0.11.5](https://github.com/LDeakin/zarrs/releases/tag/v0.11.5), arrays that used the `crc32c` codec have invalid chunk checksums +//! - Such arrays will fail to be read by other zarr implementations that validate checksums +//! - These arrays can be read by zarrs if the [validate checksums](crate::config::Config#validate-checksums) global configuration option is disabled +//! - From zarrs [v0.11.2](https://github.com/LDeakin/zarrs/releases/tag/v0.11.2)-[v0.11.3](https://github.com/LDeakin/zarrs/releases/tag/v0.11.3), the codec configuration of the `crc32c` codec or `bytes` codec (with unspecified endianness) does not conform to the zarr specification +//! - Such arrays will likely fail to be read by other zarr implementations +//! - Zarrs still supports reading these arrays, but this may become an error in a future release +//! - Fixing these arrays only requires a simple metadata correction, e.g. +//! - `sed -i -E "s/(^([ tab]+)\"(crc32c|bytes)\"(,?)$)/\2{ \"name\": \"\3\" }\4/" zarr.json` +//! +//! [zarrs_tools](https://github.com/LDeakin/zarrs_tools) v0.2.3+ can fix arrays with the above correctness issues with `zarrs_reencode`. Example: +//! ```bash +//! zarrs_reencode --ignore-checksums -p 4 array.zarr array_fixed.zarr +//! ``` +//! //! ## Licence //! zarrs is licensed under either of //! - the Apache License, Version 2.0 [LICENSE-APACHE](./LICENCE-APACHE) or or