Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set longdouble=False in cftime.date2num within the date encoding context #7171

Merged
merged 2 commits into from
Oct 18, 2022

Conversation

spencerkclark
Copy link
Member

Currently, the default behavior of cftime.date2num is to return integer values when possible (i.e. when the encoding units allow), and fall back to returning float64 values when that is not possible. Recently, cftime added the option to use float128 as the fallback dtype, which enables greater potential roundtrip precision. This is through the longdouble flag to cftime.date2num, which currently defaults to False. It was intentionally set to False by default, because netCDF does not support storing float128 values in files, and so, without any changes, would otherwise break xarray's encoding procedure.

The desire in cftime, however, is to eventually set this flag to True by default (Unidata/cftime#297). This PR makes the necessary changes in xarray to adapt to this eventual new default. Essentially if the longdouble argument is allowed in the user's version of cftime.date2num, we explicitly set it to False to preserve the current float64 fallback behavior within the context of encoding times. There are a few more places where date2num is used (some additional places in the tests, and in calendar_ops.py), but in those places using float128 values would not present a problem.

At some point we might consider relaxing this behavior in xarray, since it is possible to store float128 values in zarr stores for example, but for the time being the simplest approach seems to be to stick with float64 for all backends (it would be complicated to have backend-specific defaults).

cc: @jswhit

@dcherian
Copy link
Contributor

Seems good to me.

thanks @spencerkclark

Shall we open an issue with your first comment to keep track of this?

@dcherian dcherian added the plan to merge Final call for comments label Oct 17, 2022
@spencerkclark
Copy link
Member Author

Thanks @dcherian — yes it probably would be good to create an issue to track further discussion about this. Encoding times with float128 values might be of interest to some. I’ll create one tomorrow.

@dcherian dcherian merged commit 9df2dfc into pydata:main Oct 18, 2022
@spencerkclark spencerkclark deleted the cftime-float64 branch October 18, 2022 16:38
keewis pushed a commit to keewis/xarray that referenced this pull request Oct 19, 2022
…context (pydata#7171)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plan to merge Final call for comments topic-cftime
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants