Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flag format: uint8 vs bool #346

Open
ludwigschwardt opened this issue Dec 17, 2024 · 0 comments
Open

Flag format: uint8 vs bool #346

ludwigschwardt opened this issue Dec 17, 2024 · 0 comments

Comments

@ludwigschwardt
Copy link

ludwigschwardt commented Dec 17, 2024

From the MSv4 schema it seems that the FLAG data variable is of type bool. Looking at Zarr, this is mapped to a single byte (not a bit), although the v3 spec leaves the door open:

Zarr v3 is limited to type sizes that are a multiple of 8 bits but may support other type sizes in later versions of this specification.

The reason this interests me is because MeerKAT's MVF4 format uses uint8 for flags and actually store 8 different flags in there. It would be a shame to expand this to 6 separate flag bytes (we have 2 unused bits at the moment), although I suspect compression might come to the rescue. This is in light of the absence of flag categories in the new format.

My current thinking is to pretend that our uint8s are bools by saving our flags as flags.view(bool).

Would this be "unsafe" behaviour? What about exploring the idea to make the flags uint8 from the outset and support categories that way?

@ludwigschwardt ludwigschwardt changed the title Flag format Flag format: uint8 vs bool Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant