Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allowed syntax for virtual refs #718

Open
jbms opened this issue Feb 11, 2025 · 1 comment
Open

Allowed syntax for virtual refs #718

jbms opened this issue Feb 11, 2025 · 1 comment

Comments

@jbms
Copy link

jbms commented Feb 11, 2025

Do I understand correctly that there is basically no specific defined URL syntax for virtual refs --- and instead the URL prefix to virtual chunk store mapping must always be specified separately when opening an icechunk repository?

Currently in Neuroglancer there is no configuration mechanism for icechunk virtual refs and instead the URLs are just resolved according to the usual Neuroglancer URL syntax, and it would be somewhat awkward and inconvenient for users to have to specify a configuration.

I'm happy to change Neuroglancer's interpretation of virtual refs to match whatever is decided upon, but it would be very helpful to standardize on some specific URL syntax so that icechunk repositories can at least, in common cases, be used without configuration.

One specific thing I would recommend is to discourage the use of s3://bucket/path to ambiguously refer to either the official S3 endpoint bucket.s3.amazonaws.com or a separate S3-like endpoint. Instead, I would suggest using s3+https://hostname/path to indicate an S3-compatible server. Though, for purpose of virtual chunks it seems that just plain https://hostname/path would also be fine since you don't need to perform list operations.

@paraseba
Copy link
Collaborator

the URL prefix to virtual chunk store mapping must always be specified separately when opening an icechunk repository?

It can either be specified when opening the repo, or it can be stored as configuration together with the repo. In the later case, users don't need to pass it on open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants