Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1061 refactor core file validators based on aind data schema models expected modalities #1084

Conversation

dbirman
Copy link
Member

@dbirman dbirman commented Sep 26, 2024

This PR takes advantage of the new ExpectedFiles model in aind-data-schema-models to refactor the core metdata validator to check for required/excluded files for all modalities. Currently the validators only check for ecephys/smartspim platform to confirm they have a few required files. The code now checks by modality (not platform) and confirms that all expected/required files are included.

In the event that two modalities are in conflict the code prioritizes REQUIRED > OPTIONAL > EXCLUDED

@dbirman dbirman linked an issue Sep 26, 2024 that may be closed by this pull request
7 tasks
@dbirman dbirman requested a review from saskiad September 26, 2024 20:20
@dbirman dbirman requested a review from dyf October 3, 2024 21:39
@dbirman dbirman requested a review from jtyoung84 October 7, 2024 18:35
@dbirman
Copy link
Member Author

dbirman commented Oct 7, 2024

There's a bug in this PR -- it can't handle modalities with a '-' in the name (need to .replace('-','_'))

@jtyoung84
Copy link
Collaborator

jtyoung84 commented Oct 7, 2024

There's a bug in this PR -- it can't handle modalities with a '-' in the name (need to .replace('-','_'))

I wonder if it'd be helpful to add the modality folder name explicitly in https://github.com/AllenNeuralDynamics/aind-data-schema-models/blob/main/src/aind_data_schema_models/models/modalities.csv

I guess I'm just using the abbreviation

@dbirman
Copy link
Member Author

dbirman commented Oct 7, 2024

There's a bug in this PR -- it can't handle modalities with a '-' in the name (need to .replace('-','_'))

I wonder if it'd be helpful to add the modality folder name explicitly in https://github.com/AllenNeuralDynamics/aind-data-schema-models/blob/main/src/aind_data_schema_models/models/modalities.csv

I guess I'm just using the abbreviation

Yeah I think we only ever use the abbreviation everywhere

@dbirman dbirman removed the request for review from dyf October 9, 2024 16:46
@dbirman dbirman added this pull request to the merge queue Oct 9, 2024
Merged via the queue into dev with commit 9cacb22 Oct 9, 2024
5 checks passed
@dbirman dbirman deleted the 1061-refactor-core-file-validators-based-on-aind-data-schema-models-expected-modalities branch October 9, 2024 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor core file validators based on aind-data-schema-models expected modalities
2 participants