Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

L1processing #122

Merged
merged 4 commits into from
Aug 12, 2024
Merged

L1processing #122

merged 4 commits into from
Aug 12, 2024

Conversation

tmieslinger
Copy link
Collaborator

This PR adds the possibility to create level1 files from level0 via the ASPEN software that is run in a docker image. That way, it is not a prerequisite to process L0 files via ASPEN or BatchASPEN manually. If L1 files exist, they are used. If not, they will be created using the docker image ghcr.io/atmdrops/aspenqc generated and described in the repo aspenqc.

The implementation includes adding the L0 to L1 processing in the pipeline dict and moving the filter_no_launch_detect function upfront. Also, several error messages are removed as (1) the launch_detect function was moved and (2) if the a-file does not exist, python will raise an AttributeError anyway.

Closes #25.

@d70-t d70-t requested a review from Geet-George August 7, 2024 21:09
The check for adding the postaspenfile without launch detect isn't
useful anymore, as doing so currently has no defined behavior and it's
expected that the pipeline filters no launch detect sondes prior to
calling run_aspen.

The other tests should still work the same.
NOTE: This change ONLY adapts the existing tests to the new function
name. It does not test if the processing actually does useful things.
Testing this properly would require more complete testdata, which will
be available after Geet-George#113 gets merged.
@Geet-George
Copy link
Owner

The way this is set up looks so nice! :) Thanks so much.

One question: Currently for the package to work it would have to be connected to the internet to get the docker image right? Would it be possible that we add pull in the docker image as a dependency for the package to work? This means that in-flight processing would also be possible.

@tmieslinger
Copy link
Collaborator Author

That's a really good point. So if you run it once, it will be cached on your laptop and you can use the docker image offline, too. But I'll check whether and how it could be integrated into the first installation of halodrops 👍

@tmieslinger
Copy link
Collaborator Author

Following Tobi's (@d70-t) advice, I'd start an issue on the offline processing of L0 to L1 as it's supposedly a feature creep to pull a docker image within a pip install halodrops. I would instead add a note in the docu or readme that if you plan to process L0 to L1 the first time offline, then you'd have to pull the docker image manually first.

@Geet-George
Copy link
Owner

Geet-George commented Aug 12, 2024

Yeah, makes sense. Also, best to merge this asap, I guess. The :::warning::: in docu should be enough for users to know that it should be pulled once before.

@Geet-George Geet-George merged commit c361d33 into Geet-George:main Aug 12, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate ASPEN CLI
3 participants