Streamline for External Development

Purpose

We will collect here notes regarding the streamlining of DRAGONS to help with the development and usage of external packages. This will help Gemini with the development of DR software for new instruments which is done by external teams. It will also allow anyone from any telescope to develop using the DRAGONS infrastructure and generic primitives. Finally, if the US-ELTP decides to adopt DRAGONS, the work described here will facilitate the process of getting the TMT and GMT pipeline developed for or converted to DRAGONS.

Preliminary list of issues to resolve

Items in bold should be considered priority as they will help remove circular imports with the calmgr package.

• Split AstroData into its own package.
• Split gemini_instruments into its own package.
• Move geminidr/core to its own ...dr package. People shouldn't have to import geminidr to use generic core primitives.
• Is there anything in gemini_instruments that is generic and that would need to be taken out into its own ..._instruments package?
• We need to clean up gempy and split what's generic, what's needed by recipe_system, what's specific to Gemini.
• recipe_system is not Gemini-free. I (KL) vaguely remember some strange dependencies pointing back to geminidr or gempy.
• Calibration Association and Calibration Manager !!! This all needs to be revised now that gemini_calmgr has been returned to the main archive code. !!! We really need to figure out how to do calrules plugins. !!
  • The gemini_calmgr needs to be in it's own public repo rather in the private archive repo. This split has been done and will be used for v3.1. The next step is to move those repos from gitlab to github. (Maybe do that closer to release of v3.1 after a fair bit of testing has been done.)
  • If we are to offer gemini_calmgr to other telescopes we have to allow for the rules to be added as a plugin. We can't possibly just keep adding everyone's rules for random instruments. Also, remember that the archive uses gemini_calmgr, so we need to keep it clean.
  • A rebranding of gemini_calmgr might be in order once we've figured out the plugin system. (dragons_calmgr or something else without dragons or gemini.)
  • calibration rules should be added as third-party, per instrument, per observatory, whatever. It shouldn't be that one needs to fork gemini_calmgr or the archive code to add their rules.
• dragonsrc should be used to tell the system which third-party packages to use of the cal rules, the *_instruments, and the *dr packages. Right now they need to use --adpkg and --drpkg on the reduce command line, that could easily be replaced with lines in the config file.
• rebranding when each elements are distributed on their own : recipe_system -> dragons; geminidr -> gemini_dragons; gemini_instruments -> gemini_astrodata; geminidr/core -> core_dragons.
• gemini_calmgr is required to -r display an image. Probably because it will try to get a BPM but the calmgr should be somewhat optional (in general) since it is possible to add the calibrations on the command line.

All of the points above need to be documented.

Documentation

• Document how streams work.
• Document the decorators, what they do, when they should be used
  • for astrodata tags and descriptors (eg. which descriptors need to use the @return_list decorator)
  • for primitives
  • for tests
• Document which descriptors are used in which primitives, actually, document the primitives...