draft ADR for schema refactor

decisions/0009-schema-refactor.md

@@ -0,0 +1,95 @@
+---
+status: accepted 
+date: 2023-12-19
+deciders: Chris Mungall, Lee Ann 
+consulted: Anastasiya Prymolenna, Brynn Zalmanek, James Tessmer, Montana Smith, Sam Purvine, Yuri Corilo, Mark Miller, Michael Thorton, Alicia Clum, Mark Miller    
+informed: Emiley Eloe-Fadrosh, Shreyas Cholia, Eric Cavanna 
+---
+# Schema Refactoring: Monterey//Berkeley
+
+## Decisions Made
+
+### Class:PlannedProcess
+* `Class:PlannedProcess` will be created as a root class 
+* All "Activity" classes are now of `Class:PlannedProcess`, and the word "Activity" will be removed
+
+### Class:MaterialProcessing
+* `Class:MaterialProcessing` will be created and be a subclass of `Class:PlannedProcess`
+* All processes involving material to material transformations (Biosamples to ProcessedSample, ProcessedSample to ProcessedSample) will be subclasses of `Class:MaterialProcessing` (examples: `Class:Extraction`, `Pooling`, `MixingProcess`, etc.). 
+* The output of any subclasses of `Class:MaterialProcessing` will have an output of ProcessedSample
+* `Class:ChemicalConversionProcess` will replace `Class:ReactionProcess`
+* `Class:BiosampleProcessing` will be removed. Subclasses of `Class:MaterialProcessing` will replace what was captured. 
+* Linking a protocol to a `Class:MaterialProcess` subclass can happen at the individual class level, or at the aggregation of processes via the `Class:ProtocolExecution`
+* `slot:catalyzed_by` is instantiated on `Class:ChemicalConversionProcess` with `Range:Class:Solution`.
+  * `Class:Solution` has `slot:has_solution_components` with `Range:Class:SolutionComponent` where `slot:compound` has `Range:Class:ProteolyticEnzymeEnum`.
+  * `slot:catalyzed_by` will capture any substrate or enzyme that was used to facilitate a chemical conversion process. 
+  * Enzymes that can be used as catalysts are available in the `Class:ProteolyticEnzymeEnum`.
+  * The `slot:catalyzed_by` will be referenced by proteomics workflows.
+
+### Class:DataGeneration
+* `Class:DataGeneration` will be created and take the place of `Class:OmicsProcessing`
+* `Class:OmicsProcessing` will be removed
+  * All reference to `Class:OmicsProcessing` should be eliminated from NMDC schema & Workflows
+  * Typecode `omprc` will still be valid for `Class:DataGeneration`
+* `Class:DataGeneration` will detail the process of inputting a material sample (Biosample or ProcessedSample, some MaterialEntity) into an instrument and generating data (output)
+  * Instrument data will be denoted using `slot:data_category` on `Class:DataObject`.
+  * The output of DataGeneration should always be `instrument_data`
+* `Class:DataGeneration` will have subclasses `Class:NucleotideSequencing` and `Class:MassSpectrometry` 
+  * This allows for the expansion of additional data types as NMDC grows
+* `slot:instrument_used` will have `Range:Class:Instrument`
+  * `Class:Instrument` will have separate enumerated slots for `vendor` and `model`
+  * `slot:id` will be the identifier of the `instrument_used` and `vendor` and `model` will NOT be inlined for `Class:DataGeneration`. Rather, it will be inferred / traced using the instrument's id.
+* Schema support for complex paths of data generation and data processing.
+  * `Class:DataGeneration` can have instances where a single sample has multiple data files that need processed together during WorkflowExecution.  
+  * Relationships between samples and the data objects will be captured using `slot:part_of`, linking the 'child' back to the 'parent'.
+
+### Class:WorkflowExecution & Class:WorkflowChain
+* `Class:WorkflowExecution` will be a subclass of `Class:PlannedProcess`
+* `Class:WorkflowExecution` will replace `Class:WorkflowExecutionActivity` and will be a subclass of `Class:PlannedProcess`
+* To group subclasses of `Class:WorkflowExecution` into implemented steps, `Class:WorkflowChain` was created.
+  * This also provides clear connection back to the DataGeneration output, using `was_informed_by`
+    * `was_informed_by` is being removed from `Class:WorkflowExecution`
+  * Any implementation of `Class:WorkflowExecution` will use `Class:WorkflowChain`. Even when it's a single linked chain (ie implementation of a single subclass of `Class:WorkflowExecution`)
+* `Class:WorkflowExecution` will provide `part_of` with `Range:Class:WorkflowChain`
+  * This replaces 'used' and will no longer be on `Class:WorkflowExecution`
+* Output from `WorkflowExecution` will always have a value on `slot:data_category` of `processed_data` of `Class:DataObject`
+* `slot:analyte_category` will be used to inform the type of data that is being generated and can be instantiated on `Class:DataGeneration` and `Class:WorkflowChain`
+  * `slot:omics_type` will be removed and replaced with `slot:analyte_category` to denote the type of data generated.
+* `slot:gold_analysis_project_identifiers` and `slot:jgi_analysis_project_identifiers` will be included on `Class:WorkflowChain`
+
+### Other
+* `Class:DataObject` will NOT have subclasses. Rather, it will have `slot:data_category` which has a range `DataCategoryEnum`
+* `slot:data_object_type` will be renamed to `slot:data_object_category`. 
+* Slots associated with `Class:Biosample` or `Class:OmicsProcessing` that were incorrectly associated, were removed and added to more appropriate classes. (ex: `slot:pcr_primers` was removed from `Class:Biosample` and added to the new `Class:LibraryPrep`)
+* `Class:Reaction` will renamed to `Class:BiochemicalReaction` for clarity between this class and `Class:ChemicalConversionProcess`.
+* Subclasses will have their own typecodes.
+  * This may mean that eventually, if classes are renamed, typecodes might not make sense.
+* Reciprocal or redundant relationship slots (`has_part` + `part_of`, `was_informed_by` + `used`) will not be instantiated. Only a single direction should be provided.
+* All classes should have `slot:type` with `Range:CURIE` and `designates_type = True`
+  * `slot:type` should have a range of the class they are.
+
+### In Progress
+* When/how will instances of Class:WorkflowChain be created?
+  * By the workflows team, or by NMDC runtime?
+
+## Context and Problem Statement
+
+The NMDC schema was originally built providing one place, Class:OmicsProcessing, for capturing a wide variety of metadata fields. With expansion of data types and required metadata fields it was identified as being overloaded and requiring additional modeling.
+
+This ADR provides the decisions that were made leading up to and during the refactoring process to better capture metadata and describe samples and analyses.
+
+## Decision Drivers
+
+* Subject matter experts, schema leaders, and modelers met and discussed all decisions above. 
+* Need for expanded and improved modeling
+
+### Consequences
+
+* Major data migration required
+* Workflow updates required
+* Data portal ingest code updates required
+* Improves data representation
+* Improves workflows and automation
+
+## More Information
+