Sort mustSupports, compare DRs with direct reference codes (#300)

* Update comparison script Sort mustSupports, compare DRs with direct reference codes Direct code instead of valueset in console log January2024 -> September2023 Connectathon files add .gitkeep, fix parsing Add to .gitignore Ignore codeFilter if it is a value type * Add check for resourceType primaryCodePath * Fix lint error --------- Co-authored-by: lmd59 <laurend@mitre.org>
projecttacoma · May 1, 2024 · 44e1522 · 44e1522
1 parent 8274f2c
commit 44e1522
Show file tree

Hide file tree

Showing 8 changed files with 134 additions and 25 deletions.
diff --git a/.gitignore b/.gitignore
@@ -14,5 +14,6 @@ regression/ecqm-content-r4-2021
 regression/ecqm-content-qicore-2022
 data-requirements/fqm-e-dr/*
 data-requirements/jan-2024-connectathon/*
+data-requirements/sept-2023-connectathon/*
 data-requirements/elm-parser-dr/*
 data-requirements/elm-parser-for-ecqms
diff --git a/data-requirements/README.md b/data-requirements/README.md
@@ -4,11 +4,11 @@ This directory includes scripts for comparing the data-requirements output of [f
 
 ## Getting Data Requirements from the elm-parser-for-ecqms
 
-The scripts in this directory will get the data requirements output from the elm-parser-for-ecqms fhir_review branch for the January 2024 Connectathon bundles. On the fhir_review branch of elm-parser-for-ecqms, data-requirements are calculated for the measures in [elm-parser-for-ecqms/measures/qicore/measures](https://github.com/projecttacoma/elm-parser-for-ecqms/tree/fhir_review/measures/qicore/measures) by running the command `ruby parse_elm.rb --bundle qicore`. The results are outputted to JSON files per measure to `elm-parser-for-ecqms/data_requirements/library`. This is all done by the script and the results are moved to the `elm-parser-dr` directory.
+The scripts in this directory will get the data requirements output from the elm-parser-for-ecqms fhir_review branch for the September 2023 Connectathon bundles. On the fhir_review branch of elm-parser-for-ecqms, data-requirements are calculated for the measures in [elm-parser-for-ecqms/measures/qicore/measures](https://github.com/projecttacoma/elm-parser-for-ecqms/tree/fhir_review/measures/qicore/measures) by running the command `ruby parse_elm.rb --bundle qicore`. The results are outputted to JSON files per measure to `elm-parser-for-ecqms/data_requirements/library`. This is all done by the script and the results are moved to the `elm-parser-dr` directory.
 
 ## Getting Data Requirements from fqm-execution
 
-The scripts in this directory will get the data requirements output from fqm-execution for the January 2024 Connectathon bundles. The data requirements are calculated with fqm-execution on every run for ease of testing changes to fqm-execution. Since the January 2024 Connectathon bundles are not in a GitHub repository, they will have to be manually dropped into the `jan-2024-connectathon` directory that is empty. The data requirements JSON output files will be moved to the `fqm-e-dr` directory after calculation.
+The scripts in this directory will get the data requirements output from fqm-execution for the September 2023 Connectathon bundles. The data requirements are calculated with fqm-execution on every run for ease of testing changes to fqm-execution. Since the September 2023 Connectathon bundles are not in a GitHub repository, they will have to be manually dropped into the `sept-2023-connectathon` directory that is empty. The data requirements JSON output files will be moved to the `fqm-e-dr` directory after calculation.
 
 ## Comparing Data Requirements
 
@@ -18,7 +18,7 @@ Right now there are two ways to compare data-requirements. `compare.sh` takes a
 
 ## Running the Scripts
 
-Before running any of the scripts, be sure to populate the `jan-2024-connectathon` directory with the corresponding measure bundles. Also confirm that additional dependencies required for these scripts are installed with `npm install`.
+Before running any of the scripts, be sure to populate the `sept-2023-connectathon` directory with the corresponding measure bundles. Also confirm that additional dependencies required for these scripts are installed with `npm install`.
 
 To run `compare.sh`:
 

diff --git a/data-requirements/compare.sh b/data-requirements/compare.sh
@@ -28,7 +28,7 @@ done
 echo "Gathering data-requirements output from the fhir_review branch of elm-parser-for-ecqms"
 
 # Clone the elm-parser-for-ecqms in the data-requirements directory if it hasn't been, swtich to the fhir_review branch,
-# run parse_elm.rb with --bundle qicore to get data-requirements for measure bundles from the January 2024 Connectathon
+# run parse_elm.rb with --bundle qicore to get data-requirements for measure bundles from the September 2023 Connectathon
 if [ ! -d "elm-parser-for-ecqms" ]; then
     git clone https://github.com/projecttacoma/elm-parser-for-ecqms.git
     git fetch --all
@@ -47,7 +47,7 @@ if [ ! -d "elm-parser-for-ecqms" ]; then
     mv "$SOURCE_DIR"/* "$TARGET_DIR"
 fi
 
-echo "Gathering data-requirements output from fqm-execution using the measure bundles from the January 2024 Connectathon"
+echo "Gathering data-requirements output from fqm-execution using the measure bundles from the September 2023 Connectathon"
 
 npx ts-node fqm-e-dr.ts
 

diff --git a/data-requirements/fqm-e-dr.ts b/data-requirements/fqm-e-dr.ts
@@ -6,10 +6,10 @@ const RESET = '\x1b[0m';
 const FG_YELLOW = '\x1b[33m';
 const FG_GREEN = '\x1b[32m';
 
-const JAN_2024_CONNECTATHON_BASE_PATH = path.join(__dirname, './jan-2024-connectathon');
+const SEPT_2023_CONNECTATHON_BASE_PATH = path.join(__dirname, './sept-2023-connectathon');
 
 /**
- * The purpose of this function is to go through all of the measure bundles in the jan-2024-connectathon
+ * The purpose of this function is to go through all of the measure bundles in the sept-2023-connectathon
  * directory, calculate their data-requirements, and output their data-requirements to a JSON file
  * corresponding to the name of the measure in the fqm-e-dr directory
  */
@@ -26,13 +26,13 @@ async function main() {
     fs.mkdirSync('./fqm-e-dr');
   }
 
-  // get all of the file names (short and fullPath) from the jan-2024-connectathon directory
+  // get all of the file names (short and fullPath) from the sept-2023-connectathon directory
   const allBundles = fs
-    .readdirSync(JAN_2024_CONNECTATHON_BASE_PATH)
+    .readdirSync(SEPT_2023_CONNECTATHON_BASE_PATH)
     .filter(f => !f.startsWith('.'))
     .map(f => ({
-      shortName: f.split('v')[0],
-      fullPath: path.join(JAN_2024_CONNECTATHON_BASE_PATH, f)
+      shortName: f.split('.')[0],
+      fullPath: path.join(SEPT_2023_CONNECTATHON_BASE_PATH, f)
     }));
 
   for (const bundle of allBundles) {

diff --git a/data-requirements/primary-code-paths.json b/data-requirements/primary-code-paths.json
@@ -0,0 +1,67 @@
+{
+  "AdverseEvent": "event",
+  "AllergyIntolerance": "code",
+  "BodyStructure": "location",
+  "CarePlan": "category",
+  "CareTeam": "category",
+  "Claim": "type",
+  "Communication": "reasonCode",
+  "CommunicationNotDone": "reasonCode",
+  "CommunicationRequest": "category",
+  "Condition": "code",
+  "Coverage": "type",
+  "Device": "type",
+  "DeviceNotRequested": "code",
+  "DeviceRequest": "code",
+  "DeviceUseStatement": "device.type",
+  "DiagnosticReportLab": "code",
+  "DiagnosticReportNote": "code",
+  "Encounter": "type",
+  "FamilyMemberHistory": "relationship",
+  "Flag": "code",
+  "Goal": "category",
+  "ImagingStudy": "procedureCode",
+  "Immunization": "vaccineCode",
+  "ImmunizationEvaluation": "targetDisease",
+  "ImmunizationNotDone": "vaccineCode",
+  "ImmunizationRecommendation": "recommendation.vaccineCode",
+  "Location": "type",
+  "Medication": "code",
+  "MedicationAdministration": "medication",
+  "MedicationAdministrationNotDone": "medication",
+  "MedicationDispense": "medication",
+  "MedicationDispenseNotDone": "medication",
+  "MedicationNotRequested": "medication",
+  "MedicationRequest": "medication",
+  "MedicationStatement": "medication",
+  "Observation": "code",
+  "ObservationNotDone": "code",
+  "Organization": "type",
+  "PractitionerRole": "code",
+  "Procedure": "code",
+  "ProcedureNotDone": "code",
+  "Questionnaire": "name",
+  "RelatedPerson": "relationship",
+  "ServiceNotRequested": "code",
+  "ServiceRequest": "code",
+  "Specimen": "type",
+  "Substance": "code",
+  "Task": "code",
+  "TaskNotDone": "code",
+  "USCoreImplantableDeviceProfile": "type",
+  "USCoreLaboratoryResultObservationProfile": "code",
+  "USCorePediatricBMIforAgeObservationProfile": "code",
+  "USCorePediatricWeightForHeightObservationProfile": "code",
+  "USCorePulseOximetryProfile": "code",
+  "USCoreSmokingStatusProfile": "code",
+  "observation-bmi": "code",
+  "observation-bodyheight": "code",
+  "observation-bodytemp": "code",
+  "observation-bodyweight": "code",
+  "observation-bp": "code",
+  "observation-headcircum": "code",
+  "observation-heartrate": "code",
+  "observation-oxygensat": "code",
+  "observation-resprate": "code",
+  "observation-vitalspanel": "code"
+}
diff --git a/data-requirements/sept-2023-connectathon/.gitkeep b/data-requirements/sept-2023-connectathon/.gitkeep
diff --git a/data-requirements/summary-compare.sh b/data-requirements/summary-compare.sh
@@ -33,7 +33,7 @@ done
 echo "Gathering data-requirements output from the fhir_review branch of elm-parser-for-ecqms"
 
 # Clone the elm-parser-for-ecqms in the data-requirements directory if it hasn't been, swtich to the fhir_review branch,
-# run parse_elm.rb with --bundle qicore to get data-requirements for measure bundles from the January 2024 Connectathon
+# run parse_elm.rb with --bundle qicore to get data-requirements for measure bundles from the September 2023 Connectathon
 if [ ! -d "elm-parser-for-ecqms" ]; then
     git clone https://github.com/projecttacoma/elm-parser-for-ecqms.git
     git fetch --all
@@ -52,7 +52,7 @@ if [ ! -d "elm-parser-for-ecqms" ]; then
     mv "$SOURCE_DIR"/* "$TARGET_DIR"
 fi
 
-echo "Gathering data-requirements output from fqm-execution using the measure bundles from the January 2024 Connectathon"
+echo "Gathering data-requirements output from fqm-execution using the measure bundles from the September 2023 Connectathon"
 
 npx ts-node fqm-e-dr.ts
 

diff --git a/data-requirements/summary-compare.ts b/data-requirements/summary-compare.ts
@@ -4,6 +4,7 @@ import path from 'path';
 
 const ELM_PARSER_DR_BASE_PATH = path.join(__dirname, './elm-parser-dr');
 const FQM_E_DR_BASE_PATH = path.join(__dirname, './fqm-e-dr');
+const primaryCodePaths = JSON.parse(fs.readFileSync('./primary-code-paths.json', 'utf8'));
 
 const measure = process.argv[2] ?? 'all';
 
@@ -132,27 +133,45 @@ async function main() {
 
       // get the data requirements from the elm-parser output that do not exist in the fqm-execution output
       // i.e. is there a data requirement in the fqm-execution output of the same type that has a codeFilter entry with
-      // a valueSet that is the same
-      const missingDRs = elmParserDRByKey.filter(dr =>
-        fqmEDRByKey.every(dr2 => dr.codeFilter?.every(cf => dr2.codeFilter?.every(cf2 => cf.valueSet !== cf2.valueSet)))
-      );
+      // a valueSet that is the same OR a code.code that is the same (same direct reference code)
+      const missingDRs = elmParserDRByKey
+        .filter(dr =>
+          fqmEDRByKey.every(dr2 =>
+            dr.codeFilter?.every(cf => dr2.codeFilter?.every(cf2 => cf.valueSet !== cf2.valueSet))
+          )
+        )
+        .filter(dr =>
+          fqmEDRByKey.every(dr2 =>
+            dr.codeFilter?.every(cf =>
+              dr2.codeFilter?.every(cf2 => cf.code?.every(c => cf2.code?.every(c2 => c.code !== c2.code)))
+            )
+          )
+        );
 
       // get the data requirements from the fqm-execution output that do not exist in the elm-parser output
-      const missingELMDRs = fqmEDRByKey.filter(dr =>
-        elmParserDRByKey.every(dr2 =>
-          dr.codeFilter?.every(cf => dr2.codeFilter?.every(cf2 => cf.valueSet !== cf2.valueSet))
+      const missingELMDRs = fqmEDRByKey
+        .filter(dr =>
+          elmParserDRByKey.every(dr2 =>
+            dr.codeFilter?.every(cf => dr2.codeFilter?.every(cf2 => cf.valueSet !== cf2.valueSet))
+          )
         )
-      );
+        .filter(dr =>
+          fqmEDRByKey.every(dr2 =>
+            dr.codeFilter?.every(cf =>
+              dr2.codeFilter?.every(cf2 => cf.code?.every(c => cf2.code?.every(c2 => c.code !== c2.code)))
+            )
+          )
+        );
 
       if (missingDRs.length === 0 && missingELMDRs.length === 0 && elmParserDRByKey.length === fqmEDRByKey.length) {
         console.log(`${FG_GREEN}%s${RESET}: data requirements of type ${key} match`, `PASS (${key})`);
       } else if (missingDRs.length > 0 || missingELMDRs.length > 0) {
         console.log(`${FG_RED}%s${RESET}: `, `FAIL (${key})`);
         if (missingDRs.length > 0) {
-          const missingvValueSets = missingDRs.map(dr => dr.codeFilter?.find(cf => cf.valueSet)?.valueSet);
+          const missingValueSets = missingDRs.map(dr => dr.codeFilter?.find(cf => cf.valueSet)?.valueSet);
           console.log(
             `fqm-execution is missing the data requirement of type ${key} for the following valuesets: ${JSON.stringify(
-              missingvValueSets
+              missingValueSets
             )}`
           );
         }
@@ -178,14 +197,19 @@ async function main() {
       // TO DO: maybe make this a flag
       elmParserDRByKey.forEach(dr => {
         const elmParserMustSupports = dr.mustSupport;
-        const elmParserValueSet = dr.codeFilter?.find(cf => cf.valueSet)?.valueSet;
+        // find the primary code path for the resourceType from primary-code-paths.json (result of parsing
+        // qicore-modelinfo-4.1.1.xml)
+        const primaryCodePath = primaryCodePaths[dr.type];
+
+        const elmParserValueSet = dr.codeFilter?.find(cf => cf.path === primaryCodePath)?.valueSet;
+        const elmParserDirectCode = dr.codeFilter?.find(cf => cf.path === primaryCodePath)?.code?.[0].code;
 
         if (elmParserValueSet) {
           const fqmEMatchMustSupports = fqmEDRByKey.find(dr =>
             dr.codeFilter?.some(cf => cf.valueSet === elmParserValueSet)
           )?.mustSupport;
 
-          const equalMustSupports = _.isEqual(elmParserMustSupports, fqmEMatchMustSupports);
+          const equalMustSupports = _.isEqual(elmParserMustSupports?.sort(), fqmEMatchMustSupports?.sort());
 
           if (!equalMustSupports) {
             console.log(`${FG_RED}%s${RESET}:`, `MUST SUPPORTS FAIL (${key}-${elmParserValueSet})`);
@@ -197,6 +221,23 @@ async function main() {
               `MUST SUPPORTS PASS (${key}-${elmParserValueSet})`
             );
           }
+        } else if (elmParserDirectCode) {
+          const fqmEMatchMustSupportsByCode = fqmEDRByKey.find(dr =>
+            dr.codeFilter?.some(cf => cf.code?.some(c => c.code === elmParserDirectCode))
+          )?.mustSupport;
+
+          const equalMustSupportsByCode = _.isEqual(elmParserMustSupports?.sort(), fqmEMatchMustSupportsByCode?.sort());
+
+          if (!equalMustSupportsByCode) {
+            console.log(`${FG_RED}%s${RESET}:`, `MUST SUPPORTS FAIL (${key}-${elmParserDirectCode})`);
+            console.log(`fqm-execution has the following mustSupports: ${fqmEMatchMustSupportsByCode ?? ''}`);
+            console.log(`elm-parser-for-ecqms has the following mustSupports: ${elmParserMustSupports}`);
+          } else {
+            console.log(
+              `${FG_GREEN}%s${RESET}: matching mustSupports`,
+              `MUST SUPPORTS PASS (${key}-${elmParserDirectCode})`
+            );
+          }
         }
       });
       console.log('\n');