Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect duplicate code errors #314

Merged

Conversation

phackstock
Copy link
Contributor

@phackstock phackstock commented Jan 22, 2024

Closes #104, closes #312.

Instead of reporting duplicate code errors one by one we now collect them and raise them as one single error. For better readability the individual errors are indented and numbered.
From previously reporting:

ValueError: Duplicate item in variable codelist: Secondary Energy|Liquids

we now have:

ValueError: Collected 529 errors:
  1. Duplicate item in variable codelist: Secondary Energy|Liquids
  2. Duplicate item in variable codelist: Secondary Energy|Liquids|Biomass
  3. Duplicate item in variable codelist: Secondary Energy|Liquids|Electricity
  4. Duplicate item in variable codelist: Secondary Energy|Liquids|Other
  5. Duplicate item in variable codelist: Secondary Energy|Solids
  6. Duplicate item in variable codelist: Secondary Energy|Solids|Biomass
  7. Duplicate item in variable codelist: Secondary Energy|Solids|Fossil
  8. ...

Update 24.01.2024 10:09

Since IAMconsortium/legacy-definitions#1 has already been resolved, I had to also address #311.
The current way the test test_multiple_external_repos is configured is not the most elegant way but it should be ok.
It tests for the following:

  1. Are the two external repos correctly parsed from the yaml file
  2. Are the two external repos correctly fetched from GitHub
  3. Is the overall size of the resulting variable list (common + legacy definitions) bigger than 2000 entries.

The reason I opted for 3. instead of testing for the presence of specific variables is that we expect to migrate variables from legacy to common definitions. This way it could happen that at some point all the variables that I would test for to prove that both legacy and common definitions are imported are now wholly in common-definitions.
That's why I opted to check for the overall size of the code list. Also not 100% future proof but good enough for now I'd say.

@phackstock phackstock added the enhancement New feature or request label Jan 22, 2024
@phackstock phackstock self-assigned this Jan 22, 2024
@phackstock phackstock changed the title Feature/collect duplicate code errors Collect duplicate code errors Jan 22, 2024
@phackstock phackstock marked this pull request as ready for review January 22, 2024 16:57
@phackstock phackstock force-pushed the feature/collect-duplicate-code-errors branch from e982308 to 31bc11c Compare January 23, 2024 12:59
Copy link
Member

@danielhuppmann danielhuppmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@phackstock phackstock merged commit 320b021 into IAMconsortium:main Jan 24, 2024
8 checks passed
@phackstock phackstock deleted the feature/collect-duplicate-code-errors branch January 24, 2024 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix multiple repo test Raise all duplicate-errors during initialization
2 participants