Support data validation using value and tolerance #370
Conversation
There was a problem hiding this comment.
As just discussed bilaterally, I think it would be a good idea to subclass DataValidationCriteria.
You'd make two classes UpperLowerCriteria(DataValidationCriteria) and ValuePlusToleranceCriteria(DataValidationCriteria) (or some better name that what I can come up with now).
UpperLowerCriteria would be mostly what DataValidationCritera is now with attributes upper_bound and lower_bound.
ValuePlusToleranceCriteria(DataValidationCriteria) would have value, rtol, and atol. It would also implement two properties upper_bound and lower_bound.
This way both classes can be used in the same way when validating data.
| @@ -21,6 +23,29 @@ class DataValidationCriteria(IamcDataFilter): | |||
|
|
|||
| upper_bound: float = None | |||
| lower_bound: float = None | |||
There was a problem hiding this comment.
| lower_bound: float = None | |
| lower_bound: float | None = None |
| rtol: float = None | ||
| atol: float = None | ||
|
|
||
| @model_validator(mode="before") |
There was a problem hiding this comment.
I think the option mode="before" is not needed here:
| @model_validator(mode="before") | |
| @model_validator() |
There was a problem hiding this comment.
No, seems to be required.
|
Thanks @phackstock, implemented the optional-field and don't allow extra fields anymore. For separating this out into separate classes for bounds and value/tolerance, the issue is that this yields a very user-unfriendly error message if a validation-item has both value and bounds. I implemented this on the branch https://github.com/danielhuppmann/nomenclature/tree/feature/validate-range-subclasses, maybe you have an idea on how to generate user-intelligible error messages? |
|
@danielhuppmann, just took a look at your extra branch, implementation wise that's exactly what I meant. |
|
closing in favor of #371 |
This PR implements support for configuring validation criteria using value and atol/rtol (optional). It also adds some safeguards that only either bounds or value (and tolerance) can be configured in a DataValidationCriteria instance. And adds a few tests.