enable QCEvaluation to include path to images

[saskiad]

QC evaluation should be a list of:

• metric name
• value
• (value unit)
• path to related image

I'm struggling on a name for this entity. There might be multiple metrics associated with each image

[saskiad]

maybe also an optional description for the metric

[dbirman]

I started trying to tweak this so it would make sense to me but there's some design decisions I'm not sure about. I think it makes sense for metrics to be a list, and for each metric to have its own data, but it would be nice if the value field didn't have to be AindGenericType so that you don't have to pass it a dictionary -- but restricting the types also seems potentially problematic. Thoughts?

class QCMetric(BaseModel):
    """Description of a single quality control metric"""
    name: str = Field(title="Metric name")
    value: Union[str, bool, int, float] = Field(title="Metric value")
    description: Optional[str] = Field(None, title="Metric description")
    references: List[str] = Field(title="Metric reference URLs")


class QCEvaluation(AindModel):
    """Description of one evaluation stage, with one or more metrics"""

    evaluation_modality: Modality.ONE_OF = Field(..., title="Modality")
    evaluation_stage: str = Field(..., title="Evaluation stage")
    evaluator_full_name: str = Field(..., title="Evaluator full name")
    evaluation_date: date = Field(..., title="Evaluation date")
    qc_metrics: List[QCMetric] = Field(title="QC metrics")
    stage_status: Status = Field(..., title="Stage status")
    notes: Optional[str] = Field(None, title="Notes")


class QualityControl(AindCoreModel):
    """Description of quality metrics for a data asset"""

    _DESCRIBED_BY_URL = AindCoreModel._DESCRIBED_BY_BASE_URL.default + "aind_data_schema/core/quality_metrics.py"
    describedBy: str = Field(_DESCRIBED_BY_URL, json_schema_extra={"const": _DESCRIBED_BY_URL})
    schema_version: Literal["1.0.0"] = Field("1.0.0")
    overall_status: Status = Field(..., title="Overall status")
    overall_status_date: date = Field(..., title="Date of status")
    evaluations: List[QCEvaluation] = Field(..., title="Evaluations")
    notes: Optional[str] = Field(None, title="Notes")

[dyf]

We could use typing.Any, although I don't know how that affects deserialization.

[dbirman]

In the first QC meeting we discussed restricting types to make it easier to have a front-end for manual annotations, but we could also allow typing.Any and just warn people that if they need to do manual annotations it needs to be a primitive or a list/dict/tuple/array of primitives

[saskiad]

I'm good with typing.Any. For the metadata schema, we need it to be flexible, and this is essentially what the AindGenericType did.
For the QC tools that our teams develop, we will want them to be specific about types when they define these metrics. But I think that has to happen in the front-end.

[dbirman]

Great I'll update that and get this PR open