Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

moved QC into current schema #999

Merged
merged 16 commits into from
Aug 13, 2024
44 changes: 44 additions & 0 deletions src/aind_data_schema/core/quality_metrics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
""" Schemas for Quality Metrics """

from __future__ import annotations

from datetime import date
from enum import Enum
from typing import List, Literal, Optional

from pydantic import Field

from aind_data_schema_models.modalities import Modality
from aind_data_schema.base import AindCoreModel, AindGeneric, AindGenericType, AindModel


class Status(Enum):
saskiad marked this conversation as resolved.
Show resolved Hide resolved
"""QC Status"""

FAIL = "Fail"
PASS = "Pass"
UNKNOWN = "Unknown"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need unknown? I think this it is trying to fill a role like "comment" in on a github PR, where you want to make a note but don't want to pass judgement.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect we're going to add a bunch of other options to this.



class QCEvaluation(AindModel):
"""Description of one evaluation stage"""

evaluation_modality: Modality.ONE_OF = Field(..., title="Modality")
evaluation_stage: str = Field(..., title="Evaluation stage")
evaluator_full_name: str = Field(..., title="Evaluator full name")
evaluation_date: date = Field(..., title="Evaluation date")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

date or datetime?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think date.

qc_metrics: AindGenericType = Field(AindGeneric(), title="QC metrics")
stage_status: Status = Field(..., title="Stage status")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this a required field, do we want an Unknown option? Will it always be Pass/Fail?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you'll need to duke that out with @dyf - he removed the Unknown option. My hunch is that we're going to end up with a handful of Status options very quickly, but I don't think we want one to be Unknown.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My worry is, that since it's required, if we ever upgrade old metadata records, we'll just fall back to using QCEvaluation.model_construct() everywhere, which is frustrating.

notes: Optional[str] = Field(None, title="Notes")


class QualityMetrics(AindCoreModel):
saskiad marked this conversation as resolved.
Show resolved Hide resolved
"""Description of quality metrics for a data asset"""

_DESCRIBED_BY_URL = AindCoreModel._DESCRIBED_BY_BASE_URL.default + "aind_data_schema/core/quality_metrics.py"
describedBy: str = Field(_DESCRIBED_BY_URL, json_schema_extra={"const": _DESCRIBED_BY_URL})
schema_version: Literal["0.0.1"] = Field("0.0.1")
overall_status: Status = Field(..., title="Overall status")
overall_status_date: date = Field(..., title="Date of status")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

date or datetime?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think date.

evaluations: List[QCEvaluation] = Field(..., title="Evaluations")
notes: Optional[str] = Field(None, title="Notes")
41 changes: 41 additions & 0 deletions tests/test_quality_metrics.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""test quality metrics """

import datetime
import unittest

from pydantic import ValidationError

from aind_data_schema_models.modalities import Modality
from aind_data_schema.core.quality_metrics import QualityMetrics, QCEvaluation


class QualityMetricsTests(unittest.TestCase):
"""test quality metrics schema"""

def test_constructors(self):
"""testing constructors"""

with self.assertRaises(ValidationError):
q = QualityMetrics()

now = datetime.datetime.now()
saskiad marked this conversation as resolved.
Show resolved Hide resolved

q = QualityMetrics(
overall_status_date=now.date(),
overall_status="Pass",
evaluations=[QCEvaluation(
evaluator_full_name="Bob",
evaluation_date=now.date(),
evaluation_modality=Modality.ECEPHYS,
evaluation_stage="Spike sorting",
qc_metrics={"number_good_units": [622]},
stage_status="Pass",
),
],
)

assert q is not None


if __name__ == "__main__":
unittest.main()
Loading