Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/implement med halt tests for robust model evaluation #1170

Open
wants to merge 6 commits into
base: release/2.6.0
Choose a base branch
from

Conversation

chakravarthik27
Copy link
Collaborator

This pull request introduces several new classes and methods for transforming and running clinical and robustness tests. The changes primarily involve adding new classes for different types of tests and incorporating utility functions to handle question-answering prompts.

New Classes and Methods:

  • Added FCT and NOTA classes for clinical tests with methods for transforming and running tests on sample lists. (langtest/transform/clinical.py)
  • Introduced FCT, NOTA, and FQT classes for robustness tests, each with a transform method to modify sample lists. (langtest/transform/robustness.py)

Utility Functions:

  • Imported build_qa_input and build_qa_prompt functions to assist in constructing question-answering prompts. (langtest/transform/clinical.py)

@chakravarthik27 chakravarthik27 self-assigned this Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant