Implement Tool Use for Dynamic Template and Guideline Selection in format_text

[drankush]

Summary:

This issue proposes to enhance the format_text function in the VoxRad application by leveraging the tool use capabilities of LLMs. Currently, the format_text function requires user to select a template which is then used to structure the transcript. But LLM can be leveraged to select a template automatically if no user-selected template is available, and also can generate recommendations, if required. This enhancement will integrate tool use to make template selection and recommendation generation in a robust, dynamic, and context-aware manner.

Background:

The format_text function is responsible for formatting transcribed radiology reports using a user-selected chosen template. Currently, if no template is selected by the user, the LLM goes ahead and structures the report according to the instructions in system prompt. The application capability can be extended using function calling to analyse the transcript and select template (from files in template folder) for creating a structured report and later further analyse the structured report and fetch relevant guidelines (from files in guidelines folder) and generate recommendations required for the structured report. The objective is to improve this by introducing more refined, controlled, and dynamic functionality for template and guideline selection based on transcription content.

Proposed Enhancement:

1. Dynamic Template Selection:

   • If the user does not select a template through the dropdown, format_text should continue to use function calling to intelligently select the most suitable template from the "templates" directory (e.g., by analyzing the transcript content and available templates).
   • The _select_template function to be implemented to use the tool.
   • This will make selection contextually more accurate.

2. Intelligent Guideline Selection and Recommendation Generation:

   • After formatting the report, the application should use tool use to analyze the report content and determine whether clinical recommendations are necessary.
   • If recommendations are deemed necessary, the application should automatically select relevant guidelines (such as BIRADS.md, LIRADS.md etc.) from the "guidelines" directory using a function calling (_analyze_recommendation_needs).
   • The selected guidelines and the formatted report should then be passed to the LLM to generate tailored clinical recommendations.
   • The _generate_recommendations and _validate_guidelines can be implemented for this step.
   • The generated recommendations should be appended to the final report content.

3. Tool Use Implementation:

   • The functions (_select_template, _analyze_recommendation_needs, and _generate_recommendations) to be created to use function calling, which is essentially a form of tool use. This enhancement will simply integrate this more clearly and dynamically.
   • The main goal is to provide more dynamic template and guideline selection and use the most accurate and relevant template and guidelines with the transcription content as context.
   • The implementation may requires modifications to format_text and the existing helper functions.

Benefits:

• Improved Efficiency: An intelligent template selection, avoid users to manually select templates further increasing efficiency.
• Dynamic Recommendations: Intelligent selection of guidelines ensures that generated recommendations are more accurate, relevant to the specific findings in the report and follow user/institution preferred guidelines.
• Automation: Reduces the need for manual template and guideline selection, streamlining the report generation process.
• Extensibility: Simplifies the process of adding new templates and guidelines to the application.

Implementation Details:

• Modify the format_text function to integrate the _select_template, _analyze_recommendation_needs, and _generate_recommendations in a sequence using tool use, creating better control and dynamic output.
• Ensure that all modifications are thoroughly tested.
• Consider the following aspects in implementation:
  • Error Handling: Proper handling of API errors, file reading errors, missing guideline files etc.
  • Logging: Add appropriate logging to monitor the flow, especially in template and recommendation selection.
  • Status Updates: Update the UI status bar to notify users about each step (e.g., "Selecting template," "Analyzing recommendations," etc.).
  • Make sure that _validate_guidelines is incorporated to not break when guidelines are updated or removed.

Acceptance Criteria:

• format_text uses tool use to intelligently select report templates when no user selected template is present.
• format_text intelligently determines if recommendations are needed, and if yes, it uses tool use to select appropriate guidelines.
• The final report is formatted using the chosen template and, when appropriate, is appended with recommendations generated by using the chosen guidelines.
• Error handling is robust, preventing the application from crashing.
• UI provides adequate feedback to the user regarding the process of formatting, and recommendation generation.