MedChat won the Cohere RAG Hackathon held at the University of Waterloo! News article.
MedChat serves a dual purpose:
-
Querying Medical Literature: Using RAG om medical literature, MedChat allows easy interaction with medical literature, aiding healthcare providers in keeping up-to-date with the latest research findings, treatment protocols, and clinical guidelines.
-
AI-Powered Diagnosis Assistance: Beyond literature, MedChat integrates pre-trained medical models that assist in disease diagnosis. Doctors can submit MRI scans, X-rays and possibly other data, and MedChat can call other models to extract meaningful insights and diagnoses, offering a second opinion in medical diagnoses.
-
Utilizing Cohere API: The backbone of MedChat is the Cohere API, including Cohere Chat for natural language processing, Classify for classifying user intent, and Rerank for optimizing response relevance.
-
Frontend Development with Streamlit: We chose Streamlit for its simplicity.
-
Integrating Tensorflow for Running Medical Models: The core functionality of disease diagnosis is powered by medical models run on Tensorflow. This integration allows MedChat to process complex medical data and provide accurate, AI-driven diagnostic suggestions.
A primary challenge in developing MedChat was:
- Detecting User Intent Accurately: To ensure that MedChat responds appropriately to the queries of medical professionals, a critical task was to accurately detect user intent. This involved discerning whether a query was seeking medical literature or a diagnosis, and then triggering the correct functions within the application. We did this using Cohere Classify, which is not scalable. Now that Cohere API supports Function calling, that would be a much better tool.
![original](https://private-user-images.githubusercontent.com/20444505/286104220-b5a4bce0-c2ce-4429-bbba-59fe4d95cd29.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwNTcyNjEsIm5iZiI6MTczOTA1Njk2MSwicGF0aCI6Ii8yMDQ0NDUwNS8yODYxMDQyMjAtYjVhNGJjZTAtYzJjZS00NDI5LWJiYmEtNTlmZTRkOTVjZDI5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA4VDIzMjI0MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWI3MDc0MTNkNDhlMWI3YjFmOTQ0N2M0OTgxMTIyYmRiODNjYTUwODRmMzc3NGY1OTliNDVmMTcwYjFjZDYzYjgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.L5Yv44OJQY8OCIQJSQ63Xh1PRI2_zDqE7a_aUFhd-Js)
![original (1)](https://private-user-images.githubusercontent.com/20444505/286104162-5d6377e6-5f09-4628-97a0-ffa95c8f1bdf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwNTcyNjEsIm5iZiI6MTczOTA1Njk2MSwicGF0aCI6Ii8yMDQ0NDUwNS8yODYxMDQxNjItNWQ2Mzc3ZTYtNWYwOS00NjI4LTk3YTAtZmZhOTVjOGYxYmRmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA4VDIzMjI0MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTNiNzQ4NjBmOTg1ODA2YzVlYTY0ZTc0OGM3NzBmYmI1OTI1OWZlOWNjYmViNGM1OTNhMGI0Njc3OGE1Zjg0ZGEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.CMEAv6dFWDoV0BOA7ZXZCbh8j6-EAOxULo8fVJjM2qw)
![original (2)](https://private-user-images.githubusercontent.com/20444505/286104246-06894c8c-6bb3-4a38-9abb-5663852f4741.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwNTcyNjEsIm5iZiI6MTczOTA1Njk2MSwicGF0aCI6Ii8yMDQ0NDUwNS8yODYxMDQyNDYtMDY4OTRjOGMtNmJiMy00YTM4LTlhYmItNTY2Mzg1MmY0NzQxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA4VDIzMjI0MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZhNWI2YzAwYzI5YWRmNmQxNWRmZWExMGRhYTlmNGMyYjc5ZDkwMzdmZGFlMWQ5M2NiYTM3M2JhZjJjOWY5NDYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.iSb80SuRqepjN4-qBGQh7EtYS4QaN1Fx6WThT_F3e1U)