🌲 Made for TreeHacks 2025. Devpost · GitHub
Every day, teachers step into classrooms filled with students who have different personalities, learning styles, and challenges. Some students are eager to participate, others are easily distracted, and a few might resist authority altogether. A teacher's ability to navigate these interactions can mean the difference between an engaging classroom and a chaotic one.
Yet, there are few opportunities for educators to practice classroom management in a realistic, risk-free setting. So, we asked ourselves: What if teachers could train for the classroom the way pilots train in flight simulators? Could we create an AI-powered environment where educators could interact with dynamic student personalities, receive real-time feedback, and refine their teaching techniques so that they can accel in the classroom?
That’s how we built engagED—an interactive classroom simulation that prepares educators for real-world teaching challenges. By leveraging AI-bots that simulate student behaviors and a performance analytics dashboard, engagED helps teachers develop strong communication skills, manage classroom dynamics, and build confidence—all in a safe, controlled environment.
We believe that if engagED is implemented in teacher training programs and professional development workshops, it could revolutionize how educators prepare for the modern classroom—leading to more engaged students, less burnout, and stronger learning outcomes across the board.
engagED simulates a classroom environment where teachers interact with AI-powered student personalities, each with unique behaviors, engagement levels, and challenges. The platform allows educators to practice managing real-world classroom dynamics—whether it’s handling a disruptive student, encouraging a quiet learner to participate, or maintaining engagement during a lesson.
engagED responds to natural teacher interactions in a virtual classroom setting, adjusting student behaviors based on teaching strategies. Just like in a real class, students may ask unexpected questions, lose focus, or react differently depending on the teacher’s approach. At the end of each session, engagED provides a performance dashboard, offering insights into key performance metrics and providing feedback for improvement.
engagED was built by a mix of hackers from various backgrounds in frontend and backend roles. We began by wireframing and designing our user workflows, and then iterated upon these designs to create a seamless experience. We used a variety of cutting-edge technologies in this platform, which are outlined below.
Our tech flow
The frontend was build in React and Typescript using Next.js as our frontend framework in order to maintain a structured codebase and fast loading times. The backend server was built in Python and FastAPI, allowing us to utilize a variety APIs such as ElevenLabs, OpenAI, Groq, and LangChain. A websocket connection was also utilized between the frontend and backend in order to constantly stream audio data to the agents and LLMs, allowing for low latency. The database was created in MongoDB in order to store user sessions in a structured manner.
Client logic flow
Diving deeper into our client logic, there are a lot of moving parts that bring engagED together! We begin with the teacher user instructing the virtual class, and the AI students listening in. We then utilize OpenAI's Whisper API in concurrence with Groq in order to convert this instruction to a text transcript. We then feed this transcript back into the AI Agents in order to provide them context and prompt them to chime in if neccesary. Each agent was build with LangChain Agents and Groq to be able to respond to conversation with low latency. Initially, we ran each agent in parallel, but soon realized that this would cause them to talk over one another as they responded to the teacher. Thus, we implemented a round-robin algorithm between each agent and the teacher in order to minimize conflicts.
Then, we utilize ElevenLabs' text to speech API in order to convert the agent's conversational input into speech, and played this in the virtual classroom setting. The speech from the teacher is now combined with the latest transcripts and messages from the AI Agents to create a master transcript. This is now used as the context provided to the AI Agents for them to respond to the teacher.
Once the session is ended, the master transcript is sent to OpenAI to summarize and provide analytics data for teachers. This is then displayed to the teachers in the following page. The data is then stored with their unique session id in MongoDB for retrieval and visibility later.
Our tech stack
Zoom API Issues: We attempted to use the Zoom API to programmatically start meetings with Zoom bots. However, after numerous attempts and insights from Zoom mentors, we realized that the Meeting Bot approach was not viable due to API limitations and planned deprecation. We ended up building our own solution for streaming web and audio to the backend and adding having agents receive this context.
Low-latency Interaction: We had to optimize AI interactions to maintain near-real-time responsiveness, ensuring that the virtual classroom felt immersive and natural. We attempted to achieve low-inference at every step possible using the Groq API to speed up Speech-To-Text and Agentic LLM reasoning. We found ElevenLabs API to be quite fast for Text-to-Speech.
Agent Coordination: AI student agents needed to understand each other's context and avoid talking over one another. Synchronization and maintaining conversational order proved to be a complex challenge. We used a mixture probabilistic activations for each agent, a cooldown period specific to each agent, and locks to ensure a clean virtual classroom experience.
Analytics APIs: We wanted to utilize a dedicated API in order to give users analytics on their sessions, but had a hard time finding options that were either sufficiently documented or free to use for this task. As such, we decided to utilize OpenAI's capabilities in order to analyze our transcripts.
Audio Formats: Both the Web MediaRecorder API and the audio-generation services have limited information on how it streams "chunks" of audio. For example, one API returned base64-encoded data that consistently started with the same few octets, but looking them up as magic numbers turned up dry. We ultimately discovered that these chunks had to be concatenated into one single audio file—only the first chunk had the necessary header data needed to make the file playable. However, this was not feasible for generating instant responses and transcripts in response to the user's microphone audio. Our initial solution for this case was to accumulate chunks, but this led to large contexts for the AI models. Our breakthrough came when we used a deprecated method from the Web Audio API to collect raw audio samples, and manually formatted each chunk as a WAV file so that each chunk could be enjoyed in isolation.
There are many things we are proud of:
- Developing Multi-Agent AI Personalities – We successfully implemented multiple AI-powered student personas that react uniquely to teacher input, creating a diverse and realistic classroom dynamic.
- Achieving Real-Time, Low-Latency AI Conversations – By leveraging Groq for rapid inference and optimizing our WebSocket connections, we were able to ensure smooth and natural interactions between teachers and AI students.
- Building an AI-Driven Performance Analytics Dashboard – We implemented a system that provides teachers with personalized feedback, including engagement scores, response effectiveness, and classroom management insights.
- Creating a Structured Data Storage System – By designing a scalable MongoDB database, we ensured that teachers can access their past sessions, review feedback, and track their progress over time.
- Creating an interactive UI/UX thats super aesthetically pleasing!
We learned many things:
- We learned that achieving real-time, natural AI interaction is challenging, especially when handling multiple AI agents in parallel. Implementing Groq for low-latency processing and refining our round-robin approach helped us create smoother conversations.
- Striking the right balance between accurate student simulations and efficient AI processing was crucial. Overloading the system with complex, simultaneous AI interactions led to chaotic responses, which we mitigated through probabilistic agent activations.
- Integrating Whisper AI for speech recognition and ElevenLabs for AI-generated speech gave us firsthand experience in handling continuous audio streams with minimal delay. We had to refine our approach to ensure clarity and prevent misinterpretations.
- Since each training session generates a large amount of structured and unstructured data, we learned best practices for storing, retrieving, and analyzing this data efficiently using MongoDB.
- While AI can provide powerful insights, we realized that clear, intuitive dashboards and actionable feedback are essential for users to gain value from the system. Simplifying the UI/UX to present feedback in a digestible format was a major learning experience.
There are many aspects of engagED that we could improve:
- Using a real-world dataset of teacher-student interactions to fine-tune the LLM model would be beneficial to getting more realistic practice for teachers.
- Leveraging video analytics to understand expressions and hand-movements of the teacher would be useful for giving them more insightful feedback.
- Implementing visual-language models to understand content that is shared via "screenshare" could help provide useful context for the LLM when it is generating feedback.
- Creating a feature for teachers to customize the student persona.
- Having more agents in the meeting & figuring out how to better implement student-student interactions as well to best simulate the classroom enviorment.
For the backend:
-
Setup
.env. -
Install dependencies.
$ cd backend/ $ pip install -r requirements.txt -
Then start the backend server:
$ uvicorn main:app --reload
For the frontend:
-
Install dependencies.
$ cd client/ $ npm install -
Start the Next.js development server:
$ npm run dev