diff --git a/02-explore-agentic-frameworks/README.md b/02-explore-agentic-frameworks/README.md index 19932194..5c3ffd8b 100644 --- a/02-explore-agentic-frameworks/README.md +++ b/02-explore-agentic-frameworks/README.md @@ -4,14 +4,14 @@ AI agent frameworks are software platforms designed to simplify the creation, de These frameworks help developers focus on the unique aspects of their applications by providing standardized approaches to common challenges in AI agent development. They enhance scalability, accessibility, and efficiency in building AI systems. -## Introduction +## Introduction This lesson will cover: - What are AI Agent Frameworks and what do they enable developers to do? - How can teams use these to quickly prototype, iterate, and improve my agent’s capabilities? -- What are the differences between the frameworks and tools created by Microsoft AutoGen, Semantic Kernel, and Azure AI Agent +- What are the differences between the frameworks and tools created by Microsoft AutoGen, Semantic Kernel, and Azure AI Agent - Can I integrate my existing Azure ecosystem tools directly, or do I need standalone solutions? - What is Azure AI Agents service and how is this helping me? @@ -29,13 +29,13 @@ The goals of this lesson is to help you understand: Traditional AI Frameworks can help you integrate AI into your apps and make these apps better in the following ways: - **Personalization**: AI can analyze user behavior and preferences to provide personalized recommendations, content, and experiences. -Example: Streaming services like Netflix use AI to suggest movies and shows based on viewing history, enhancing user engagement and satisfaction. + Example: Streaming services like Netflix use AI to suggest movies and shows based on viewing history, enhancing user engagement and satisfaction. - **Automation and Efficiency**: AI can automate repetitive tasks, streamline workflows, and improve operational efficiency. -Example: Customer service apps use AI-powered chatbots to handle common inquiries, reducing response times and freeing up human agents for more complex issues. + Example: Customer service apps use AI-powered chatbots to handle common inquiries, reducing response times and freeing up human agents for more complex issues. - **Enhanced User Experience**: AI can improve the overall user experience by providing intelligent features such as voice recognition, natural language processing, and predictive text. -Example: Virtual assistants like Siri and Google Assistant use AI to understand and respond to voice commands, making it easier for users to interact with their devices. + Example: Virtual assistants like Siri and Google Assistant use AI to understand and respond to voice commands, making it easier for users to interact with their devices. -### That all sounds great right, so why we do we need AI Agent Framework? +### That all sounds great right, so why do we need AI Agent Framework? AI Agent frameworks represent something more than just AI frameworks. They are designed to enable the creation of intelligent agents that can interact with users, other agents, and the environment to achieve specific goals. These agents can exhibit autonomous behavior, make decisions, and adapt to changing conditions. Let's look at some key capabilities enabled by AI Agent Frameworks: @@ -68,7 +68,7 @@ Frameworks like LangChain and Microsoft Semantic Kernel offer pre-built componen // Semantic Kernel example ChatHistory chatHistory = []; -chatHistory.AddUserMessage("I'd like to go To New York"); +chatHistory.AddUserMessage("I'd like to go to New York"); // Define a plugin that contains the function to book travel public class BookTravelPlugin( @@ -175,7 +175,7 @@ What you see in the previous code is how you can create a task that involves mul ### Learn in Real-Time -Advanced frameworks provide capabilities for real-time context understanding and adaptation. +Advanced frameworks provide capabilities for real-time context understanding and adaptation. **How teams can use these**: Teams can implement feedback loops where agents learn from interactions and adjust their behavior dynamically, leading to continuous improvement and refinement of capabilities. @@ -187,7 +187,7 @@ There are many ways to compare these frameworks, but let's look at some key diff ## AutoGen -Open-source framework developed by Microsoft Research's AI Frontiers Lab. Focuses on event-driven, distributed *agentic* applications, enabling multiple LLMs and SLMs, tools, and advanced multi-agent design patterns. +Open-source framework developed by Microsoft Research's AI Frontiers Lab. Focuses on event-driven, distributed _agentic_ applications, enabling multiple LLMs and SLMs, tools, and advanced multi-agent design patterns. AutoGen is built around the core concept of agents, which are autonomous entities that can perceive their environment, make decisions, and take actions to achieve specific goals. Agents communicate through asynchronous messages, allowing them to work independently and in parallel, enhancing system scalability and responsiveness. @@ -197,111 +197,113 @@ AutoGen is built around the core concept of agents, which are autonomous entitie Here are some important core concepts of AutoGen: -- **Agents**. An agent is a software entity that: +- **Agents**. An agent is a software entity that: + - **Communicates via messages**, these messages can be synchronous or asynchronous. - **Maintains its own state**, which can be modified by incoming messages. - **Performs actions** in response to received messages or changes in its state. These actions may modify the agent’s state and produce external effects, such as updating message logs, sending new messages, executing code, or making API calls. - + Here you have a short code snippet in which you create your own agent with Chat capabilities: - ```python - from autogen_agentchat.agents import AssistantAgent - from autogen_agentchat.messages import TextMessage - from autogen_ext.models.openai import OpenAIChatCompletionClient - - - class MyAssistant(RoutedAgent): - def __init__(self, name: str) -> None: - super().__init__(name) - model_client = OpenAIChatCompletionClient(model="gpt-4o") - self._delegate = AssistantAgent(name, model_client=model_client) - - @message_handler - async def handle_my_message_type(self, message: MyMessageType, ctx: MessageContext) -> None: - print(f"{self.id.type} received message: {message.content}") - response = await self._delegate.on_messages( - [TextMessage(content=message.content, source="user")], ctx.cancellation_token - ) - print(f"{self.id.type} responded: {response.chat_message.content}") - ``` - - In the previous code, `MyAssistant` has been created and inherits from `RoutedAgent`. It has a message handler that prints the content of the message and then sends a response using the `AssistantAgent` delegate. Especially note how we assign to `self._delegate` an instance of `AssistantAgent` which is a pre-built agent that can handle chat completions. - - - Let's let AutoGen know about this agent type and kick off the program next: - - ```python - - # main.py - runtime = SingleThreadedAgentRuntime() - await MyAgent.register(runtime, "my_agent", lambda: MyAgent()) - - runtime.start() # Start processing messages in the background. - await runtime.send_message(MyMessageType("Hello, World!"), AgentId("my_agent", "default")) - ``` - - In the previous code the agents are registered with the runtime and then a message is sent to the agent resulting in the following output: - - ```text - # Output from the console: - my_agent received message: Hello, World! - my_assistant received message: Hello, World! - my_assistant responded: Hello! How can I assist you today? - ``` + ```python + from autogen_agentchat.agents import AssistantAgent + from autogen_agentchat.messages import TextMessage + from autogen_ext.models.openai import OpenAIChatCompletionClient -- **Multi agents**. AutoGen supports the creation of multiple agents that can work together to achieve complex tasks. Agents can communicate, share information, and coordinate their actions to solve problems more efficiently. To create a multi-agent system, you can define different types of agents with specialized functions and roles, such as data retrieval, analysis, decision-making, and user interaction. Let's see how such a creation looks like so we get a sense of it: - ```python - editor_description = "Editor for planning and reviewing the content." - - # Example of declaring an Agent - editor_agent_type = await EditorAgent.register( - runtime, - editor_topic_type, # Using topic type as the agent type. - lambda: EditorAgent( - description=editor_description, - group_chat_topic_type=group_chat_topic_type, - model_client=OpenAIChatCompletionClient( - model="gpt-4o-2024-08-06", - # api_key="YOUR_API_KEY", - ), - ), - ) + class MyAssistant(RoutedAgent): + def __init__(self, name: str) -> None: + super().__init__(name) + model_client = OpenAIChatCompletionClient(model="gpt-4o") + self._delegate = AssistantAgent(name, model_client=model_client) - # remaining declarations shortened for brevity - - # Group chat - group_chat_manager_type = await GroupChatManager.register( - runtime, - "group_chat_manager", - lambda: GroupChatManager( - participant_topic_types=[writer_topic_type, illustrator_topic_type, editor_topic_type, user_topic_type], - model_client=OpenAIChatCompletionClient( - model="gpt-4o-2024-08-06", - # api_key="YOUR_API_KEY", - ), - participant_descriptions=[ - writer_description, - illustrator_description, - editor_description, - user_description - ], - ), - ) - ``` + @message_handler + async def handle_my_message_type(self, message: MyMessageType, ctx: MessageContext) -> None: + print(f"{self.id.type} received message: {message.content}") + response = await self._delegate.on_messages( + [TextMessage(content=message.content, source="user")], ctx.cancellation_token + ) + print(f"{self.id.type} responded: {response.chat_message.content}") + ``` + + In the previous code, `MyAssistant` has been created and inherits from `RoutedAgent`. It has a message handler that prints the content of the message and then sends a response using the `AssistantAgent` delegate. Especially note how we assign to `self._delegate` an instance of `AssistantAgent` which is a pre-built agent that can handle chat completions. + + Let's let AutoGen know about this agent type and kick off the program next: + + ```python + + # main.py + runtime = SingleThreadedAgentRuntime() + await MyAgent.register(runtime, "my_agent", lambda: MyAgent()) + + runtime.start() # Start processing messages in the background. + await runtime.send_message(MyMessageType("Hello, World!"), AgentId("my_agent", "default")) + ``` - In the previous code we have a `GroupChatManager` that is registered with the runtime. This manager is responsible for coordinating the interactions between different types of agents, such as writers, illustrators, editors, and users. + In the previous code the agents are registered with the runtime and then a message is sent to the agent resulting in the following output: + + ```text + # Output from the console: + my_agent received message: Hello, World! + my_assistant received message: Hello, World! + my_assistant responded: Hello! How can I assist you today? + ``` + +- **Multi agents**. AutoGen supports the creation of multiple agents that can work together to achieve complex tasks. Agents can communicate, share information, and coordinate their actions to solve problems more efficiently. To create a multi-agent system, you can define different types of agents with specialized functions and roles, such as data retrieval, analysis, decision-making, and user interaction. Let's see how such a creation looks like so we get a sense of it: + + ```python + editor_description = "Editor for planning and reviewing the content." + + # Example of declaring an Agent + editor_agent_type = await EditorAgent.register( + runtime, + editor_topic_type, # Using topic type as the agent type. + lambda: EditorAgent( + description=editor_description, + group_chat_topic_type=group_chat_topic_type, + model_client=OpenAIChatCompletionClient( + model="gpt-4o-2024-08-06", + # api_key="YOUR_API_KEY", + ), + ), + ) + + # remaining declarations shortened for brevity + + # Group chat + group_chat_manager_type = await GroupChatManager.register( + runtime, + "group_chat_manager", + lambda: GroupChatManager( + participant_topic_types=[writer_topic_type, illustrator_topic_type, editor_topic_type, user_topic_type], + model_client=OpenAIChatCompletionClient( + model="gpt-4o-2024-08-06", + # api_key="YOUR_API_KEY", + ), + participant_descriptions=[ + writer_description, + illustrator_description, + editor_description, + user_description + ], + ), + ) + ``` + + In the previous code we have a `GroupChatManager` that is registered with the runtime. This manager is responsible for coordinating the interactions between different types of agents, such as writers, illustrators, editors, and users. - **Agent Runtime**. The framework provides a runtime environment, enabling communication between agents, manages their identities and lifecycles, and enforce security and privacy boundaries. This means that you can run your agents in a secure and controlled environment, ensuring that they can interact safely and efficiently. There are two runtimes of interest: + - **Stand-alone runtime**. This is a good choice for single-process applications where all agents are implemented in the same programming language and run in the same process. Here's an illustration of how it works: - - Stand-alone runtime -Application stack - *agents communicate via messages through the runtime, and the runtime manages the lifecycle of agents* + Stand-alone runtime + + Application stack + + *agents communicate via messages through the runtime, and the runtime manages the lifecycle of agents* - **Distributed agent runtime**, is suitable for multi-process applications where agents may be implemented in different programming languages and running on different machines. Here's an illustration of how it works: - + Distributed runtime ## Semantic Kernel + Agent Framework @@ -312,115 +314,116 @@ Let's first talk about the Semantic Kernel. It has the following core concepts: - **Connections**: This is an interface with external AI services and data sources. - ```csharp - using Microsoft.SemanticKernel; + ```csharp + using Microsoft.SemanticKernel; - // Create kernel - var builder = Kernel.CreateBuilder(); - - // Add a chat completion service: - builder.Services.AddAzureOpenAIChatCompletion( - "your-resource-name", - "your-endpoint", - "your-resource-key", - "deployment-model"); - var kernel = builder.Build(); - ``` + // Create kernel + var builder = Kernel.CreateBuilder(); - Here you have a simple example of how you can create a kernel and add a chat completion service. Semantic Kernel creates a connection to an external AI service, in this case, Azure OpenAI Chat Completion. + // Add a chat completion service: + builder.Services.AddAzureOpenAIChatCompletion( + "your-resource-name", + "your-endpoint", + "your-resource-key", + "deployment-model"); + var kernel = builder.Build(); + ``` + + Here you have a simple example of how you can create a kernel and add a chat completion service. Semantic Kernel creates a connection to an external AI service, in this case, Azure OpenAI Chat Completion. - **Plugins**: Encapsulate functions that an application can use. There are both ready-made plugins and plugins you can create yourself. There's a concept here called "Semantic functions". What makes it semantic is that you provide it semantic information that helps Semantic Kernel figure out that this function needs to be called. Here's an example: - ```csharp - var userInput = Console.ReadLine(); + ```csharp + var userInput = Console.ReadLine(); + + // Define semantic function inline. + string skPrompt = @"Summarize the provided unstructured text in a sentence that is easy to understand. + Text to summarize: {{$userInput}}"; - // Define semantic function inline. - string skPrompt = @"Summarize the provided unstructured text in a sentence that is easy to understand. - Text to summarize: {{$userInput}}"; - - // Register the function - kernel.CreateSemanticFunction( - promptTemplate: skPrompt, - functionName: "SummarizeText", - pluginName: "SemanticFunctions" - ); - ``` + // Register the function + kernel.CreateSemanticFunction( + promptTemplate: skPrompt, + functionName: "SummarizeText", + pluginName: "SemanticFunctions" + ); + ``` - Here, you first have a template prompt `skPrompt` that leaves room for the user to input text, `$userInput`. Then you register the function `SummarizeText` with the plugin `SemanticFunctions`. Note the name of the function that helps Semantic Kernel understand what the function does and when it should be called. + Here, you first have a template prompt `skPrompt` that leaves room for the user to input text, `$userInput`. Then you register the function `SummarizeText` with the plugin `SemanticFunctions`. Note the name of the function that helps Semantic Kernel understand what the function does and when it should be called. - **Native function**: There's also native functions that the framework can call directly to carry out the task. Here's an example of such a function retrieving the content from a file: - ```csharp - public class NativeFunctions { + ```csharp + public class NativeFunctions { - [SKFunction, Description("Retrieve content from local file")] - public async Task RetrieveLocalFile(string fileName, int maxSize = 5000) - { - string content = await File.ReadAllTextAsync(fileName); - if (content.Length <= maxSize) return content; - return content.Substring(0, maxSize); - } - } - - //Import native function - string plugInName = "NativeFunction"; - string functionName = "RetrieveLocalFile"; - - var nativeFunctions = new NativeFunctions(); - kernel.ImportFunctions(nativeFunctions, plugInName); - ``` + [SKFunction, Description("Retrieve content from local file")] + public async Task RetrieveLocalFile(string fileName, int maxSize = 5000) + { + string content = await File.ReadAllTextAsync(fileName); + if (content.Length <= maxSize) return content; + return content.Substring(0, maxSize); + } + } + + //Import native function + string plugInName = "NativeFunction"; + string functionName = "RetrieveLocalFile"; + + var nativeFunctions = new NativeFunctions(); + kernel.ImportFunctions(nativeFunctions, plugInName); + ``` - **Planner**: The planner orchestrates execution plans and strategies based on user input. The idea is to express how things should be carried out which then surveys as an instruction for Semantic Kernel to follow. It then invokes the necessary functions to carry out the task. Here's an example of such a plan: - ```csharp - string planDefinition = "Read content from a local file and summarize the content."; - SequentialPlanner sequentialPlanner = new SequentialPlanner(kernel); - - string assetsFolder = @"../../assets"; - string fileName = Path.Combine(assetsFolder,"docs","06_SemanticKernel", "aci_documentation.txt"); - - ContextVariables contextVariables = new ContextVariables(); - contextVariables.Add("fileName", fileName); - - var customPlan = await sequentialPlanner.CreatePlanAsync(planDefinition); - - // Execute the plan - KernelResult kernelResult = await kernel.RunAsync(contextVariables, customPlan); - Console.WriteLine($"Summarization: {kernelResult.GetValue()}"); - ``` - - Note especially `planDefinition` which is a simple instruction for the planner to follow. The appropriate functions are then called based on this plan, in this case our semantic function `SummarizeText` and the native function `RetrieveLocalFile`. -- **Memory**: Abstracts and simplifies context management for AI apps. The idea with memory is that this is something the LLM should know about. You can store this information in a vector store which ends up being an in-memory database or a vector database or similar. Here's an example of a very simplified scenario where *facts* are added to the memory: - - ```csharp - var facts = new Dictionary(); - facts.Add( - "Azure Machine Learning; https://learn.microsoft.com/azure/machine-learning/", - @"Azure Machine Learning is a cloud service for accelerating and - managing the machine learning project lifecycle. Machine learning professionals, - data scientists, and engineers can use it in their day-to-day workflows" - ); - - facts.Add( - "Azure SQL Service; https://learn.microsoft.com/azure/azure-sql/", - @"Azure SQL is a family of managed, secure, and intelligent products - that use the SQL Server database engine in the Azure cloud." - ); - - string memoryCollectionName = "SummarizedAzureDocs"; - - foreach (var fact in facts) { - await memoryBuilder.SaveReferenceAsync( - collection: memoryCollectionName, - description: fact.Key.Split(";")[1].Trim(), - text: fact.Value, - externalId: fact.Key.Split(";")[2].Trim(), - externalSourceName: "Azure Documentation" - ); - } - ``` + ```csharp + string planDefinition = "Read content from a local file and summarize the content."; + SequentialPlanner sequentialPlanner = new SequentialPlanner(kernel); + + string assetsFolder = @"../../assets"; + string fileName = Path.Combine(assetsFolder,"docs","06_SemanticKernel", "aci_documentation.txt"); + + ContextVariables contextVariables = new ContextVariables(); + contextVariables.Add("fileName", fileName); + + var customPlan = await sequentialPlanner.CreatePlanAsync(planDefinition); + + // Execute the plan + KernelResult kernelResult = await kernel.RunAsync(contextVariables, customPlan); + Console.WriteLine($"Summarization: {kernelResult.GetValue()}"); + ``` + + Note especially `planDefinition` which is a simple instruction for the planner to follow. The appropriate functions are then called based on this plan, in this case our semantic function `SummarizeText` and the native function `RetrieveLocalFile`. - These facts are then stored in the memory collection `SummarizedAzureDocs`. This is a very simplified example, but you can see how you can store information in the memory for the LLM to use. +- **Memory**: Abstracts and simplifies context management for AI apps. The idea with memory is that this is something the LLM should know about. You can store this information in a vector store which ends up being an in-memory database or a vector database or similar. Here's an example of a very simplified scenario where _facts_ are added to the memory: + + ```csharp + var facts = new Dictionary(); + facts.Add( + "Azure Machine Learning; https://learn.microsoft.com/azure/machine-learning/", + @"Azure Machine Learning is a cloud service for accelerating and + managing the machine learning project lifecycle. Machine learning professionals, + data scientists, and engineers can use it in their day-to-day workflows" + ); + + facts.Add( + "Azure SQL Service; https://learn.microsoft.com/azure/azure-sql/", + @"Azure SQL is a family of managed, secure, and intelligent products + that use the SQL Server database engine in the Azure cloud." + ); + + string memoryCollectionName = "SummarizedAzureDocs"; + + foreach (var fact in facts) { + await memoryBuilder.SaveReferenceAsync( + collection: memoryCollectionName, + description: fact.Key.Split(";")[1].Trim(), + text: fact.Value, + externalId: fact.Key.Split(";")[2].Trim(), + externalSourceName: "Azure Documentation" + ); + } + ``` + + These facts are then stored in the memory collection `SummarizedAzureDocs`. This is a very simplified example, but you can see how you can store information in the memory for the LLM to use. So that's the basics of the Semantic Kernel framework, what about the Agent Framework? @@ -428,7 +431,7 @@ So that's the basics of the Semantic Kernel framework, what about the Agent Fram Azure AI Agent Service is a more recent addition, introduced at Microsoft Ignite 2024. It allows for the development and deployment of AI agents with more flexible models, such as directly calling open-source LLMs like Llama 3, Mistral, and Cohere. -Azure AI Agent Service provides stronger enterprise security mechanisms and data storage methods, making it suitable for enterprise applications. +Azure AI Agent Service provides stronger enterprise security mechanisms and data storage methods, making it suitable for enterprise applications. It works out-of-the-box with multi-agent orchestration frameworks like AutoGen and Semantic Kernel. @@ -440,37 +443,37 @@ Azure AI Agent Service has the following core concepts: - **Agent**. Azure AI Agent Service integrates with Azure AI Foundry. Within AI Foundry, an AI Agent acts as a "smart" microservice that can be used to answer questions (RAG), perform actions, or completely automate workflows. It achieves this by combining the power of generative AI models with tools that allow it to access and interact with real-world data sources. Here's an example of an agent: - ```python - agent = project_client.agents.create_agent( - model="gpt-4o-mini", - name="my-agent", - instructions="You are helpful agent", - tools=code_interpreter.definitions, - tool_resources=code_interpreter.resources, - ) - ``` + ```python + agent = project_client.agents.create_agent( + model="gpt-4o-mini", + name="my-agent", + instructions="You are helpful agent", + tools=code_interpreter.definitions, + tool_resources=code_interpreter.resources, + ) + ``` - In this example, an agent is created with the model `gpt-4o-mini`, a name `my-agent`, and instructions `You are helpful agent`. The agent is equipped with tools and resources to perform code interpretation tasks. + In this example, an agent is created with the model `gpt-4o-mini`, a name `my-agent`, and instructions `You are helpful agent`. The agent is equipped with tools and resources to perform code interpretation tasks. - **Thread and messages**. The thread is another important concept. It represents a conversation or interaction between an agent and a user. Threads can be used to track the progress of a conversation, store context information, and manage the state of the interaction. Here's an example of a thread: - ```python - thread = project_client.agents.create_thread() - message = project_client.agents.create_message( - thread_id=thread.id, - role="user", - content="Could you please create a bar chart for the operating profit using the following data and provide the file to me? Company A: $1.2 million, Company B: $2.5 million, Company C: $3.0 million, Company D: $1.8 million", - ) - - # Ask the agent to perform work on the thread - run = project_client.agents.create_and_process_run(thread_id=thread.id, agent_id=agent.id) - - # Fetch and log all messages to see the agent's response - messages = project_client.agents.list_messages(thread_id=thread.id) - print(f"Messages: {messages}") - ``` + ```python + thread = project_client.agents.create_thread() + message = project_client.agents.create_message( + thread_id=thread.id, + role="user", + content="Could you please create a bar chart for the operating profit using the following data and provide the file to me? Company A: $1.2 million, Company B: $2.5 million, Company C: $3.0 million, Company D: $1.8 million", + ) + + # Ask the agent to perform work on the thread + run = project_client.agents.create_and_process_run(thread_id=thread.id, agent_id=agent.id) + + # Fetch and log all messages to see the agent's response + messages = project_client.agents.list_messages(thread_id=thread.id) + print(f"Messages: {messages}") + ``` - In the previous code, a thread is created. Thereafter, a message is sent to the thread. By calling `create_and_process_run`, the agent is asked to perform work on the thread. Finally, the messages are fetched and logged to see the agent's response. The messages indicate the progress of the conversation between the user and the agent. It's also important to understand that the messages can be of different types such as text, image, or file, that is the agents work has resulted in for example an image or a text response for example. As a developer, you can then use this information to further process the response or present it to the user. + In the previous code, a thread is created. Thereafter, a message is sent to the thread. By calling `create_and_process_run`, the agent is asked to perform work on the thread. Finally, the messages are fetched and logged to see the agent's response. The messages indicate the progress of the conversation between the user and the agent. It's also important to understand that the messages can be of different types such as text, image, or file, that is the agents work has resulted in for example an image or a text response for example. As a developer, you can then use this information to further process the response or present it to the user. - **Integrates with other AI frameworks**. Azure AI Agent service can interact with other frameworks like AutoGen and Semantic Kernel, which means you can build part of your app in one of these frameworks and for example using the Agent service as an orchestrator or you can build everything in the Agent service. @@ -492,13 +495,13 @@ Let's see if we can help you by going through some common use cases: > Q: My team is working on a project that involves automating code generation and data analysis tasks. Which framework should we use? > ->A: AutoGen would be a good choice for this scenario, as it focuses on event-driven, distributed agentic applications and supports advanced multi-agent design patterns. +> A: AutoGen would be a good choice for this scenario, as it focuses on event-driven, distributed agentic applications and supports advanced multi-agent design patterns. > Q: What makes AutoGen a better choice than Semantic Kernel and Azure AI Agent Service for this use case? > > A: AutoGen is specifically designed for event-driven, distributed agentic applications, making it well-suited for automating code generation and data analysis tasks. It provides the necessary tools and capabilities to build complex multi-agent systems efficiently. ->Q: Sounds like Azure AI Agent Service could work here too, it has tools for code generation and more? +> Q: Sounds like Azure AI Agent Service could work here too, it has tools for code generation and more? > > A: Yes, Azure AI Agent Service also supports code generation and data analysis tasks, but it may be more suitable for enterprise applications that require secure, scalable, and flexible AI agent deployment. AutoGen is more focused on event-driven, distributed agentic applications and advanced multi-agent design patterns. @@ -508,16 +511,16 @@ Let's see if we can help you by going through some common use cases: Let's summarize the key differences in a table: -| Framework | Focus | Core Concepts | Use Cases | -| --- | --- | --- | --- | -| AutoGen | Event-driven, distributed agentic applications | Agents, Personas, Functions, Data | Code generation, data analysis tasks | -| Semantic Kernel | Understanding and generating human-like text content | Agents, Modular Components, Collaboration | Natural language understanding, content generation | +| Framework | Focus | Core Concepts | Use Cases | +| ---------------------- | ------------------------------------------------------------------- | ------------------------------------------------ | -------------------------------------------------- | +| AutoGen | Event-driven, distributed agentic applications | Agents, Personas, Functions, Data | Code generation, data analysis tasks | +| Semantic Kernel | Understanding and generating human-like text content | Agents, Modular Components, Collaboration | Natural language understanding, content generation | | Azure AI Agent Service | Flexible models, enterprise security, Code generation, Tool calling | Modularity, Collaboration, Process Orchestration | Secure, scalable, and flexible AI agent deployment | What's the ideal use case for each of these frameworks? - **AutoGen**: Event-driven, distributed agentic applications, advanced multi-agent design patterns. Ideal for automating code generation, data analysis tasks. -- **Semantic Kernel**: Understanding and generating human-like text content, automating complex workflows, initiating tasks based on project goals. Ideal for natural language understanding, content generation. +- **Semantic Kernel**: Understanding and generating human-like text content, automating complex workflows, initiating tasks based on project goals. Ideal for natural language understanding, content generation. - **Azure AI Agent Service**: Flexible models, enterprise security mechanisms, data storage methods. Ideal for secure, scalable, and flexible AI agent deployment in enterprise applications. ## Can I integrate my existing Azure ecosystem tools directly, or do I need standalone solutions? diff --git a/04-tool-use/README.md b/04-tool-use/README.md index d100e0f7..6acbcdf9 100644 --- a/04-tool-use/README.md +++ b/04-tool-use/README.md @@ -1,6 +1,6 @@ -# Tool Use Design Pattern +# Tool Use Design Pattern -Tools are interesting because they allow AI agents to have a broader range of capabilities. Instead of the agent having a limited set of actions it can perform, by adding a tool, the agent can now perform a wide range of actions. In this chapter, we will look at the Tool Use Design Pattern, which describes how AI agents can use specific tools to achieve their goals. +Tools are interesting because they allow AI agents to have a broader range of capabilities. Instead of the agent having a limited set of actions it can perform, by adding a tool, the agent can now perform a wide range of actions. In this chapter, we will look at the Tool Use Design Pattern, which describes how AI agents can use specific tools to achieve their goals. ## Introduction @@ -26,7 +26,7 @@ The **Tool Use Design Pattern** focuses on giving LLMs the ability to interact w ## What are the use cases it can be applied to? -AI Agents can leverage tools to complete complex tasks, retrieve information, or make decisions. The tool use design pattern is often used in scenarios requiring dynamic interaction with external systems, such as databases, web services, or code interpreters. This ability is useful for a number of different use cases including: +AI Agents can leverage tools to complete complex tasks, retrieve information, or make decisions. The Tool Use Design Pattern is often used in scenarios requiring dynamic interaction with external systems, such as databases, web services, or code interpreters. This ability is useful for a number of different use cases including: - **Dynamic Information Retrieval:** Agents can query external APIs or databases to fetch up-to-date data (e.g., querying a SQLite database for data analysis, fetching stock prices or weather information). - **Code Execution and Interpretation:** Agents can execute code or scripts to solve mathematical problems, generate reports, or perform simulations. @@ -34,24 +34,24 @@ AI Agents can leverage tools to complete complex tasks, retrieve information, or - **Customer Support:** Agents can interact with CRM systems, ticketing platforms, or knowledge bases to resolve user queries. - **Content Generation and Editing:** Agents can leverage tools like grammar checkers, text summarizers, or content safety evaluators to assist with content creation tasks. -## What are the elements/building blocks needed to implement the tool use design pattern? +## What are the elements/building blocks needed to implement the Tool Use Design Pattern? These building blocks allow the AI agent to perform a wide range of tasks. Let's look at the key elements needed to implement the Tool Use Design Pattern: -- **Function/Tool Calling**: This is the primary way to enable LLMs to interact with tools. Functions or tools are blocks of reusable code that agents use to carry out tasks. These can range from simple functions like a calculator to API calls to third-party services such as stock price lookups or weather forecasts1. +- **Function/Tool Calling**: This is the primary way to enable LLMs to interact with tools. Functions or tools are blocks of reusable code that agents use to carry out tasks. These can range from simple functions like a calculator to API calls to third-party services such as stock price lookups or weather forecasts. -- **Dynamic Information Retrieval**: Agents can query external APIs or databases to fetch up-to-date data. This is useful for tasks like data analysis, fetching stock prices, or weather information1. +- **Dynamic Information Retrieval**: Agents can query external APIs or databases to fetch up-to-date data. This is useful for tasks like data analysis, fetching stock prices, or weather information. -- **Code Execution and Interpretation**: Agents can execute code or scripts to solve mathematical problems, generate reports, or perform simulations1. +- **Code Execution and Interpretation**: Agents can execute code or scripts to solve mathematical problems, generate reports, or perform simulations. -- **Workflow Automation**: This involves automating repetitive or multi-step workflows by integrating tools like task schedulers, email services, or data pipelines1. +- **Workflow Automation**: This involves automating repetitive or multi-step workflows by integrating tools like task schedulers, email services, or data pipelines. -- **Customer Support**: Agents can interact with CRM systems, ticketing platforms, or knowledge bases to resolve user queries1. +- **Customer Support**: Agents can interact with CRM systems, ticketing platforms, or knowledge bases to resolve user queries. - **Content Generation and Editing: Agents can leverage tools like grammar checkers, text summarizers, or content safety evaluators to assist with content creation tasks**. Next, let's look at Function/Tool Calling in more detail. - + ### Function/Tool Calling Function calling is the primary way we enable Large Language Models (LLMs) to interact with tools. You will often see 'Function' and 'Tool' used interchangeably because 'functions' (blocks of reusable code) are the 'tools' agents use to carry out tasks. In order for a function's code to be invoked, an LLM must compare the users request against the functions description. To do this a schema containing the descriptions of all the available functions is sent to the LLM. The LLM then selects the most appropriate function for the task and returns its name and arguments. The selected function is invoked, it's response is sent back to the LLM, which uses the information to respond to the users request. @@ -66,136 +66,136 @@ Let's use the example of getting the current time in a city to illustrate: 1. **Initialize an LLM that supports function calling:** - Not all models support function calling, so it's important to check that the LLM you are using does. Azure OpenAI supports function calling. We can start by initiating the Azure OpenAI client. + Not all models support function calling, so it's important to check that the LLM you are using does. Azure OpenAI supports function calling. We can start by initiating the Azure OpenAI client. - ```python - # Initialize the Azure OpenAI client - client = AzureOpenAI( - azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), - api_key=os.getenv("AZURE_OPENAI_API_KEY"), - api_version="2024-05-01-preview" - ) - ``` + ```python + # Initialize the Azure OpenAI client + client = AzureOpenAI( + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), + api_key=os.getenv("AZURE_OPENAI_API_KEY"), + api_version="2024-05-01-preview" + ) + ``` 1. **Create a Function Schema**: - Next we will define a JSON schema that contains the function name, description of what the function does, and the names and descriptions of the function parameters. - We will then take this schema and pass it to the client created previously, along with the users request to find the time in San Francisco. What's important to note is that a **tool call** is what is returned, **not** the final answer to the question. As mentioned earlier, the LLM returns the name of the function it selected for the task, and the arguments that will be passed to it. - - ```python - # Function description for the model to read - tools = [ - { - "type": "function", - "function": { - "name": "get_current_time", - "description": "Get the current time in a given location", - "parameters": { - "type": "object", - "properties": { - "location": { - "type": "string", - "description": "The city name, e.g. San Francisco", - }, - }, - "required": ["location"], - }, - } - } - ] - ``` - - ```python - - # Initial user message - messages = [{"role": "user", "content": "What's the current time in San Francisco"}] - - # First API call: Ask the model to use the function - response = client.chat.completions.create( - model=deployment_name, - messages=messages, - tools=tools, - tool_choice="auto", - ) - - # Process the model's response - response_message = response.choices[0].message - messages.append(response_message) - - print("Model's response:") - - print(response_message) - - ``` - - ```bash - Model's response: - ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_pOsKdUlqvdyttYB67MOj434b', function=Function(arguments='{"location":"San Francisco"}', name='get_current_time'), type='function')]) - ``` - + Next we will define a JSON schema that contains the function name, description of what the function does, and the names and descriptions of the function parameters. + We will then take this schema and pass it to the client created previously, along with the users request to find the time in San Francisco. What's important to note is that a **tool call** is what is returned, **not** the final answer to the question. As mentioned earlier, the LLM returns the name of the function it selected for the task, and the arguments that will be passed to it. + + ```python + # Function description for the model to read + tools = [ + { + "type": "function", + "function": { + "name": "get_current_time", + "description": "Get the current time in a given location", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "The city name, e.g. San Francisco", + }, + }, + "required": ["location"], + }, + } + } + ] + ``` + + ```python + + # Initial user message + messages = [{"role": "user", "content": "What's the current time in San Francisco"}] + + # First API call: Ask the model to use the function + response = client.chat.completions.create( + model=deployment_name, + messages=messages, + tools=tools, + tool_choice="auto", + ) + + # Process the model's response + response_message = response.choices[0].message + messages.append(response_message) + + print("Model's response:") + + print(response_message) + + ``` + + ```bash + Model's response: + ChatCompletionMessage(content=None, role='assistant', function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_pOsKdUlqvdyttYB67MOj434b', function=Function(arguments='{"location":"San Francisco"}', name='get_current_time'), type='function')]) + ``` + 1. **The function code required to carry out the task:** - Now that the LLM has chosen which function needs to be run the code that carries out the task needs to be implemented and executed. - We can implement the code to get the current time in Python. We will also need to write the code to extract the name and arguments from the response_message to get the final result. - - ```python - def get_current_time(location): - """Get the current time for a given location""" - print(f"get_current_time called with location: {location}") - location_lower = location.lower() - - for key, timezone in TIMEZONE_DATA.items(): - if key in location_lower: - print(f"Timezone found for {key}") - current_time = datetime.now(ZoneInfo(timezone)).strftime("%I:%M %p") - return json.dumps({ - "location": location, - "current_time": current_time + Now that the LLM has chosen which function needs to be run the code that carries out the task needs to be implemented and executed. + We can implement the code to get the current time in Python. We will also need to write the code to extract the name and arguments from the response_message to get the final result. + + ```python + def get_current_time(location): + """Get the current time for a given location""" + print(f"get_current_time called with location: {location}") + location_lower = location.lower() + + for key, timezone in TIMEZONE_DATA.items(): + if key in location_lower: + print(f"Timezone found for {key}") + current_time = datetime.now(ZoneInfo(timezone)).strftime("%I:%M %p") + return json.dumps({ + "location": location, + "current_time": current_time + }) + + print(f"No timezone data found for {location_lower}") + return json.dumps({"location": location, "current_time": "unknown"}) + ``` + + ```python + # Handle function calls + if response_message.tool_calls: + for tool_call in response_message.tool_calls: + if tool_call.function.name == "get_current_time": + + function_args = json.loads(tool_call.function.arguments) + + time_response = get_current_time( + location=function_args.get("location") + ) + + messages.append({ + "tool_call_id": tool_call.id, + "role": "tool", + "name": "get_current_time", + "content": time_response, }) - - print(f"No timezone data found for {location_lower}") - return json.dumps({"location": location, "current_time": "unknown"}) - ``` - - ```python - # Handle function calls - if response_message.tool_calls: - for tool_call in response_message.tool_calls: - if tool_call.function.name == "get_current_time": - - function_args = json.loads(tool_call.function.arguments) - - time_response = get_current_time( - location=function_args.get("location") - ) - - messages.append({ - "tool_call_id": tool_call.id, - "role": "tool", - "name": "get_current_time", - "content": time_response, - }) - else: - print("No tool calls were made by the model.") - - # Second API call: Get the final response from the model - final_response = client.chat.completions.create( - model=deployment_name, - messages=messages, - ) - - return final_response.choices[0].message.content - ``` - - ```bash - get_current_time called with location: San Francisco - Timezone found for san francisco - The current time in San Francisco is 09:24 AM. - ``` + else: + print("No tool calls were made by the model.") + + # Second API call: Get the final response from the model + final_response = client.chat.completions.create( + model=deployment_name, + messages=messages, + ) + + return final_response.choices[0].message.content + ``` + + ```bash + get_current_time called with location: San Francisco + Timezone found for san francisco + The current time in San Francisco is 09:24 AM. + ``` Function Calling is at the heart of most, if not all agent tool use design, however implementing it from scratch can sometimes be challenging. As we learned in [Lesson 2](../02-explore-agentic-frameworks/) agentic frameworks provide us with pre-built building blocks to implement tool use. - + ## Tool Use Examples with Agentic Frameworks Here are some examples of how you can implement the Tool Use Design Pattern using different agentic frameworks: @@ -225,7 +225,7 @@ class GetCurrentTimePlugin: ``` -```python +```python from semantic_kernel import Kernel # Create the kernel @@ -237,7 +237,7 @@ get_current_time_plugin = GetCurrentTimePlugin(location) # Add the plugin to the kernel kernel.add_plugin(get_current_time_plugin) ``` - + ### Azure AI Agent Service Azure AI Agent Service is a newer agentic framework that is designed to empower developers to securely build, deploy, and scale high-quality, and extensible AI agents without needing to manage the underlying compute and storage resources. It is particularly useful for enterprise applications since it is a fully managed service with enterprise grade security. @@ -251,15 +251,15 @@ When compared to developing with the LLM API directly, Azure AI Agent Service pr The tools available in Azure AI Agent Service can be divided into two categories: 1. Knowledge Tools: - - Grounding with Bing Search - - File Search - - Azure AI Search + - Grounding with Bing Search + - File Search + - Azure AI Search 2. Action Tools: - - Function Calling - - Code Interpreter - - OpenAI defined tools - - Azure Functions + - Function Calling + - Code Interpreter + - OpenAI defined tools + - Azure Functions The Agent Service allows us to be able to use these tools together as a `toolset`. It also utilizes `threads` which keep track of the history of messages from a particular conversation. @@ -271,7 +271,7 @@ The following image illustrates how you could use Azure AI Agent Service to anal To use any of these tools with the service we can create a client and define a tool or toolset. To implement this practically we can use the following Python code. The LLM will be able to look at the toolset and decide whether to use the user created function, `fetch_sales_data_using_sqlite_query`, or the pre-built Code Interpreter depending on the user request. -```python +```python import os from azure.ai.projects import AIProjectClient from azure.identity import DefaultAzureCredential @@ -288,20 +288,20 @@ fetch_data_function = FunctionTool(fetch_sales_data_using_sqlite_query) toolset = ToolSet() toolset.add(fetch_data_function) -# Initialize Code Interpreter tool and adding it to the toolset. +# Initialize Code Interpreter tool and adding it to the toolset. code_interpreter = code_interpreter = CodeInterpreterTool() toolset = ToolSet() toolset.add(code_interpreter) agent = project_client.agents.create_agent( - model="gpt-4o-mini", name="my-agent", instructions="You are helpful agent", + model="gpt-4o-mini", name="my-agent", instructions="You are helpful agent", toolset=toolset ) ``` ## What are the special considerations for using the Tool Use Design Pattern to build trustworthy AI agents? -A common concern with SQL dynamically generated by LLMs is security, particularly the risk of SQL injection or malicious actions, such as dropping or tampering with the database. While these concerns are valid, they can be effectively mitigated by properly configuring database access permissions. For most databases this involves configuring the database as read-only. For database services like PostgreSQL or Azure SQL, the app should be assigned a read-only (SELECT) role. R +A common concern with SQL dynamically generated by LLMs is security, particularly the risk of SQL injection or malicious actions, such as dropping or tampering with the database. While these concerns are valid, they can be effectively mitigated by properly configuring database access permissions. For most databases this involves configuring the database as read-only. For database services like PostgreSQL or Azure SQL, the app should be assigned a read-only (SELECT) role. Running the app in a secure environment further enhances protection. In enterprise scenarios, data is typically extracted and transformed from operational systems into a read-only database or data warehouse with a user-friendly schema. This approach ensures that the data is secure, optimized for performance and accessibility, and that the app has restricted, read-only access. diff --git a/07-planning-design/README.md b/07-planning-design/README.md index e85ffc85..e3dd711e 100644 --- a/07-planning-design/README.md +++ b/07-planning-design/README.md @@ -29,7 +29,7 @@ While it is simple to state, it still needs refinement. The clearer the goal, th ### Task Decomposition - Large or intricate tasks become more manageable when split into smaller, goal-oriented subtasks. +Large or intricate tasks become more manageable when split into smaller, goal-oriented subtasks. For the travel itinerary example, you could decompose the goal into: * Flight Booking @@ -175,9 +175,9 @@ e.g sample code ```python from autogen_core.models import UserMessage, SystemMessage, AssistantMessage - #.. same as previous code and pass on the user history, current plan + #.. same as previous code and pass on the user history, current plan messages = [ - SystemMessage(content="""You are a planner agent to optimize the + SystemMessage(content="""You are a planner agent to optimize the Your job is to decide which agents to run based on the user's request. Below are the available agents specialized in different tasks: - FlightBooking: For booking flights and providing flight information @@ -200,4 +200,4 @@ In this article we have looked at an example of how we can create a planner that ## Additional Resources -* AutoGen Magentic One - A Generalist multi agent system for solving complex tasks and has achieved impressive results on multiple challenging agentic benchmarks. Reference: autogen-magentic-one. In this implementation the orchestrator create task specific plan and delegates these tasks to the available agents. In addition to planning the orchestrator also employs a tracking mechanism to monitor the progress of the task and re-plans as required. +* AutoGen Magentic One - A Generalist multi-agent system for solving complex tasks and has achieved impressive results on multiple challenging agentic benchmarks. Reference: autogen-magentic-one. In this implementation the orchestrator create task specific plan and delegates these tasks to the available agents. In addition to planning the orchestrator also employs a tracking mechanism to monitor the progress of the task and re-plans as required. diff --git a/08-multi-agent/README.md b/08-multi-agent/README.md index 09ed97a2..7290ed87 100644 --- a/08-multi-agent/README.md +++ b/08-multi-agent/README.md @@ -1,6 +1,6 @@ # Multi agent design patterns -As soon as you start working on a project that involves multiple agents, you will need to consider the multi-agent design pattern. However, it might not be immediately clear when to switch to multi-agents and what the advantages are. +As soon as you start working on a project that involves multiple agents, you will need to consider the multi-agent design pattern. However, it might not be immediately clear when to switch to multi-agents and what the advantages are. ## Introduction @@ -21,9 +21,9 @@ After this lesson, you should be able to: What's the bigger picture? -*Multi agents are a design pattern that allows multiple agents to work together to achieve a common goal*. +*Multi-agents are a design pattern that allows multiple agents to work together to achieve a common goal*. -This pattern is widely used in various fields, including robotics, autonomous systems, and distributed computing. +This pattern is widely used in various fields, including robotics, autonomous systems, and distributed computing. ## Scenarios Where Multi-Agents Are Applicable @@ -52,7 +52,7 @@ Before you can implement the multi-agent design pattern, you need to understand Let's make this more concrete by again looking at the example of booking a trip for a user. In this case, the building blocks would include: - **Agent Communication**: Agents for finding flights, booking hotels, and rental cars need to communicate and share information about the user's preferences and constraints. You need to decide on the protocols and methods for this communication. What this means concretely is that the agent for finding flights needs to communicate with the agent for booking hotels to ensure that the hotel is booked for the same dates as the flight. That means that the agents need to share information about the user's travel dates, meaning that you need to decide *which agents are sharing info and how they are sharing info*. -- **Coordination Mechanisms**: Agents need to coordinate their actions to ensure that the user's preferences and constraints are met. A user preference could be that they want a hotel close to the airport whereas a constraint could be that rental cars are only available at the airport. This means that the agent for booking hotels needs to coordinate with the agent for booking rental cars to ensure that the user's preferences and constraints are met. This means that you need to decide *how the agents are coordinating their actions*. +- **Coordination Mechanisms**: Agents need to coordinate their actions to ensure that the user's preferences and constraints are met. A user preference could be that they want a hotel close to the airport whereas a constraint could be that rental cars are only available at the airport. This means that the agent for booking hotels needs to coordinate with the agent for booking rental cars to ensure that the user's preferences and constraints are met. This means that you need to decide *how the agents are coordinating their actions*. - **Agent Architecture**: Agents need to have the internal structure to make decisions and learn from their interactions with the user. This means that the agent for finding flights needs to have the internal structure to make decisions about which flights to recommend to the user. This means that you need to decide *how the agents are making decisions and learning from their interactions with the user*. Examples of how an agent learns and improves could be that the agent for finding flights could use a machine learning model to recommend flights to the user based on their past preferences. - **Visibility into Multi-Agent Interactions**: You need to have visibility into how the multiple agents are interacting with each other. This means that you need to have tools and techniques for tracking agent activities and interactions. This could be in the form of logging and monitoring tools, visualization tools, and performance metrics. - **Multi-Agent Patterns**: There are different patterns for implementing multi-agent systems, such as centralized, decentralized, and hybrid architectures. You need to decide on the pattern that best fits your use case. @@ -72,15 +72,15 @@ Let's look at each of these aspects more in detail. - **Performance Metrics**: Performance metrics can help you track the effectiveness of the multi-agent system. For example, you could track the time taken to complete a task, the number of tasks completed per unit of time, and the accuracy of the recommendations made by the agents. This information can help you identify areas for improvement and optimize the system. -## Multi agent patterns +## Multi-Agent Patterns -Let's dive into some concrete patterns we can use to create multi agent apps. Here are some interesting patterns worth considering: +Let's dive into some concrete patterns we can use to create multi-agent apps. Here are some interesting patterns worth considering: ### Group chat This pattern is useful when you want to create a group chat application where multiple agents can communicate with each other. Typical use cases for this pattern include team collaboration, customer support, and social networking. -In this pattern, each agent represents a user in the group chat, and messages are exchanged between agents using a messaging protocol. The agents can send messages to the group chat, receive messages from the group chat, and respond to messages from other agents. +In this pattern, each agent represents a user in the group chat, and messages are exchanged between agents using a messaging protocol. The agents can send messages to the group chat, receive messages from the group chat, and respond to messages from other agents. This pattern can be implemented using a centralized architecture where all messages are routed through a central server, or a decentralized architecture where messages are exchanged directly. @@ -104,9 +104,9 @@ Why you would want multiple agents to collaborate is because each agent can have Let's take an example where a user wants a recommendation on the best stock to buy on the stock market. -- **Industry expert**:. One agent could be an expert in a specific industry. -- **Technical analysis**: Another agent could be an expert in technical analysis. -- **Fundamental analysis**: and another agent could be an expert in fundamental analysis. By collaborating, these agents can provide a more comprehensive recommendation to the user. +- **Industry expert**:. One agent could be an expert in a specific industry. +- **Technical analysis**: Another agent could be an expert in technical analysis. +- **Fundamental analysis**: and another agent could be an expert in fundamental analysis. By collaborating, these agents can provide a more comprehensive recommendation to the user. ![Recommendation](./images/multi-agent-filtering.png) @@ -161,7 +161,7 @@ Question: When should you consider using multi-agents? - [] A2: When you have a large workload - [] A3: When you have a simple task. -[Solution quiz](./solution/solution-quiz.md) +[Solution quiz](./solution/solution-quiz.md) ## Summary