Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev -> main (Mar 29) #1294

Merged
merged 175 commits into from
Apr 7, 2025
Merged

dev -> main (Mar 29) #1294

merged 175 commits into from
Apr 7, 2025

Conversation

creatorrr
Copy link
Contributor

@creatorrr creatorrr commented Mar 29, 2025

PR Type

Enhancement, Tests, Documentation, Bug fix, Configuration changes


Description

  • Introduced the Open Responses API with endpoints for creating and retrieving responses, including support for tools, multi-modal inputs, and structured outputs.

  • Added extensive enhancements to metadata filtering, tool execution, and API integration for improved functionality and security.

  • Implemented new Pydantic models, TypeSpec definitions, and OpenAPI specifications to support the Open Responses API.

  • Added comprehensive tests for metadata filtering, agent queries, and utility functions to ensure robustness and security.

  • Updated documentation with concepts, quickstart guides, CLI usage, examples, and a roadmap for the Open Responses API.

  • Enhanced Docker Compose configurations and environment settings for better deployment and configuration management.

  • Fixed multiple bugs, including typos in model names and assertion messages, ensuring consistency and correctness.


Changes walkthrough 📝

Relevant files
Enhancement
25 files
Responses.py
Introduced Pydantic models for Open Responses API.             

agents-api/agents_api/autogen/Responses.py

  • Added a new file defining multiple Pydantic models for handling
    responses.
  • Introduced models for tools, actions, and response structures like
    CreateResponse, Response, and ToolChoice.
  • Included support for multi-modal inputs, tool calls, and structured
    outputs.
  • Added detailed type annotations and documentation for each model.
  • +1037/-0
    model_converters.py
    Added model conversion utilities for Open Responses API. 

    agents-api/agents_api/routers/utils/model_converters.py

  • Added utility functions to convert CreateResponse into internal models
    like ChatInput.
  • Implemented logic to handle various input types, including tools and
    multi-modal data.
  • Added support for converting chat responses back into API-compatible
    responses.
  • +421/-0 
    create_response.py
    Added create response endpoint for Open Responses API.     

    agents-api/agents_api/routers/responses/create_response.py

  • Added a new endpoint to create responses using the Open Responses API.
  • Implemented tool call processing and recursive handling of tool
    interactions.
  • Integrated with the convert_create_response utility for input
    transformation.
  • +331/-0 
    tool_executor.py
    Implemented tool execution logic for Open Responses API. 

    agents-api/agents_api/activities/tool_executor.py

  • Added functionality to execute tool calls, including web search tools.
  • Implemented integration with Brave search for web search tool
    execution.
  • Added error handling and result formatting for tool execution.
  • +191/-0 
    get_response.py
    Added get response endpoint for Open Responses API.           

    agents-api/agents_api/routers/responses/get_response.py

  • Added a new endpoint to retrieve responses by ID.
  • Implemented logic to reconstruct response history and output.
  • Added support for handling tool calls and output messages.
  • +143/-0 
    web.py
    Integrated responses router into the application.               

    agents-api/agents_api/web.py

  • Added conditional inclusion of the responses router based on
    environment configuration.
  • Updated router registration logic to support the Open Responses API.
  • +12/-9   
    bulk_delete_docs.py
    Refactored bulk delete logic for secure metadata filtering.

    agents-api/agents_api/queries/docs/bulk_delete_docs.py

  • Refactored metadata filtering logic to use a utility function.
  • Improved SQL injection prevention in bulk delete operations.
  • Simplified query construction for metadata conditions.
  • +13/-13 
    openapi_model.py
    Enhanced OpenAPI models for Open Responses API.                   

    agents-api/agents_api/autogen/openapi_model.py

  • Added new models for tool execution results and web preview tool
    calls.
  • Introduced Includable type for specifying response inclusion options.
  • Updated existing models to support Open Responses API features.
  • +27/-1   
    utils.py
    Added utility for secure metadata filtering in queries.   

    agents-api/agents_api/queries/utils.py

  • Added a utility function for building metadata filter conditions
    securely.
  • Improved SQL query construction to prevent injection attacks.
  • Enhanced reusability of metadata filtering logic across queries.
  • +38/-0   
    litellm.py
    Enhanced LiteLLM client for Open Responses API.                   

    agents-api/agents_api/clients/litellm.py

  • Added support for enabling responses in LiteLLM client.
  • Updated model list retrieval to respect the enable_responses flag.
  • Improved handling of custom API keys for model interactions.
  • +7/-2     
    list_agents.py
    Improved metadata filtering in agent listing queries.       

    agents-api/agents_api/queries/agents/list_agents.py

  • Refactored metadata filtering logic to use JSONB containment.
  • Simplified query construction for agent listing with metadata filters.
  • +12/-7   
    list_docs.py
    Improved metadata filtering in document listing queries. 

    agents-api/agents_api/queries/docs/list_docs.py

  • Refactored metadata filtering logic to use a utility function.
  • Enhanced SQL query construction for secure metadata filtering.
  • +11/-6   
    algolia.py
    Improved Algolia search request handling.                               

    integrations-service/integrations/utils/integrations/algolia.py

  • Refactored search request construction for Algolia integration.
  • Added conditional handling for attributes to retrieve in search
    requests.
  • +11/-9   
    render.py
    Enhanced tool handling in session rendering.                         

    agents-api/agents_api/routers/sessions/render.py

  • Updated tool retrieval logic to include user-provided tools.
  • Ensured tools are not duplicated when combining agent and user tools.
  • +8/-2     
    __init__.py
    Added responses router to application routing.                     

    agents-api/agents_api/routers/init.py

  • Added responses router to the module imports.
  • Integrated responses functionality into the application routing.
  • +1/-0     
    __init__.py
    Updated entry queries module exports.                                       

    agents-api/agents_api/queries/entries/init.py

  • Added add_entry_relations to the module exports.
  • Updated module imports for entry-related queries.
  • +2/-1     
    __init__.py
    Initialized responses-related endpoints.                                 

    agents-api/agents_api/routers/responses/init.py

  • Added initialization for responses-related endpoints.
  • Included create_response and get_response in the module exports.
  • +5/-0     
    router.py
    Added router for responses-related endpoints.                       

    agents-api/agents_api/routers/responses/router.py

    • Added a new router for responses-related endpoints.
    +3/-0     
    openapi-1.0.0.yaml
    Added OpenAPI specifications for new response endpoints and schemas.

    typespec/tsp-output/@typespec/openapi3/openapi-1.0.0.yaml

  • Added new endpoints for creating and retrieving responses under
    /responses.
  • Introduced multiple new schemas for response handling, including
    Responses.CreateResponse, Responses.Response, and various tool-related
    schemas.
  • Fixed a typo in schema reference from Chat.CompetionUsage to
    Chat.CompletionUsage.
  • Added new components for handling input types, reasoning, and tool
    configurations.
  • +1369/-2
    models.tsp
    Introduced TypeSpec models for response creation and management.

    typespec/responses/models.tsp

  • Defined models for creating and managing responses, including
    CreateResponse and Response.
  • Added support for reasoning, tool choice, and metadata in response
    models.
  • Introduced enums and aliases for response formats and reasoning
    effort.
  • +841/-0 
    create_agent_request.json
    Modified schema for agent creation request.                           

    schemas/create_agent_request.json

  • Updated attributes_to_retrieve field to accept an array of strings
    instead of an object.
  • Adjusted schema validation rules accordingly.
  • +8/-2     
    create_task_request.json
    Modified schema for task creation request.                             

    schemas/create_task_request.json

  • Updated attributes_to_retrieve field to accept an array of strings
    instead of an object.
  • Adjusted schema validation rules accordingly.
  • +8/-2     
    main.tsp
    Integrated responses endpoints into the main API namespace.

    typespec/main.tsp

  • Added ResponsesRoute to the main API namespace.
  • Integrated response-related endpoints into the API structure.
  • +4/-0     
    endpoints.tsp
    Added TypeSpec endpoints for response operations.               

    typespec/responses/endpoints.tsp

  • Defined endpoints for creating and retrieving responses.
  • Included support for query parameters and request bodies.
  • +33/-0   
    main.tsp
    Added main TypeSpec file for responses module.                     

    typespec/responses/main.tsp

  • Created a main entry point for response-related TypeSpec files.
  • Organized imports and namespace for better modularity.
  • +8/-0     
    Tests
    3 files
    test_docs_metadata_filtering.py
    Added tests for secure metadata filtering in document queries.

    agents-api/tests/test_docs_metadata_filtering.py

  • Added tests for secure metadata filtering in document queries.
  • Verified handling of SQL injection attempts in metadata filters.
  • Tested exact matching for special characters in metadata keys and
    values.
  • +228/-0 
    test_agent_metadata_filtering.py
    Added tests for secure metadata filtering in agent queries.

    agents-api/tests/test_agent_metadata_filtering.py

  • Added tests for secure metadata filtering in agent queries.
  • Verified handling of SQL injection attempts in agent metadata filters.
  • Tested filtering by shared and specific metadata keys.
  • +137/-0 
    test_metadata_filter_utils.py
    Added tests for metadata filter utility functions.             

    agents-api/tests/test_metadata_filter_utils.py

  • Added tests for metadata filter utility functions.
  • Verified prevention of SQL injection in metadata filter conditions.
  • Tested various scenarios, including empty and complex filters.
  • +92/-0   
    Configuration changes
    9 files
    env.py
    Updated environment configuration for Open Responses API.

    agents-api/agents_api/env.py

  • Added new environment variables for enabling responses and Brave API
    key.
  • Updated default values for optional API keys.
  • Consolidated environment variables for better configuration
    management.
  • +8/-3     
    docker-compose.yml
    Updated Docker Compose configuration for Open Responses API.

    agents-api/docker-compose.yml

  • Added environment variable for enabling responses in the Docker
    Compose file.
  • Included Brave API key configuration for tool execution.
  • +2/-0     
    standalone-docker-compose.yaml
    Added standalone Docker Compose configuration for Open Responses API.

    deploy/standalone-docker-compose.yaml

  • Added a standalone Docker Compose configuration for the Open Responses
    API.
  • Configured services for API, integrations, memory store, and
    vectorizer worker.
  • Included environment variable placeholders for API keys and service
    settings.
  • +99/-0   
    docker-compose-api.yml
    Added Docker Compose configuration for agents API service.

    agents-api/docker-compose-api.yml

  • Added Docker Compose configuration for the agents API.
  • Included shared environment variables and build configurations.
  • Configured service for development and production environments.
  • +68/-0   
    .env.example
    Added example environment configuration for Open Responses API.

    deploy/.env.example

  • Added example environment variables for configuring the Open Responses
    API.
  • Included placeholders for API keys and service settings.
  • +33/-0   
    .env.example
    Added flag for enabling responses in the API.                       

    .env.example

  • Added ENABLE_RESPONSES flag to control response feature in the API.
  • Included comments for better understanding of the new flag.
  • +7/-0     
    docker-compose.yml
    Enhanced Docker Compose configuration for integrations service.

    integrations-service/docker-compose.yml

  • Updated build configuration for the integrations service.
  • Added platform-specific build settings.
  • +5/-1     
    Dockerfile.migrations
    Added Dockerfile for memory store migrations.                       

    memory-store/Dockerfile.migrations

  • Added a Dockerfile for handling database migrations.
  • Configured migration commands and paths for the memory store.
  • +11/-0   
    responses-build-docker-compose.yaml
    Added build configuration for Open Responses API.               

    deploy/responses-build-docker-compose.yaml

  • Added a Docker Compose configuration for building the Open Responses
    API.
  • Included references to dependent services like agents API and memory
    store.
  • +7/-0     
    Bug fix
    4 files
    Chat.py
    Fixed and updated completion usage model.                               

    integrations-service/integrations/autogen/Chat.py

  • Fixed typo in CompletionUsage class name.
  • Updated usage statistics model for consistency.
  • +2/-2     
    Chat.py
    Fixed and updated completion usage model.                               

    agents-api/agents_api/autogen/Chat.py

  • Fixed typo in CompletionUsage class name.
  • Updated usage statistics model for consistency.
  • +2/-2     
    developer_id.py
    Fixed assertion message for developer ID header.                 

    agents-api/agents_api/dependencies/developer_id.py

  • Fixed assertion message for developer ID header in single-tenant mode.
  • +1/-1     
    models.tsp
    Corrected typo in chat model naming.                                         

    typespec/chat/models.tsp

  • Fixed a typo in the CompetionUsage model, renaming it to
    CompletionUsage.
  • Updated references to the corrected model name.
  • +2/-2     
    Documentation
    9 files
    concepts.mdx
    Added concepts documentation for Open Responses API.         

    documentation/responses/concepts.mdx

  • Added documentation for the concepts of the Open Responses API.
  • Explained key components, input formats, and response structure.
  • Provided best practices and next steps for users.
  • +255/-0 
    quickstart.mdx
    Added quickstart guide for Open Responses API.                     

    documentation/responses/quickstart.mdx

  • Added quickstart guide for the Open Responses API.
  • Provided installation instructions and example usage.
  • Included CLI and Docker setup steps.
  • +280/-0 
    mint.json
    Updated documentation navigation for Open Responses API. 

    documentation/mint.json

  • Updated navigation to include Open Responses API documentation.
  • Added links to quickstart, concepts, and examples pages.
  • +23/-5   
    examples.mdx
    Added usage examples for Open Responses API.                         

    documentation/responses/examples.mdx

  • Added examples for using the Open Responses API with Python SDKs.
  • Demonstrated reasoning features, web search tool, and maintaining
    conversation history.
  • Included examples for creating agents with custom tools like weather
    assistants.
  • +311/-0 
    cli.mdx
    Added CLI documentation for Open Responses API setup and usage.

    documentation/responses/cli.mdx

  • Documented CLI for setting up and managing the Open Responses API.
  • Included installation instructions for Go, npm, and Python.
  • Detailed CLI commands for setup, starting/stopping services, and
    managing API keys.
  • +439/-0 
    roadmap.mdx
    Documented roadmap for Open Responses API development.     

    documentation/responses/roadmap.mdx

  • Added a roadmap for the Open Responses API.
  • Highlighted implemented, partially implemented, and planned features.
  • Included future considerations for system enhancements and scaling.
  • +149/-0 
    algolia.mdx
    Updated Algolia integration documentation for parameter changes.

    documentation/docs/integrations/search/algolia.mdx

  • Updated attributes_to_retrieve parameter to accept a list instead of a
    dictionary.
  • Adjusted examples and descriptions to reflect the change.
  • +2/-4     
    README.md
    Added announcement for Open Responses API in README.         

    README.md

  • Announced the launch of the Julep Open Responses API in alpha.
  • Highlighted key features such as OpenAI compatibility and
    self-hosting.
  • +15/-0   
    julep.mdx
    Improved introduction documentation for Julep platform.   

    documentation/docs/introduction/julep.mdx

  • Updated introduction to Julep with improved formatting and emphasis on
    features.
  • Enhanced readability and structure of the content.
  • +7/-8     

    Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • HamadaSalhab and others added 30 commits March 13, 2025 13:54
    … endpoint
    
    Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
    feat(agents-api): add logic to create_response endpoint
    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    ❌ Changes requested. Reviewed everything up to 85f71d8 in 3 minutes and 48 seconds

    More details
    • Looked at 7480 lines of code in 50 files
    • Skipped 1 files when reviewing.
    • Skipped posting 26 drafted comments based on config settings.
    1. agents-api/agents_api/env.py:148
    • Draft comment:
      Using default=None for keys like SAPLING_API_KEY and BRAVE_API_KEY is fine if downstream code handles None safely. Consider adding explicit error handling or validation to provide clear error messages if required API keys are missing.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50%
      The comment is speculative - it says "if downstream code handles None safely" but doesn't know if it does or not. It's suggesting potential error handling without knowing if it's needed. The comment is also not clearly actionable - it's more of a general suggestion without specific guidance. We can't see the downstream code that uses these API keys.
      I might be wrong because proper error handling for missing API keys is generally important. The suggestion could prevent runtime errors.
      While API key error handling is important, this comment is too speculative without seeing the downstream code. It's asking the author to "consider" something without clear evidence it's needed.
      Delete the comment because it's speculative and not clearly actionable without more context about how these API keys are used.
    2. agents-api/agents_api/queries/utils.py:338
    • Draft comment:
      The newly added build_metadata_filter_conditions utility function correctly parameterizes metadata filters to prevent SQL injection. Consider adding a brief inline comment explaining the parameter indexing logic for future maintainers.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
      The code is quite readable as is. The variable names param_idx_key and param_idx_value are descriptive. The logic is simple - just incrementing based on params length. The function already has comprehensive docstring explaining its purpose and parameters. An additional inline comment would be redundant and could make the code more cluttered.
      The parameter indexing logic, while simple, could be confusing to developers not familiar with SQL parameter indexing. The increment by 2 pattern might not be immediately obvious.
      The variable names and context make it clear we're dealing with SQL parameter indices. The pattern is a common one for handling key-value pairs in SQL parameters.
      The comment should be deleted. The code is self-documenting with clear variable names and already has a comprehensive docstring. An additional inline comment would add noise without significant value.
    3. agents-api/tests/test_agent_metadata_filtering.py:1
    • Draft comment:
      The tests for SQL injection attempts in metadata filtering are comprehensive. Ensure similar tests are maintained or added for all queries (e.g. docs queries) to cover edge cases across the application.
    • Reason this comment was not posted:
      Comment was on unchanged code.
    4. schemas/create_agent_request.json:196
    • Draft comment:
      Changing the schema for 'attributes_to_retrieve' from an object to an array of strings improves clarity. Verify that this change is communicated to client developers to ensure backward compatibility or enable necessary migration steps.
    • Reason this comment was not posted:
      Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
      The comment is asking the PR author to ensure communication with client developers about a schema change. This falls under the category of asking the author to ensure something is done, which is against the rules.
    5. agents-api/env.py:148
    • Draft comment:
      Changing the default value for SAPLING_API_KEY, ZEROGPT_API_KEY, and BRAVE_API_KEY from an empty string to None may break downstream consumers if they do not handle null values. Please verify that all usages correctly check for None.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    6. agents-api/integrations/utils/integrations/algolia.py:27
    • Draft comment:
      The refactoring for the Algolia search request now conditionally adds 'attributesToRetrieve' as an array if provided. Confirm that this aligns with Algolia's API expectations and that any previous object usage is no longer needed.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    7. documentation/docs/integrations/search/algolia.mdx:42
    • Draft comment:
      The example now shows 'attributes_to_retrieve' as an array with a '$' prefix. Please ensure that the '$' notation is intentional and documented, as it differs from the prior object format.
    • Reason this comment was not posted:
      Comment did not seem useful. Confidence is useful = 40% <= threshold 50%
      The comment is asking the PR author to ensure that the $ notation is intentional and documented. This falls under asking the author to confirm their intention, which is against the rules. However, it does point out a specific change in the code that might be an issue, which could be useful. I need to evaluate if the comment is more about confirming intention or if it provides a specific suggestion or observation that could be useful.
    8. agents-api/routers/responses/create_response.py:150
    • Draft comment:
      There are several FIXME comments (e.g., 'Adaptive context is not yet implemented', 'Generate summary is not supported yet'). These indicate incomplete functionality that must be addressed or clearly documented as postponed work before production release.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    9. agents-api/web.py:265
    • Draft comment:
      The responses router is conditionally included based on the enable_responses flag. Verify that this feature flag and the router dependencies are consistent with the intended deployment model, and that the ordering of router inclusions does not cause conflicts.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    10. agents-api/docker-compose.yml:46
    • Draft comment:
      There is an inconsistency in the boolean representation for ENABLE_RESPONSES across docker-compose files (one file uses "True" as a string, another uses a default of false). Ensure that the environment variable is handled consistently as a boolean.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    11. memory-store/Dockerfile.migrations:1
    • Draft comment:
      The migration Dockerfile hardcodes the password 'obviously_not_a_safe_password' in the connection string. Although likely a development placeholder, ensure this is not used in production and consider clearly documenting the security risk.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50%
      The password is clearly a placeholder and the filename suggests this is for migrations. The comment is speculative ("ensure this is not used in production") and asks for documentation rather than suggesting a specific code change. The password being obviously fake suggests this is intentionally for development.
      The hardcoded password could be a legitimate security concern if this Dockerfile is used as a base for other environments. Maybe there should be an environment variable instead?
      While using an environment variable would be better practice, the comment itself doesn't suggest this solution - it just asks for documentation and verification. Per our rules, we shouldn't make speculative comments or ask for verification.
      Delete this comment as it violates our rules by being speculative and asking for verification/documentation rather than suggesting a specific code change.
    12. typespec/responses/endpoints.tsp:1
    • Draft comment:
      New endpoints for the responses API have been added. Please double-check that the model definitions and enum values align with the existing API design and that all necessary properties (e.g., in Response, CreateResponse models) are consistent with the OpenAPI spec.
    • Reason this comment was not posted:
      Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
      The comment is asking the PR author to double-check the model definitions and enum values, which violates the rule against asking the author to confirm or double-check things. It does not provide a specific suggestion or point out a specific issue.
    13. agents-api/agents_api/autogen/Chat.py:54
    • Draft comment:
      Typographical error: the field 'usage' was originally typed as 'CompetionUsage'. It has been corrected to 'CompletionUsage'.
    • Reason this comment was not posted:
      Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
      This comment is purely informative, pointing out a typographical correction. It doesn't provide a suggestion or ask for confirmation on an intentional change. According to the rules, purely informative comments should be removed.
    14. agents-api/agents_api/autogen/Chat.py:127
    • Draft comment:
      Typographical error: the class name was misspelled as 'CompetionUsage'. It has been fixed to 'CompletionUsage'.
    • Reason this comment was not posted:
      Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
      This comment is purely informative, pointing out a typographical error that has already been fixed. It does not provide any actionable feedback or suggestions for improvement.
    15. agents-api/agents_api/queries/entries/__init__.py:2
    • Draft comment:
      Typo: The docstring refers to the module as entry but all other references (including the file structure and all list) use entries. Consider updating the docstring to 'entries' for consistency.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    16. agents-api/agents_api/queries/utils.py:84
    • Draft comment:
      It appears there is a typographical error in the generic function definition. The syntax 'def pg_query[**P](' does not seem valid. It is likely intended to be 'def pg_query[P](' or similar. Please review the intended generic syntax to ensure correctness.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    17. agents-api/agents_api/queries/utils.py:338
    • Draft comment:
      In the serialize_model_data function, the isinstance check uses 'list | tuple'. This syntax for isinstance is non-standard; it is recommended to use a tuple of types instead (i.e., isinstance(data, (list, tuple))). This change will prevent potential compatibility issues.
    • Reason this comment was not posted:
      Comment was on unchanged code.
    18. agents-api/docker-compose-api.yml:31
    • Draft comment:
      Typographical inconsistency: On line 31, the docker image is referenced as 'julepai/agents-api'. Please confirm if this is intentional or if it should be updated to 'julep/agents-api' to maintain consistency.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
      Since this is a new file, the image name is part of the changes. However, we don't have enough context to know if 'julepai' is the correct organization name or if it should be 'julep'. The comment is asking for verification rather than pointing out a definite issue. According to the rules, we should not ask authors to verify or confirm their intentions.
      The image name could be intentionally different for valid reasons, such as being a different organization or repository. We don't have access to verify the correct naming convention.
      The rules clearly state not to make speculative comments or ask authors to confirm their intentions. This comment does both.
      The comment should be removed as it asks for confirmation and is speculative without strong evidence of an actual issue.
    19. deploy/standalone-docker-compose.yaml:24
    • Draft comment:
      Typo: The comment on line 24 is missing a closing parenthesis. It currently reads '# Brave Search API Key (For web search tool'. Consider adding a closing parenthesis to complete it, e.g., '# Brave Search API Key (For web search tool)'.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
      While technically correct, this is an extremely minor issue in a comment that doesn't affect functionality. Comments are documentation, and while clean documentation is good, this level of nitpicking about punctuation doesn't meet the bar for "clearly a code change required". The meaning is perfectly clear without the parenthesis.
      The missing parenthesis could be seen as a small documentation inconsistency since other similar comments in the file do have proper parentheses.
      Even with the inconsistency, this is far too minor to warrant a PR comment. It doesn't affect code functionality or readability in any meaningful way.
      Delete this comment as it's too minor and doesn't meet the bar for "clearly a code change required".
    20. documentation/responses/quickstart.mdx:190
    • Draft comment:
      There's a typographical error in the Info block. The sentence 'To learn more about the CLI one can use the checkout the CLI Documentation.' could be reworded to 'To learn more about the CLI, check out the CLI Documentation.' to improve clarity.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50%
      While the comment identifies a real grammatical issue and the suggestion improves readability, our rules state we should not make purely informative comments or focus on obvious/unimportant issues. Documentation typos and minor grammar fixes, unless they significantly impact understanding, fall into this category.
      The grammar error could potentially cause confusion for non-native English speakers. The current wording is clearly incorrect.
      While the error is real, it's a minor grammatical issue in documentation that doesn't significantly impact comprehension. The meaning is still clear despite the awkward phrasing.
      This comment should be deleted as it addresses a minor documentation wording issue that doesn't require immediate attention or impact functionality.
    21. documentation/responses/roadmap.mdx:9
    • Draft comment:
      Typographical suggestion: The phrase "The Julep's Open Responses API" in line 9 reads awkwardly. Consider revising it for clarity (e.g., remove the leading "The" if the intended name is "Julep's Open Responses API").
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
      The comment is about a minor grammatical suggestion. While technically correct that "The Julep's" is slightly awkward, this kind of minor wording suggestion in documentation doesn't meet our bar for "clearly requiring a code change". Documentation style/wording preferences are subjective and don't impact functionality.
      The wording could potentially confuse readers, and clear documentation is important for API adoption.
      While clear documentation is important, this minor wording issue isn't confusing enough to warrant a required change - both versions are perfectly understandable.
      Delete this comment as it's a minor stylistic suggestion that doesn't meet our bar for requiring a change.
    22. integrations-service/docker-compose.yml:4
    • Draft comment:
      Typographical error: The anchor name 'x--shared-environment' appears to have an extra dash. Typically, YAML extension fields are named with a single dash (i.e., 'x-shared-environment'). Consider updating it to avoid potential confusion.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    23. schemas/create_agent_request.json:3125
    • Draft comment:
      Typo: The definition 'CompetionUsage' appears to be misspelled. Consider renaming it to 'CompletionUsage' for clarity.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.
    24. typespec/responses/main.tsp:8
    • Draft comment:
      The file is missing a newline at the end. Please add a newline at the end of the file for consistency.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
      Since this is a new file, all code is technically "changed". Missing newlines at end of file can cause issues with some tools and is generally considered good practice. However, this is something that should be enforced by linting rules or editor settings rather than code review comments. It's a very minor issue that doesn't affect functionality.
      The comment is technically correct and follows a common best practice. Missing newlines can cause issues with some version control systems and tools.
      While technically valid, this is exactly the kind of minor, obvious issue that should be handled by automated tools rather than manual review comments. It adds noise to the review without providing significant value.
      Delete this comment as it's too minor and should be handled by automated tooling rather than manual review comments.
    25. typespec/responses/models.tsp:138
    • Draft comment:
      Typo in the doc comment for ToolChoiceTypes: change "should to use" to "should use".
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50%
      While this is a real grammatical error in the documentation, it's a very minor typo fix. The meaning is still clear even with the error. Documentation typo fixes are generally low value unless they significantly impact understanding.
      The error does make the documentation slightly less professional. Some could argue that maintaining high quality documentation is important for API users.
      However, this particular error is so minor that it barely impacts readability or professionalism. The meaning is still completely clear.
      Delete the comment. While it points out a real error, it's too minor to be worth a PR comment. Documentation typos should be batched into larger documentation cleanup efforts.
    26. typespec/tsp-output/@typespec/openapi3/openapi-1.0.0.yaml:10082
    • Draft comment:
      Typographical error: In the Tools.FunctionDef description, 'overriden' should be corrected to 'overridden' for proper spelling.
    • Reason this comment was not posted:
      Comment was not on a location in the diff, so it can't be submitted as a review comment.

    Workflow ID: wflow_BkmoQUyqSSf7rYR2


    Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    # Track if there are more tool calls to process
    has_tool_calls = True

    # Set a reasonable limit for tool call iterations to prevent infinite loops
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    The iterative loop processing tool calls (using while has_tool_calls and iterations < max_iterations:) is complex. Consider refactoring this block into separate helper functions to improve readability, maintainability, and testability.

    Copy link
    Contributor Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    ellipsis-dev bot and others added 14 commits March 29, 2025 07:36
    Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
    Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
    Signed-off-by: Diwank Singh Tomer <diwank.singh@gmail.com>
    …_jWOC
    
    [Ellipsis] feat: add user context to API and embedding requests
    fix(agents-api): add developer id to litellm requests
    feat(agents-api): Add static task validations to catch errors beforehand
    hotfix(agents-api): Fix subquery cardinality issue
    chore(docker): update litellm-database image to version v1.65.0-stable
    feat(agents-api): Made trigram/fuzzy text search parameters configurable + relevant typespec/test changes
    Copy link
    Contributor

    qodo-merge-pro-for-open-source bot commented Apr 1, 2025

    CI Feedback 🧐

    (Feedback updated until commit 7a91610)

    A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

    Action: Test

    Failed stage: Run tests [❌]

    Failed test name: task_validation: Task validator detects invalid Python expressions in tasks

    Failure summary:

    The action failed because the test "task_validation: Task validator detects invalid Python
    expressions in tasks" failed with an AssertionError. The test is located in
    tests/test_task_validation.py.

    The test was expecting the task validator to detect invalid Python expressions in tasks, but the
    validation result showed is_valid=True with empty lists for python_expression_issues, schema_issues,
    and other_issues (lines 2077-2082), which contradicts the expected behavior.

    Additionally, there was a RuntimeWarning about a coroutine that was never awaited: "coroutine
    '_.._resp' was never awaited" in the pygments lexer (line 2031).

    Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    1452:  35/u enhanced_indices (128.939745ms)
    1453:  PASS  test_agent_metadata_filtering:14 query: list_agents with metadata      3%
    1454:  filtering                               
    1455:  PASS  test_agent_metadata_filtering:67 query: list_agents with SQL           3%
    1456:  injection attempt in metadata           
    1457:  filter                                  
    1458:  PASS  test_agent_queries:27 query: create agent sql                          4%
    1459:  PASS  test_agent_queries:43 query: create or update agent sql                4%
    1460:  PASS  test_agent_queries:62 query: update agent sql                          4%
    1461:  PASS  test_agent_queries:89 query: get agent not exists sql                  5%
    1462:  PASS  test_agent_queries:100 query: get agent exists sql                     5%
    1463:  PASS  test_agent_queries:121 query: list agents sql                          5%
    1464:  PASS  test_agent_queries:132 query: patch agent sql                          6%
    1465:  PASS  test_agent_queries:156 query: delete agent sql                         6%
    1466:  INFO:httpx:HTTP Request: POST http://testserver/agents "HTTP/1.1 403 Forbidden"
    1467:  PASS  test_agent_routes:9 route: unauthorized should fail                    6%
    1468:  INFO:httpx:HTTP Request: POST http://testserver/agents "HTTP/1.1 201 Created"
    ...
    
    1537:  PASS  test_docs_queries:205 query: delete user doc                          20%
    1538:  PASS  test_docs_queries:242 query: delete agent doc                         20%
    1539:  PASS  test_docs_queries:279 query: search docs by text                      21%
    1540:  PASS  test_docs_queries:316 query: search docs by text with technical       21%
    1541:  terms and phrases                                  
    1542:  PASS  test_docs_queries:379 query: search docs by embedding                 21%
    1543:  PASS  test_docs_queries:407 query: search docs by hybrid                    22%
    1544:  INFO:httpx:HTTP Request: POST http://testserver/users/067ee812-9030-7f32-8000-06dc61486db0/docs "HTTP/1.1 201 Created"
    1545:  PASS  test_docs_routes:15 route: create user doc                            22%
    1546:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee812-958e-7944-8000-3a44eb5c4fa4/docs "HTTP/1.1 201 Created"
    1547:  PASS  test_docs_routes:32 route: create agent doc                           22%
    1548:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee812-9ae2-7ca1-8000-59de124bf11a/docs "HTTP/1.1 201 Created"
    1549:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee812-9ae2-7ca1-8000-59de124bf11a/docs "HTTP/1.1 409 Conflict"
    1550:  INFO:httpx:HTTP Request: POST http://testserver/users/067ee812-9e55-778f-8000-00e5da222484/docs "HTTP/1.1 201 Created"
    1551:  PASS  test_docs_routes:49 route: create agent doc with duplicate title      22%
    1552:  should fail                                          
    1553:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee812-a3b8-72ec-8000-1adb9e0a5c08/docs "HTTP/1.1 201 Created"
    ...
    
    1622:  PASS  test_execution_queries:33 query: create execution                     29%
    1623:  PASS  test_execution_queries:58 query: get execution                        30%
    1624:  PASS  test_execution_queries:71 query: lookup temporal id                   30%
    1625:  PASS  test_execution_queries:84 query: list executions                      30%
    1626:  PASS  test_execution_queries:103 query: count executions                    31%
    1627:  PASS  test_execution_queries:121 query: create execution transition         31%
    1628:  PASS  test_execution_queries:142 query: create execution transition -       31%
    1629:  validate transition targets                   
    1630:  PASS  test_execution_queries:187 query: create execution transition with    32%
    1631:  execution update                              
    1632:  PASS  test_execution_queries:214 query: get execution with transitions      32%
    1633:  count                                         
    1634:  PASS  test_execution_queries:229 query: list executions with                32%
    1635:  latest_executions view                        
    1636:  PASS  test_execution_queries:252 query: execution with finish transition    33%
    1637:  PASS  test_execution_queries:286 query: execution with error transition     33%
    1638:  SKIP  test_execution_workflow… workflow: evaluate step   needs to be fixed  33%
    1639:  single                                          
    1640:  SKIP  test_execution_workflow… workflow: evaluate step   needs to be fixed  34%
    1641:  multiple                                        
    1642:  SKIP  test_execution_workflo… workflow: variable access  needs to be fixed  34%
    1643:  in expressions                                   
    1644:  SKIP  test_execution_workflo… workflow: yield step       needs to be fixed  34%
    1645:  SKIP  test_execution_workflo… workflow: sleep step       needs to be fixed  34%
    1646:  SKIP  test_execution_workflo… workflow: return step      needs to be fixed  35%
    1647:  direct                                           
    1648:  SKIP  test_execution_workflo… workflow: return step      needs to be fixed  35%
    1649:  nested                                           
    1650:  SKIP  test_execution_workflo… workflow: log step         needs to be fixed  35%
    1651:  SKIP  test_execution_workflo… workflow: log step         needs to be fixed  36%
    1652:  expression fail                                  
    1653:  SKIP  test_execution_workf… workflow: system call   workflow: thread race   36%
    ...
    
    1748:  simple filter                              
    1749:  PASS  test_metadata_filter_utils:37 utility:                                52%
    1750:  build_metadata_filter_conditions with      
    1751:  multiple filters                           
    1752:  PASS  test_metadata_filter_utils:50 utility:                                52%
    1753:  build_metadata_filter_conditions with      
    1754:  table alias                                
    1755:  PASS  test_metadata_filter_utils:66 utility:                                53%
    1756:  build_metadata_filter_conditions with      
    1757:  SQL injection attempts                     
    1758:  PASS  test_mmr:24 utility: test to apply_mmr_to_docs                        53%
    1759:  PASS  test_mmr:61 utility: test mmr with different mmr_strength values      53%
    1760:  PASS  test_mmr:101 utility: test mmr with empty docs list                   53%
    1761:  PASS  test_model_validation:10 validate_model: succeeds when model is       54%
    1762:  available in model list                         
    1763:  PASS  test_model_validation:19 validate_model: fails when model is          54%
    1764:  unavailable in model list                       
    1765:  PASS  test_model_validation:31 validate_model: fails when model is None     54%
    1766:  PASS  test_nlp_utilities:6 utility: clean_keyword                           55%
    ...
    
    1785:  PASS  test_query_utils:5 utility: sanitize_string - strings                 59%
    1786:  PASS  test_query_utils:15 utility: sanitize_string - nested data            59%
    1787:  structures                                           
    1788:  PASS  test_query_utils:41 utility: sanitize_string - non-string types       60%
    1789:  PASS  test_session_queries:37 query: create session sql                     60%
    1790:  PASS  test_session_queries:60 query: create or update session sql           60%
    1791:  PASS  test_session_queries:84 query: get session exists                     61%
    1792:  PASS  test_session_queries:100 query: get session does not exist            61%
    1793:  PASS  test_session_queries:114 query: list sessions                         61%
    1794:  PASS  test_session_queries:131 query: list sessions with filters            62%
    1795:  PASS  test_session_queries:150 query: count sessions                        62%
    1796:  PASS  test_session_queries:164 query: update session sql                    62%
    1797:  PASS  test_session_queries:199 query: patch session sql                     63%
    1798:  PASS  test_session_queries:226 query: delete session sql                    63%
    1799:  INFO:httpx:HTTP Request: GET http://testserver/sessions "HTTP/1.1 403 Forbidden"
    1800:  PASS  test_session_routes:7 route: unauthorized should fail                 63%
    1801:  INFO:httpx:HTTP Request: POST http://testserver/sessions "HTTP/1.1 201 Created"
    ...
    
    1875:  PASS  test_task_execution_workflow:1620 task execution workflow: evaluate   75%
    1876:  yield expressions assertion            
    1877:  PASS  test_task_queries:24 query: create task sql                           75%
    1878:  PASS  test_task_queries:47 query: create or update task sql                 75%
    1879:  PASS  test_task_queries:70 query: get task sql - exists                     76%
    1880:  PASS  test_task_queries:89 query: get task sql - not exists                 76%
    1881:  PASS  test_task_queries:107 query: delete task sql - exists                 76%
    1882:  PASS  test_task_queries:143 query: delete task sql - not exists             77%
    1883:  PASS  test_task_queries:162 query: list tasks sql - with filters            77%
    1884:  PASS  test_task_queries:183 query: list tasks sql - no filters              77%
    1885:  PASS  test_task_queries:201 query: update task sql - exists                 78%
    1886:  PASS  test_task_queries:237 query: update task sql - not exists             78%
    1887:  PASS  test_task_queries:264 query: patch task sql - exists                  78%
    1888:  PASS  test_task_queries:312 query: patch task sql - not exists              78%
    1889:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee815-c246-7357-8000-d0513ae10084/tasks "HTTP/1.1 403 Forbidden"
    1890:  PASS  test_task_routes:27 route: unauthorized should fail                   79%
    1891:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee815-c5c3-7384-8000-a42023520f14/tasks "HTTP/1.1 201 Created"
    ...
    
    1901:  INFO:httpx:HTTP Request: GET http://testserver/tasks/067ee815-d648-74cf-8000-d79333187706 "HTTP/1.1 200 OK"
    1902:  PASS  test_task_routes:124 route: get task exists                           81%
    1903:  INFO:httpx:HTTP Request: GET http://testserver/executions/067ee815-e0ba-7830-8000-be46b611dc72/transitions "HTTP/1.1 200 OK"
    1904:  PASS  test_task_routes:134 route: list all execution transition             81%
    1905:  INFO:httpx:HTTP Request: GET http://testserver/executions/067ee815-eb39-784e-8000-e92c1b22e292/transitions/067ee815-eeb3-733d-8000-c7a47aaa6c6f "HTTP/1.1 200 OK"
    1906:  PASS  test_task_routes:149 route: list a single execution transition        81%
    1907:  INFO:httpx:HTTP Request: GET http://testserver/tasks/067ee813-349f-7d1f-8000-4d0ac587c2f7/executions "HTTP/1.1 200 OK"
    1908:  PASS  test_task_routes:190 route: list task executions                      82%
    1909:  INFO:httpx:HTTP Request: GET http://testserver/agents/067ee815-f24d-7309-8000-9be79a538335/tasks "HTTP/1.1 200 OK"
    1910:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee815-f24d-7309-8000-9be79a538335/tasks "HTTP/1.1 201 Created"
    1911:  INFO:httpx:HTTP Request: GET http://testserver/agents/067ee815-f24d-7309-8000-9be79a538335/tasks "HTTP/1.1 200 OK"
    1912:  PASS  test_task_routes:205 route: list tasks                                82%
    1913:  SKIP  test_task_routes:248 route: update execution   Temporal connection    82%
    1914:  issue              
    1915:  PASS  test_task_validation:7 task_validation: Python expression validator   83%
    1916:  detects syntax errors                             
    1917:  PASS  test_task_validation:16 task_validation: Python expression validator  83%
    1918:  detects undefined names                          
    1919:  PASS  test_task_validation:25 task_validation: Python expression validator  83%
    1920:  allows steps variable access                     
    1921:  PASS  test_task_validation:33 task_validation: Python expression validator  84%
    1922:  detects unsafe operations                        
    1923:  PASS  test_task_validation:42 task_validation: Python expression validator  84%
    1924:  detects unsafe dunder attributes                 
    1925:  PASS  test_task_validation:63 task_validation: Python expression validator  84%
    1926:  detects potential runtime errors                 
    1927:  PASS  test_task_validation:72 task_validation: Python expression            84%
    ...
    
    1954:  PASS  test_tool_queries:21 query: create tool                               89%
    1955:  PASS  test_tool_queries:47 query: delete tool                               89%
    1956:  PASS  test_tool_queries:79 query: get tool                                  89%
    1957:  PASS  test_tool_queries:92 query: list tools                                90%
    1958:  PASS  test_tool_queries:108 query: patch tool                               90%
    1959:  PASS  test_tool_queries:141 query: update tool                              90%
    1960:  PASS  test_user_queries:36 query: create user sql                           91%
    1961:  PASS  test_user_queries:55 query: create or update user sql                 91%
    1962:  PASS  test_user_queries:75 query: update user sql                           91%
    1963:  PASS  test_user_queries:95 query: get user not exists sql                   91%
    1964:  PASS  test_user_queries:111 query: get user exists sql                      92%
    1965:  PASS  test_user_queries:126 query: list users sql                           92%
    1966:  PASS  test_user_queries:141 query: patch user sql                           92%
    1967:  PASS  test_user_queries:161 query: delete user sql                          93%
    1968:  INFO:httpx:HTTP Request: POST http://testserver/users "HTTP/1.1 403 Forbidden"
    1969:  PASS  test_user_routes:9 route: unauthorized should fail                    93%
    1970:  INFO:httpx:HTTP Request: POST http://testserver/users "HTTP/1.1 201 Created"
    ...
    
    1975:  PASS  test_user_routes:53 route: get user exists                            94%
    1976:  INFO:httpx:HTTP Request: POST http://testserver/users "HTTP/1.1 201 Created"
    1977:  INFO:httpx:HTTP Request: DELETE http://testserver/users/067ee816-53e4-7344-8000-9cf3eed3ca60 "HTTP/1.1 202 Accepted"
    1978:  INFO:httpx:HTTP Request: GET http://testserver/users/067ee816-53e4-7344-8000-9cf3eed3ca60 "HTTP/1.1 404 Not Found"
    1979:  PASS  test_user_routes:65 route: delete user                                94%
    1980:  INFO:httpx:HTTP Request: PUT http://testserver/users/067ee816-571c-7223-8000-50dd3b65c0d4 "HTTP/1.1 200 OK"
    1981:  INFO:httpx:HTTP Request: GET http://testserver/users/067ee816-571c-7223-8000-50dd3b65c0d4 "HTTP/1.1 200 OK"
    1982:  PASS  test_user_routes:94 route: update user                                95%
    1983:  INFO:httpx:HTTP Request: PATCH http://testserver/users/067ee816-5a8e-7994-8000-2ed3f25ea423 "HTTP/1.1 200 OK"
    1984:  INFO:httpx:HTTP Request: GET http://testserver/users/067ee816-5a8e-7994-8000-2ed3f25ea423 "HTTP/1.1 200 OK"
    1985:  PASS  test_user_routes:124 query: patch user                                95%
    1986:  INFO:httpx:HTTP Request: GET http://testserver/users "HTTP/1.1 200 OK"
    1987:  PASS  test_user_routes:155 query: list users                                95%
    1988:  INFO:httpx:HTTP Request: GET http://testserver/users?metadata_filter=%7B%27test%27%3A+%27test%27%7D "HTTP/1.1 200 OK"
    1989:  PASS  test_user_routes:170 query: list users with right metadata filter     96%
    1990:  PASS  test_validation_errors:9 format_location: formats error location      96%
    1991:  paths correctly                                 
    1992:  PASS  test_validation_errors:31 get_error_suggestions: generates helpful    96%
    1993:  suggestions for missing fields                 
    1994:  PASS  test_validation_errors:42 get_error_suggestions: generates helpful    97%
    1995:  suggestions for type errors                    
    1996:  PASS  test_validation_errors:64 get_error_suggestions: generates helpful    97%
    1997:  suggestions for string length errors           
    1998:  PASS  test_validation_errors:85 get_error_suggestions: generates helpful    97%
    1999:  suggestions for number range errors            
    2000:  WARNING:agents_api.web:Validation error: [{'type': 'dict_type', 'msg': 'Input should be a valid dictionary', 'loc': 'metadata', 'received': 'not-an-object'}, {'type': 'missing', 'msg': 'Field required', 'loc': 'name', 'fix': 'Add this required field to your request', 'example': '{ "field_name": "value" }', 'received': "{'about': 'Test agent description', 'model': 'invalid-model-id', 'metadata': 'not-an-object'}"}]
    2001:  INFO:httpx:HTTP Request: POST http://testserver/agents "HTTP/1.1 422 Unprocessable Entity"
    2002:  PASS  test_validation_errors:107 validation_error_handler: returns          97%
    2003:  formatted error response for validation       
    2004:  errors                                        
    2005:  PASS  test_validation_errors:148 validation_error_suggestions: function     98%
    2006:  generates helpful suggestions for all         
    2007:  error types                                   
    2008:  PASS  test_workflow_helpers:25 execute_map_reduce_step_parallel:            98%
    ...
    
    2015:  INFO:httpx:HTTP Request: POST http://testserver/tasks/067ee816-622f-7437-8000-2d202b38ea9c/executions "HTTP/1.1 201 Created"
    2016:  PASS  test_workflow_routes:10 workflow route: evaluate step single          99%
    2017:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee816-678e-7539-8000-249da10e15f1/tasks "HTTP/1.1 201 Created"
    2018:  INFO:httpx:HTTP Request: POST http://testserver/tasks/067ee816-6989-74bc-8000-977db2e12085/executions "HTTP/1.1 201 Created"
    2019:  PASS  test_workflow_routes:41 workflow route: evaluate step single with     99%
    2020:  yaml                                             
    2021:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee816-6d04-7e88-8000-c3694a83de24/tasks "HTTP/1.1 201 Created"
    2022:  INFO:httpx:HTTP Request: POST http://testserver/tasks/067ee816-6f16-7b8c-8000-3eff7a33475a/executions "HTTP/1.1 201 Created"
    2023:  PASS  test_workflow_routes:83 workflow route: evaluate step single with    100%
    2024:  yaml - nested                                    
    2025:  INFO:httpx:HTTP Request: POST http://testserver/agents/067ee816-729b-7be2-8000-1aa52baf1502/tasks/067ee816-72dd-7db7-8000-bc180eeb250f "HTTP/1.1 201 Created"
    2026:  INFO:httpx:HTTP Request: POST http://testserver/tasks/067ee816-72dd-7db7-8000-bc180eeb250f/executions "HTTP/1.1 201 Created"
    2027:  PASS  test_workflow_routes:128 workflow route: create or update: evaluate  100%
    2028:  step single with yaml                           
    2029:  ─ task_validation: Task validator detects invalid Python expressions in tasks ──
    2030:  Failed at tests/test_task_validation.py                                       
    2031:  /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packages/pygments/lexer.py:385: RuntimeWarning: coroutine '_.<locals>._resp' was never awaited
    ...
    
    2070:  │ │                     │   │   )                                        │ │  
    2071:  │ │                     │   ],                                           │ │  
    2072:  │ │                     │   input_schema=None,                           │ │  
    2073:  │ │                     │   tools=[],                                    │ │  
    2074:  │ │                     │   inherit_tools=True,                          │ │  
    2075:  │ │                     │   metadata=None                                │ │  
    2076:  │ │                     )                                                │ │  
    2077:  │ │ validation_result = TaskValidationResult(                            │ │  
    2078:  │ │                     │   is_valid=True,                               │ │  
    2079:  │ │                     │   python_expression_issues=[],                 │ │  
    2080:  │ │                     │   schema_issues=[],                            │ │  
    2081:  │ │                     │   other_issues=[]                              │ │  
    2082:  │ │                     )                                                │ │  
    2083:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2084:  ╰──────────────────────────────────────────────────────────────────────────╯  
    2085:  AssertionError                                                                
    2086:  ────────────────────────────────────────────────────────────────────────────────
    2087:  ╭───────────── Results ─────────────╮
    2088:  │  316  Tests Encountered           │
    2089:  │  286  Passes             (90.5%)  │
    2090:  │    1  Failures           (0.3%)   │
    2091:  │   29  Skips              (9.2%)   │
    2092:  ╰───────────────────────────────────╯
    2093:  ─────────────────────────── FAILED in 176.13 seconds ───────────────────────────
    2094:  ##[error]Process completed with exit code 1.
    2095:  Post job cleanup.
    

    fix(agents-api): improve expression validation for comprehensions and unsupported features
    @Ahmad-mtos Ahmad-mtos merged commit 5fa9093 into main Apr 7, 2025
    17 of 18 checks passed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    4 participants