You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I notice that when multiple tools are requested by the Agent the ToolNode node executes all tools in parallel. Although this might make execution time shorter it causes problems if the Agent is expecting the tools to execute on the specified order.
Keeping tool execution order the same as expected by the LLM is desired if the tools executed are against a stateful system.
This can be the case when working with: Databases, API's, Desktop/Browser manipulation, etc..
Example scenario I just encountered; I am building a tool-set for Desktop Control (how creative i know) and given a task the LLM will try to execute multiple tools at the same time.
Move the mouse.
Click on element.
Type (text) on keyboard.
Click Enter.
Tool calls as the above require that the tools get invoked on the order specified by the LLM.
Currently this is not possible as ToolNode will try to execute the tools parallelly and there is no telling which one will end first. Clicking Enter will probably finish faster than typing or moving the mouse.
Solution:
I believe that the ToolNode should have an option to force tool calls to happen sequentially, thus guaranteeing a predictable execution of the tasks, and also adds the guarantee that if one tool call fails subsequent tool calls will not be invoked which could further leave the target system of the tool calls on a broken state.
It may even be safer for ToolNode to execute tools sequentially as the default mode and parallel as the opt-in approach.
I have already implemented a version of ToolNode for my project to fit this change and it was quite easy to do so. I can create a PR with the change to ToolNode if this feature is liked.
The text was updated successfully, but these errors were encountered:
Problem:
I notice that when multiple tools are requested by the Agent the
ToolNode
node executes all tools in parallel. Although this might make execution time shorter it causes problems if the Agent is expecting the tools to execute on the specified order.Keeping tool execution order the same as expected by the LLM is desired if the tools executed are against a stateful system.
This can be the case when working with: Databases, API's, Desktop/Browser manipulation, etc..
Example scenario I just encountered; I am building a tool-set for Desktop Control (how creative i know) and given a task the LLM will try to execute multiple tools at the same time.
Tool calls as the above require that the tools get invoked on the order specified by the LLM.
Currently this is not possible as
ToolNode
will try to execute the tools parallelly and there is no telling which one will end first. Clicking Enter will probably finish faster than typing or moving the mouse.Solution:
I believe that the ToolNode should have an option to force tool calls to happen sequentially, thus guaranteeing a predictable execution of the tasks, and also adds the guarantee that if one tool call fails subsequent tool calls will not be invoked which could further leave the target system of the tool calls on a broken state.
It may even be safer for ToolNode to execute tools sequentially as the default mode and parallel as the opt-in approach.
I have already implemented a version of ToolNode for my project to fit this change and it was quite easy to do so. I can create a PR with the change to ToolNode if this feature is liked.
The text was updated successfully, but these errors were encountered: