How to Identify Which Functions Were Executed in runWithTools?

[juliandavidmr]

Hi Cloudflare team,

I am using the @hf/nousresearch/hermes-2-pro-mistral-7b model with the runWithTools function and an array of tools (the tools object). I have the following question:

Is it possible to identify which functions within the tools array were actually executed by the model, if any?

For example:

const r = await runWithTools(ai, '@hf/nousresearch/hermes-2-pro-mistral-7b', {
   messages: [.....],
   tools: [....]
});

console.log(r.tool_calls)

I noticed that the constant r returns an object, but the tool_calls field in the response does not seem to include information about the executed functions. My specific questions are:

1. Under what circumstances does the tool_calls field in the response include relevant data?
2. Is there any explicit way to track which functions were called during the model's execution?

Thanks in advance for your help!

[kbhalerao]

I think this may be a bug with how runWithTools is configured. In the code, you can see that it retrieves the tool call messages and shoves it into the messages array with the role of assistant.

By setting the flag verbose: true in the method options, you can see it in the generated response.

A better option may be to push these outputs onto the tool_calls array instead of in the messages array. I'm not sure what the industry spec on this is, but happy to help fix this if appropriate.