The model never runs your function. It fills out a form: a tool name and JSON arguments, and your code does every bit of the actual work. Put affirmatively, function calling is the mechanism that lets an LLM request an external action: given a list of tool definitions, the model returns a tool name plus JSON arguments, and a separate runtime executes it. OpenAI states this plainly: the model decides which function to call and what arguments to pass, but your backend does the execution. Anthropic frames Claude the same way: it returns a tool_use block your application runs before you return a tool_result. The model proposes. Your code disposes.
Function calling is the model filling out a form, not running your code. The model proposes. Your code disposes.
Five things most tutorials never make explicit: (1) The model never executes code. It fills out a form: a tool name plus JSON arguments. (2) Your code runs the function and reads the result back to the model in a second turn. (3) Tool definitions are JSON Schema. They describe options, not implementations. (4) Models can request several tools in one turn. (5) Strict structured outputs can guarantee argument shape, but never argument correctness.
- tool_use: the block Claude returns naming a tool and its arguments.
- tool_result: the block you send back carrying what your code produced.
- tool_choice: controls whether the model must call a tool (auto, any, or a named tool).
- stop_reason: tool_use signals Claude paused to wait for your execution.
- strict / Structured Outputs: forces arguments to match your JSON Schema shape.
- disable_parallel_tool_use: caps the model to at most one tool per turn.
A tool call is just structured text
The LLM produces text, and a tool call is just structured text. When you pass a list of tools alongside your prompt, the model can respond with a request to use one: a name and a set of arguments encoded as JSON. It does not open a network socket, query a database, or import a library. OpenAI describes the response as the model specifying the tool name and the parameters it wants passed, after which the result goes back to the model so it can write a final reply.
The tool definition is a form, and the model fills it out. It writes the function name in one field and the arguments in the others. Your application is the clerk that takes the completed form, performs the real action, and reports back what happened.
How function calling works, step by step
A complete function call spans two model turns with your code in the middle. Trace one request for the weather in Paris. You send the prompt plus a get_weather tool definition; the model replies not with an answer but with a structured request to call get_weather with location set to Paris.
- Turn 1, request: You send the user message and the tool definitions to the API.
- Turn 1, response: The model returns a tool call (OpenAI) or a tool_use block (Anthropic), and stops. With Claude, the response carries stop_reason of tool_use.
- Your code: You parse the arguments, call the real weather API, and get back a number.
- Turn 2, request: You append the model's tool call and your tool result to the conversation and send it back.
- Turn 2, response: The model reads the result and writes the natural-language answer for the user.
The waiting is the point. Anthropic documents this sequence: Claude responds with a tool_use block, your code executes the operation, and you send back a tool_result so the model can formulate a final response. The model has no way to fetch the weather itself, so it pauses and depends on you.
What is a tool definition? JSON Schema, name, and description
A tool definition is a JSON Schema describing the function's name, purpose, and parameters, and nothing about how it works. You give the model a name, a natural-language description, and a typed list of arguments. When you call the Claude API with the tools parameter, the API constructs a special system prompt from those definitions so the model knows the menu of available actions.
The description field does heavy lifting. It is the signal that tells the model when get_weather is the right choice versus get_stock_price. Anthropic's guidance stresses extremely detailed tool descriptions because the model decides when and how to use tools based largely on those descriptions and the user's request. A vague description produces a vague selector. The schema constrains the shape of the arguments; the description constrains the judgment about whether to call at all.
Write tool descriptions for a new engineer, not for yourself. State what the tool does, when to use it, when not to, and what each argument means. The model has no access to your source code, only to the words in the schema.
How does the model decide which tool to call?
By default the model decides on each turn whether to call a tool or answer directly. Anthropic's default tool_choice of auto lets Claude choose, per turn, between calling a tool and replying in plain text. You can force the issue: setting tool_choice to any or to a specific tool requires the model to call something, which is useful when you know a tool is mandatory.
Choosing not to call a tool is a feature, not a failure. If a user asks a question the model can answer from its own training, forcing a tool call wastes a round-trip and invites errors. The decision is probabilistic, driven by the prompt, the conversation, and the tool descriptions. That is why two near-identical prompts can produce different tool choices, and why description quality changes behavior more than most developers expect.
Can an LLM call multiple tools at once? Parallel tool use
A model can request several tools in one turn. By default Claude may use multiple tools to answer a single query, returning multiple tool_use blocks in one response. Anthropic notes that the tool calls in a single turn are unordered: you can run them concurrently, sequentially, or in any order, and the model does not assume one finished before another.
The execution is still yours. You receive a batch of requests and decide how to fan them out, perhaps with Promise.all or asyncio.gather. Then you return every result in one user message whose content carries the matching tool_result blocks. A frequent bug is sending each tool result in its own separate user message, which teaches the model to stop calling tools in parallel. You can also cap concurrency: setting disable_parallel_tool_use to true forces at most one tool per turn.
Sending the tool result back to the model
The model cannot use a tool result it never receives. After your code runs the function, you must append both the model's tool call and your result to the message history and call the API again. Skip this and the model is left holding a request with no answer, unable to finish its reply. The second turn is where the model converts a raw number into a sentence a user can read.
This is also where context cost compounds. Every turn re-sends the full history: the original prompt, the tool definitions, prior tool calls, and prior results. In a multi-step task with several tools, the same definitions and earlier outputs ride along on every request. The model is stateless between calls, so the burden of remembering falls entirely on the bytes you resend.
Why function calling fails: wrong tool and hallucinated arguments
Here is the failure nobody warns you about: a perfectly valid argument that is completely wrong. The two common failures are wrong-tool selection and invented arguments. The Berkeley Function-Calling Leaderboard, a standard benchmark, evaluates whether a model picks the correct function without being misled by unrelated tool docs and whether it knows when not to call anything at all. Even strong models still produce incorrect function calls in more complex multi-tool scenarios, selecting the wrong tool or passing mismatched parameters.
Strict schemas help with shape, not substance. OpenAI's Structured Outputs, enabled with strict set to true, ensures the model always generates arguments that adhere to your supplied JSON Schema. But matching the schema is not the same as being right. A value can be perfectly typed and still be the wrong city. Strict mode stops malformed JSON. It does not stop a confident, well-formatted lie, so you still have to validate.
Validate arguments before you execute. Treat every value the model produces as untrusted input: check enums against a real list, confirm IDs exist, and reject out-of-range numbers. Schema validity is not the same as correctness.
Function calling vs structured outputs vs agents
These three are related but distinct. Function calling produces a request for an external action that your code runs. Structured outputs constrain any model response to a schema, with no execution implied. An agent is a loop that chains many function calls toward a goal, deciding the next step from the last result, the pattern formalized by the ReAct framework that interleaves reasoning and actions.
| Aspect | Function calling | Structured outputs | Agents |
|---|---|---|---|
| What the model emits | A tool name plus JSON arguments | A JSON object matching a schema | A sequence of tool calls over many turns |
| Does the model run code? | No, your code does | No code runs at all | No, your runtime executes each step |
| Number of turns | Two: request then result | One | Many, looped until done |
| Primary purpose | Trigger an external action | Force a parseable answer shape | Pursue a multi-step goal |
| Failure to watch | Wrong tool or invented argument | Correct shape, wrong values | Compounding errors across steps |
Read the second row across: in every column, the answer is no. The model never runs anything. The lines blur because agents are built from function calls, and function calls often use structured outputs to lock the argument shape. But conflating them hides the core fact: in all three, the model emits text, and a separate runtime turns that text into action.
Cutting the re-sent tool context
Because the model is stateless, a tool-heavy session re-sends the same definitions and prior results every turn, which inflates cost and can crowd out the signal the model needs to pick the right tool. Trimming what you resend, and storing durable facts outside the prompt, keeps each turn focused.
- What gets re-sent each turn, turn 1: system prompt plus tool definitions plus the user message.
- What gets re-sent each turn, turn 2: everything from turn 1, plus the model's tool_use block, plus your tool_result.
- Across a multi-step agent: the same tool definitions ride along on every single turn, growing the request as the conversation grows.
Where MemX fits: if your tool-calling loop keeps re-sending the same reference material every turn, an external memory layer like MemX (memx.app) lets you store it once and retrieve only the relevant slice per step. MemX captures your documents, screenshots, photos, voice notes, and forwarded messages, indexes them, and lets you query them against any model. That keeps the prompt lean and the tool-selection signal clear. It does not change how function calling works; it reduces the context you have to resend each turn.
Key takeaway: Function calling is the model filling out a form, not running your code. It emits a structured request, your runtime executes it, and you feed the result back in a second turn. Strict schemas guarantee the form is shaped right, never that the answers on it are correct, so validate every argument before you act. The model proposes. Your code disposes.
01Does the model actually run my function?
No. The model writes the request; your code runs the function. The model emits a tool name and JSON arguments, then your application executes it and sends the result back. OpenAI and Anthropic both document that execution happens in your application, not inside the model.
02How many turns does a function call take?
Two. Turn one: the model returns a tool call and stops. Your code runs the function. Turn two: you send the result back and the model writes the final answer. Without that second turn, the model holds a request with no answer and cannot finish its reply.
03Can an LLM call two tools at once?
Yes. A model can return multiple tool calls in a single turn. Anthropic notes these are unordered, so you can run them concurrently or sequentially. The model runs none of them; your code executes each and returns all results together in one user message.
04Why did the model invent a function argument?
The model predicts a plausible value when it lacks the real one, a known function-calling failure. Strict structured outputs fix argument shape, not correctness. Validate each argument against real data before executing, since a well-formed value can still be the wrong city.
05Is function calling the same as an agent?
No. An agent is a loop that chains many function calls toward a goal, deciding each next step from the last result. A single function call is one request-execute-respond round-trip. Agents are built from function calls, but a function call is not an agent.
