Function Calling (Tool Calling)

What is Function Calling in Large Language Models?

Function calling, frequently referred to as tool calling, is a capability that lets a large language model (LLM) interface with external software, APIs, and databases instead of relying only on text generation. By defining a set of available functions (or tools) along with their expected parameters, developers enable models to determine when and how to invoke these functions to fulfill complex user requests.

Instead of directly querying a live database or accessing real-time weather information, an LLM trained for function calling will output a structured data object, typically in JSON format, that precisely matches the developer-provided schema. The host application intercepts this structured output, executes the corresponding real-world code (such as a REST API call, a database query, or a local script), and then returns the results back to the model. This bridge between natural language understanding and deterministic execution allows AI systems to access private information, control smart home devices, browse the internet, and automate multi-step enterprise workflows.

By decoupling the reasoning capabilities of the neural network from the deterministic logic of code execution, function calling effectively mitigates hallucinations. The model learns to rely on authoritative external systems for factual data, calculations, and state-changing operations, significantly expanding the scope of viable AI use cases.

The End-to-End Tool Calling Lifecycle

The process of executing a function call involves a carefully orchestrated dialogue between the LLM and the host application. This lifecycle can be broken down into four distinct phases: Declaration, Model Selection, Execution, and Context Re-injection.

1. Declaration: The developer provides the LLM with a list of available tools. Each tool is defined by a name, a comprehensive natural language description of what it does, and a strict JSON schema detailing the required and optional arguments. The quality of the tool descriptions is critical; the model relies heavily on these semantic cues to determine if a tool is applicable to the user's prompt.

2. Model Selection: When the user submits a prompt, the LLM evaluates the request against the declared tools. If it determines that an external action is necessary to formulate an accurate response, the model will output a specialized token or message type indicating a tool call. Instead of generating conversational text, it synthesizes a structured JSON object containing the chosen tool's name and the specific arguments to be passed, inferred from the user's context.

3. Execution: The model pauses its generation process. The host application parses the JSON output, validates it against the predefined schema, and executes the actual function on the backend. This might involve querying a PostgreSQL database, sending an email via SendGrid, or fetching data from a third-party API. The application captures the output (or error message) of this execution.

4. Context Re-injection: The host application passes the results of the executed function back to the LLM. This is typically done by appending a new "tool response" message to the conversation history. Armed with this new, real-world data, the model resumes generation, synthesizing the function's output into a cohesive natural language response for the user.

This lifecycle underlies agentic AI systems, allowing for dynamic interactions that go beyond static knowledge retrieval.

Parallel and Recursive Tool Calling

As function calling capabilities have matured, modern LLMs support advanced paradigms such as parallel and recursive tool calling, increasing system efficiency and problem-solving capacity.

Parallel Tool Calling: When a user's request involves multiple independent tasks, capable models can synthesize and output multiple function calls simultaneously. For example, if a user asks for the weather in New York, Tokyo, and London, a capable model will generate three distinct get_weather tool calls in a single generation step. The host application can then execute these API requests concurrently, cutting latency and improving the overall user experience.

Recursive Tool Calling: Complex objectives often require a sequence of dependent actions, where the output of one function dictates the input of the next. Recursive tool calling allows the model to enter a loop of continuous interaction. After receiving the result of an initial tool call, the model may realize it needs more information and generate a subsequent, different tool call. For instance, an agentic RAG research assistant might first use a search_web tool to find an article, read the result, and then use a summarize_document tool on the specific URL returned. This recursive reasoning loop continues until the model has gathered sufficient context to finalize its answer, enabling autonomous, multi-step problem solving.

Provider Schema Standards: OpenAI, Anthropic, and Open-Source Models

While the conceptual foundation of function calling is universal, the exact implementation and schema formats vary across major AI providers.

OpenAI pioneered the widespread adoption of function calling, utilizing standard JSON Schema to define parameters. Their API expects tools to be passed in a specific tools array, and the model outputs tool_calls objects containing a unique ID, the function name, and the arguments.

Anthropic's Claude models approach tool use with a similar JSON-based interface, and are uniquely strong at explaining their reasoning before outputting the tool call, a technique known as Chain-of-Thought tool use. Anthropic also co-developed the Model Context Protocol, an open standard for connecting models to external tools and data sources.

Open-weight models such as Llama 4, Qwen3, and DeepSeek-V3.2 have historically relied on prompt engineering to simulate tool use, but current releases are fine-tuned natively for function calling. Frameworks like LangChain, LlamaIndex, and Klu provide abstraction layers to standardize tool schemas across these disparate providers, ensuring portability and reducing vendor lock-in.

Both major providers now offer schema-guaranteed tool arguments via constrained decoding: OpenAI's strict: true on function definitions (since August 2024) and Anthropic's strict tool use (generally available alongside its structured outputs API in 2026). With strict mode enabled, argument payloads cannot be syntactically malformed, which eliminates one whole class of validation failures, though the model can still pick the wrong tool or pass semantically wrong values.

Tool definitions are not free. Everything in the tools array (names, descriptions, parameter schemas) bills as input tokens on every request, and providers inject an additional tool-use system prompt: Anthropic documents this at roughly 290 to 800 tokens depending on model and tool-choice setting, and its full computer-use toolset adds about 4,500 tokens of definitions. Large tool catalogs are a common hidden cost driver, which is why production agents cache the tool block with prompt caching and prune unused tools rather than declaring everything.

Benchmarks: How Tool-Calling Ability Is Measured

The Berkeley Function-Calling Leaderboard (BFCL) is the de facto standard evaluation. Its v4 release (presented at ICML 2025) moved beyond single-shot calls to holistic agentic evaluation: simple, parallel, and multi-turn calling plus web search, memory, and format-sensitivity tracks. The consistent finding across BFCL results is that frontier models have largely saturated single-turn function calling, while multi-turn tool use with state tracking and long-horizon decision-making remains the differentiator. Complementary benchmarks such as tau-bench (and its successor tau2-bench) measure the same skills in simulated customer-service conversations where the agent must follow policy while calling tools. For provider selection, these benchmarks matter more than marketing claims: rankings shift with every model release, so check the live leaderboard rather than relying on a printed snapshot.

Error Recovery, Schema Validation, and Self-Correction Loops

Even the most advanced LLMs can occasionally generate malformed JSON, omit required parameters, or hallucinate non-existent tools. Robust function calling systems incorporate defensive engineering practices, primarily through schema validation and self-correction loops, which form one layer of broader LLM guardrails.

When the host application receives a tool call from the model, it must strictly validate the payload against the original JSON schema. If the model outputs a string where an integer is expected, the application should intercept the error rather than crashing or propagating bad data to the backend.

Modern LLMs can also self-correct. When a schema validation error occurs, or if the external API returns a functional error (e.g., "404 Not Found" or "Invalid Date Format"), the host application can inject this error message back into the context window as a tool response. The model, reading its own mistake and the resulting error, will attempt to rectify the issue by generating a corrected tool call. This feedback loop lets the AI agent navigate ambiguous instructions and recover from structural or logical errors, which becomes especially important in multi-agent orchestration systems where one agent's tool failure can otherwise cascade into another's input.

Frequently Asked Questions

Can I use function calling to execute arbitrary code? Not directly. The model can only select from tools the developer declared and populate their arguments; it cannot invent new functions or step outside the schema. But schema validation only constrains the shape of a call, not whether making it is a good idea: a model whose context has been prompt-injected by untrusted tool output or document content can still request a legitimate tool with harmful arguments, such as a file-write tool pointed at a sensitive path or an email tool aimed at exfiltrating data. Real security lives in the host application, not the schema: authorize each call against the current user's permissions, issue tools least-privilege credentials scoped to what they need, validate arguments against business rules in addition to types, and run execution in a sandbox that limits blast radius. Treat the model as a decision-engine that can be manipulated, not a trusted gatekeeper.

Do I need to fine-tune a model to use tools? While older models required specific fine-tuning, modern frontier models are natively trained on tool-use datasets and offer excellent out-of-the-box function calling performance using only prompt engineering and well-defined JSON schemas.

What happens if the model hallucinates a tool? Robust applications should always validate the model's output. If the model requests a tool that does not exist in the defined schema, the application should catch the error and prompt the model to try again using only the provided tools, usually through a self-correction loop.

How does function calling affect latency and costs? Function calling introduces additional latency and costs, as it often requires multiple round trips to the LLM API. The initial generation, the wait for external execution, and the final context re-injection all consume tokens and processing time, making optimization techniques like parallel execution vital for production environments.