$ cat ai-engineering/tool-use.md

Function Calling and Tool Use

LLM tool use as typed RPC with schema validation, dispatch, retries, and result handling.

Jatin Bansal@blog:~/ai-engineering$ open tool-use

A travel-booking assistant ships on a Tuesday. Forty-eight hours in, a customer asks it to “rebook me on the same flight tomorrow at the same fare.” The model’s text reply describes, in confident prose, that it has changed the booking. It has not. There is no booking system attached. The model can describe actions but it cannot take them; every side effect lives outside the LLM, and bridging that gap is the entire job of function calling. Tool use is how the model stops narrating the world and starts touching it.

Tool calls

Function calling, tool use, function invocation; different vendors, same idea. You declare a set of typed functions the model is allowed to ask for. The model decides, at decode time, whether to emit a normal text response or a structured request to call one of those functions. Your runtime executes the request, packages the result, and feeds it back into the next turn. The model can then call more tools, or stop and answer.

A tool is three things:

A name the model uses to address it.
A description; natural-language prose explaining when to call it and when not to. The description is the API doc for an LLM consumer; treat it that way.
An input schema; JSON Schema describing the arguments, with required fields, types, enums, and format constraints. The model’s emitted arguments are validated against this schema.

The model emits a structured tool request, and the application runtime executes the function. Tool calls are produced by the same decoder as text, using behavior learned from tool-use training data.

Typed RPC

Tool use resembles a remote procedure call where the LLM is the caller and the application runtime is the server. The model selects a function and supplies typed arguments. The runtime validates and executes the call, serializes the result, and returns control to the model.

Once you frame it this way, the whole surface area follows. Tool schemas are IDL. Tool descriptions are docstrings on the IDL. The agent loop is an RPC client driver. Timeouts, retries, idempotency, error propagation, and observability; every problem that the RPC literature solved in the 1980s reappears here, with the twist that the caller is non-deterministic.

Call and result loop

A single tool-using exchange looks like this on Anthropic’s API:

You send messages plus a tools array. The tools array is a list of objects with name, description, and input_schema.
The model responds with stop_reason: "tool_use" and an assistant message whose content is a list of blocks; typically a text block followed by one or more tool_use blocks, each with an id, a name, and a JSON input.
You execute each requested tool and append a user message whose content is a list of tool_result blocks. Each tool_result carries the matching tool_use_id, the result payload, and an optional is_error: true flag.
You re-call the API with the appended history. The model either calls more tools (back to step 2) or stops with stop_reason: "end_turn" and a final text response.

OpenAI’s surface is conceptually identical with different field names: tools[].type: "function", tools[].function.{name, description, parameters}, the model emits tool_calls on the assistant message, and you reply with messages of role: "tool" carrying a tool_call_id. The shapes are nearly isomorphic (Anthropic’s tool-use guide, OpenAI’s function-calling guide). The semantic differences worth memorizing:

Forcing a call. Anthropic’s tool_choice is {type: "auto" | "any" | "tool" | "none"}. any forces some tool; tool forces a specific one. OpenAI’s tool_choice accepts "auto" | "required" | "none" or {type: "function", function: {name}}; required is the equivalent of any.
Parallel tool calls. Both providers will emit multiple tool_use/tool_calls blocks in a single assistant message when the calls are independent. To opt out; useful for tools that mutate state and must be serialized; Anthropic exposes disable_parallel_tool_use: true (on the tool_choice object), OpenAI exposes top-level parallel_tool_calls: false.
Strict schemas. OpenAI’s strict mode (strict: true on a tool) and Anthropic’s strict tool use both schema-validate tool inputs at decode time, the same FSM-over-vocabulary trick used by schema-constrained structured output. Use it on tools whose inputs you’d rather not re-validate in application code.
Tool descriptions are not free. Every tool’s schema and description is serialized into a hidden system prompt on every call. A library of 50 tools can easily cost 5–10k tokens per turn. This is the wedge that motivates MCP and dynamic tool routing; covered in depth in the tool-selection-at-scale article.

Anthropic tool loop in Python

A two-tool example; one for weather, one for time; using the official Anthropic SDK. Install: pip install anthropic.

python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
import json
from anthropic import Anthropic

client = Anthropic()

TOOLS = [
    {
        "name": "get_weather",
        "description": (
            "Get the current weather for a given location. Returns temperature "
            "and conditions. Use only when the user asks about weather; do not "
            "use for forecasts more than 24 hours out."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and state, e.g. 'San Francisco, CA'",
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit (default: celsius)",
                },
            },
            "required": ["location"],
        },
    },
    {
        "name": "get_current_time",
        "description": "Get the current local time for an IANA timezone like 'America/Los_Angeles'.",
        "input_schema": {
            "type": "object",
            "properties": {"tz": {"type": "string"}},
            "required": ["tz"],
        },
    },
]

def execute_tool(name: str, args: dict) -> str:
    if name == "get_weather":
        return json.dumps({"temp": 18, "unit": args.get("unit", "celsius"), "conditions": "fog"})
    if name == "get_current_time":
        return json.dumps({"tz": args["tz"], "time": "2026-05-18T11:14:00-07:00"})
    return json.dumps({"error": f"unknown tool: {name}"})

def run_loop(user_msg: str, max_steps: int = 8) -> str:
    messages = [{"role": "user", "content": user_msg}]
    for _ in range(max_steps):
        resp = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=1024,
            tools=TOOLS,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": resp.content})

        if resp.stop_reason != "tool_use":
            return "".join(b.text for b in resp.content if b.type == "text")

        results = []
        for block in resp.content:
            if block.type != "tool_use":
                continue
            try:
                output = execute_tool(block.name, block.input)
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": output})
            except Exception as e:
                results.append({
                    "type": "tool_result", "tool_use_id": block.id,
                    "content": f"error: {e}", "is_error": True,
                })
        messages.append({"role": "user", "content": results})

    raise RuntimeError("max_steps exceeded")

The entire resp.content array is appended as the assistant turn because the API expects the original block list. Tool errors are returned as is_error: true results so the model can recover from timeouts, upstream failures, or invalid arguments without terminating the loop.

The step cap is the load-bearing safety net. Without it, a confused model can loop indefinitely, especially on ambiguous tasks where each tool result triggers another retrieval. The JIT context-engineering article called out tool-loop drift as JIT’s silent failure mode; the step cap is the bluntest version of the no-progress detector recommended there.

Failure modes

The token tax on tools is invisible until it isn’t. Every tool in the tools array is serialized into a system prompt on every call, even when the user asks a question that has nothing to do with any tool. With 30+ tools and verbose descriptions, you can burn 8–12k tokens before the user message starts. The mitigations are real but bounded: trim descriptions, consolidate fine-grained tools into action-parameterized ones (per Anthropic’s tool-writing guidance), and turn to MCP-style dynamic tool routing once N > ~30.

Parallel tool calls assume independence. If the model emits transfer_funds(A→B) and transfer_funds(B→C) in the same turn, your runtime executes them concurrently; and the model has no notion of ordering between them. For state-touching tools, force serial execution: disable_parallel_tool_use: true on Anthropic, parallel_tool_calls: false on OpenAI. The default is “concurrent unless you opt out,” which is the wrong default for most production CRUD tools.

Tool selection collapses past ~30 tools. Once the tool count grows large, accuracy on tool selection drops faster than you’d expect; the model has trouble disambiguating between similar tools, and descriptions start interfering with each other. The standard fix is two-stage retrieval: embed the tool descriptions, retrieve a top-k subset relevant to the user’s turn, and only pass those into the API. Anthropic ships a built-in tool search tool for this; the tool-selection-at-scale article covers the pattern in depth.

Hallucinated arguments. The model occasionally invents tool inputs that aren’t in the schema (get_weather(zipcode: "94103") when the schema only accepts location). Strict mode catches this at decode time. Without strict, validate every argument before execution and return a clear error in tool_result; the model is good at correcting on a single round-trip given a precise error message; don’t crash the loop with a Python KeyError.

tool_choice: "any" prefills the assistant. When you force a tool call, the API prefills the assistant turn with the start of a tool_use block. The model cannot precede it with a natural-language explanation or reasoning, and forcing any together with extended thinking is currently rejected on Claude. If you want the model to “think first then call,” use auto with explicit instructions, or call without forcing and check stop_reason.

Idempotency at the runtime, not at the model. If a tool_result doesn’t arrive cleanly, the easy thing is to retry; and the model will happily call the same tool again on the next turn if the conversation is replayed. Any tool that mutates state needs an idempotency key at the runtime layer; specifically, at the harness, not in each tool author’s code. Treat the LLM as an at-least-once caller, not an exactly-once one.

Schema strictness fights schema expressiveness. Strict mode requires every property to be required (use union with null for optional fields) and forbids additionalProperties: true. Existing schemas with oneOf, recursive references, or unbounded objects may not compile. The same subset that bit you in the structured-output article bites again here; the production answer is usually to define the tool input as the smallest legal envelope and validate the richer constraints in your execute function.

Tool descriptions are prompt code, version them. Edits to a tool’s description change the model’s call rate the same way edits to a prompt change behavior. They belong in version control and behind eval gates; treat description churn as production-affecting change, the same way you’d treat a prompt edit.