$ cat ai-engineering/agent-loop.md

The Agent Loop: ReAct and Its Descendants

How ReAct agents execute tools, process observations, enforce stop conditions, and compare with plan-and-execute.

Jatin Bansal@blog:~/ai-engineering$ open agent-loop

A tool-using agent repeatedly asks a model for the next action, executes that action, and returns the observation. The runtime must decide when to stop, validate tool arguments, enforce budgets, and preserve enough state for the next iteration. Without those controls, a capable model can still enter an expensive loop that never produces a final answer.

The agent control loop

Simon Willison uses a practical definition: “An LLM agent runs tools in a loop to achieve a goal.” It identifies the model, tool interface, and iterative control flow.

LLM: The decision-maker. Picks the next tool call, interprets the result, decides when to stop.
Tools in a loop: Not one call, not one tool; repeated invocations where each result feeds the next decision. This is the part that separates an agent from a glorified function call.
To achieve a goal: There’s a terminal condition. Without it you have an infinite loop, not an agent.

Lilian Weng’s earlier framing; “Agent = LLM + memory + planning skills + tool use” (June 2023); adds the two ingredients that the loop sits on top of: memory (the conversation history is the cheapest, and the only one most agents need) and planning (which may be implicit in the loop or explicit as a separate step, see below).

Frame it operationally and the loop is a control structure with five moving parts: the driver (your code) calls the model with the context (prompt + history + tool schemas); the model emits tool calls that the driver executes against a tool surface; results splice back into context; loop. Every word in that sentence has its own engineering decisions. Most production failures are in the driver, not the model.

ReAct: the canonical loop

The pattern that nailed the loop down for LLMs is ReAct (Yao et al., 2022); Reasoning + Acting. The model alternates three step types:

Thought: Free-text reasoning about what to do next. The model talks to itself.
Action: A tool invocation with typed arguments; the structured request covered in the tool-use article.
Observation: The tool result, fed back into the context for the next thought.

ReAct’s original framing was prompt-only; you few-shot the model with examples of Thought:/Action:/Observation: triples and it learns the rhythm. Modern LLMs internalize this without examples; the prompt-only framing has been absorbed into the post-training data of every frontier model. What ReAct contributed wasn’t a prompt template; it was the insight that explicit reasoning steps between actions sharply improve tool-using accuracy. On HotpotQA the original paper saw the model overcome hallucination by reasoning aloud about what it had observed and what it still needed; on ALFWorld and WebShop the gains over imitation learning were 34 and 10 absolute percentage points respectively.

The mechanical artifact of ReAct in 2026 is the structured assistant turn: a text block containing the thought, followed by one or more tool_use blocks containing the action. Anthropic’s tool-use API bakes this in; the response with stop_reason: "tool_use" is exactly one ReAct iteration. Set extended_thinking and the thought becomes a separate thinking block; the structure is the same. OpenAI’s tool calls have the same shape with different field names.

Execute one iteration

Walk through a single ReAct iteration end to end. The driver holds three pieces of state: a messages list (the conversation), a tools array (the schemas), and a step counter.

Call the model: The driver POSTs messages and tools to the provider. With prompt caching configured on the tool block and system prompt, this call is cheap; without it, you re-prefill the tools on every step.
Receive an assistant turn: The response is a list of content blocks. The driver appends the entire content array back to messages; the API expects byte-identical blocks on the next call. This is the part most home-grown loops botch: stripping the thought blocks and keeping only the tool calls breaks future-turn coherence on every frontier model.
Inspect stop_reason: If it isn’t "tool_use" (or the OpenAI equivalent "tool_calls"), the loop is done; return the assistant text.
For each tool_use block, dispatch: Look up the tool, validate args (or trust strict-mode constraints), execute. Capture the result, the success/error flag, and the duration.
Append a user turn of tool_result blocks: Each must carry the matching tool_use_id. Errors go in the same shape with is_error: true; the model recovers cheaply from a typed error message; raising a Python exception out of the loop is almost never what you want.
Increment the step counter: Compare against the cap. If exceeded, abort with whatever partial progress you have.

That’s it. Every production agent loop in existence is some elaboration of this six-step body. The elaborations are where the interesting decisions live.

Enforce stopping conditions

The runtime decides when to stop. A model may continue requesting tools because each observation suggests another possible check. The driver enforces the limit independently of the prompt.

A defensible stopping condition is a disjunction of cheap predicates evaluated after each step:

stop_reason != "tool_use": The natural exit; the model didn’t ask for another tool. This is the only stopping condition the model fires; everything else is the driver.
Step cap: Hard maximum number of iterations, no negotiation. The Vercel AI SDK’s default is stepCountIs(20); most production agents tune this between 5 and 50 depending on task variance.
Wall-clock deadline: Total time across the loop. Critical for user-facing chat where p99 latency matters; less critical for batch agents.
Token budget: Sum of input + output tokens across all calls. The cheapest predicate to evaluate, the easiest to forget.
Dollar budget: Same shape as tokens, denominated in money. Worth maintaining as a separate counter because the dollar/token ratio differs across cached vs uncached input and across models.
No-progress detection: Did the last N steps make distinguishable changes to state? Two identical tool calls in a row is a strong signal the loop is stuck; three is decisive. The simplest implementation is hashing the (tool_name, args) tuple and looking for repeats.
Goal predicate: “Has the model called the submit_final_answer tool?”; a tool whose only purpose is to terminate the loop. Vercel’s hasToolCall(name) is exactly this pattern.

The disjunction matters. Step cap alone is a hammer that takes effective work and partial results with it; a goal predicate alone never fires for a confused model that hallucinates the wrong-named tool. Compose them: stop on the first of (stop_reason, step cap, deadline, budget, no-progress, goal). The fork-bomb parallel from operating systems is exact; every kernel needs an OOM killer, every agent needs an enforced budget. The agent-budgets-and-runaway-prevention article is the dedicated walk-through: the seven primitives every budget needs (step cap, deadline, token ceiling, dollar cap, per-tool quota, no-progress streak, oscillation detection, external abort), the OS heritage of each one, and the runnable harness implementations.

Plan-and-execute: the batching alternative

ReAct normally uses one model call per tool decision. Twelve tool invocations can therefore require twelve model round trips, repeated prefix processing, and repeated network latency. Predictable tasks may be cheaper with a plan-and-execute design. Planning Agents vs Reactive Agents compares the cost of replanning with ReAct, ReWOO, and search-based variants.

Plan-and-execute inverts the loop. First call: ask the model to produce a step-by-step plan as a structured object (a JSON list of typed steps). Then execute the steps sequentially; either with smaller, cheaper executor calls or with raw code, depending on how rigidly the plan can be specified. Optionally re-plan after each step or on failure. LangChain’s plan-and-execute writeup is the canonical reference; the original research lineage runs through Plan-and-Solve (Wang et al., ACL 2023).

The advantages and disadvantages are mirror-image to ReAct’s:

	ReAct	Plan-and-execute
LLM calls per task	One per step	One plan + one per step (executor can be smaller)
Adaptability	High; every step replanned implicitly	Low; plan is fixed until re-planning
Quality on well-specified tasks	Variable; model can drift mid-task	Better; plan forces whole-task reasoning
Quality on ill-specified tasks	Better; reacts to surprises	Worse; plan may be wrong; replanning is expensive
Latency	Higher; serial round-trips	Lower; executor can use smaller model
Debuggability	Hard; reasoning is interleaved	Easier; plan is a typed artifact
Token cost	High; model + tools re-prefilled	Lower; executor sees less

A hybrid can use ReAct while the path is uncertain, switch to a plan when the remaining sequence is predictable, and return to ReAct after an unexpected result. Reflexion adds a written reflection after a failed attempt without updating model weights. Tree of Thoughts explores and scores several candidate paths when branching is worth the extra calls.

Code: a ReAct loop in Python with the Anthropic SDK

A research-assistant agent with three tools: web search, document fetch, and final answer. Install: pip install anthropic. Uses the Anthropic SDK.

python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
import json, time
from anthropic import Anthropic

client = Anthropic()

TOOLS = [
    {
        "name": "search_web",
        "description": "Search the web. Returns a list of {title, url, snippet}. "
                       "Use for any question whose answer isn't in working memory.",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}, "k": {"type": "integer", "default": 5}},
            "required": ["query"],
        },
    },
    {
        "name": "fetch_url",
        "description": "Fetch the readable text of a single URL. Use after search_web "
                       "to read a promising result.",
        "input_schema": {
            "type": "object",
            "properties": {"url": {"type": "string"}},
            "required": ["url"],
        },
    },
    {
        "name": "submit_final_answer",
        "description": "Submit the final answer. Calling this terminates the loop.",
        "input_schema": {
            "type": "object",
            "properties": {"answer": {"type": "string"}, "sources": {"type": "array", "items": {"type": "string"}}},
            "required": ["answer"],
        },
    },
]

# stub tool implementations — wire to real services in prod
def search_web(args): return [{"title": "Example", "url": "https://example.com/x", "snippet": "..."}]
def fetch_url(args):  return {"url": args["url"], "text": "<page text>"}
TOOL_FNS = {"search_web": search_web, "fetch_url": fetch_url}

def run(goal: str, *, max_steps=12, deadline_s=60.0, max_tokens_total=200_000):
    messages = [{"role": "user", "content": goal}]
    started = time.monotonic()
    tokens_used = 0
    last_calls: list[tuple] = []  # for no-progress detection

    for step in range(max_steps):
        # --- check budgets BEFORE the call so we never overspend ---
        if time.monotonic() - started > deadline_s:
            raise TimeoutError(f"deadline exceeded at step {step}")
        if tokens_used > max_tokens_total:
            raise RuntimeError(f"token budget exhausted at step {step}")

        resp = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=2048,
            tools=TOOLS,
            messages=messages,
        )
        tokens_used += resp.usage.input_tokens + resp.usage.output_tokens
        messages.append({"role": "assistant", "content": resp.content})

        # --- natural exit: the model didn't ask for a tool ---
        if resp.stop_reason != "tool_use":
            return "".join(b.text for b in resp.content if b.type == "text"), None

        # --- dispatch tool calls; check goal predicate; check no-progress ---
        results = []
        for block in resp.content:
            if block.type != "tool_use":
                continue

            if block.name == "submit_final_answer":
                return block.input["answer"], block.input.get("sources", [])

            call_sig = (block.name, json.dumps(block.input, sort_keys=True))
            last_calls.append(call_sig)
            if last_calls[-3:].count(call_sig) >= 3:
                raise RuntimeError(f"no-progress: repeated {block.name} 3x at step {step}")

            try:
                output = TOOL_FNS[block.name](block.input)
                results.append({"type": "tool_result",
                                "tool_use_id": block.id,
                                "content": json.dumps(output)})
            except Exception as e:
                results.append({"type": "tool_result",
                                "tool_use_id": block.id,
                                "content": f"error: {type(e).__name__}: {e}",
                                "is_error": True})

        messages.append({"role": "user", "content": results})

    raise RuntimeError(f"step cap reached: {max_steps}")

the budgets are checked before the API call; checking after the fact is how you accidentally double your spend right at the limit. The no-progress detector is dumb but effective: a hash of the last three (tool_name, args) tuples. If they’re identical, the model is stuck in a loop and no amount of additional steps will help. submit_final_answer is the only “happy path” exit other than the model declining to call a tool; having an explicit terminator tool is much easier to test and reason about than relying on the model to know when to stop emitting tool calls.

The step cap (max_steps=12) is the floor of the abstraction. Without it, a confused model with access to search_web will happily search forever for any sufficiently vague question. The cap is the fork-bomb backstop; the deadline and the no-progress detector are finer-grained nets above it.

What the harness owns

A useful frame: the agent loop has two layers, and the harness; the runtime that drives the model; owns the bottom layer entirely.

Top layer (model decisions): which tool, with what arguments, when to stop emitting tool calls. The model’s job.
Bottom layer (harness duties): budget enforcement, retry policy, observability, error transformation, prompt-cache hygiene, message-list construction, streaming, cancellation. Always the driver’s job.

A prompt cannot enforce a token budget or deterministically detect repeated calls. The harness can count usage, compare call signatures, and stop execution. It also owns context assembly, tool dispatch, streaming, prompt-cache management, error recovery, cost accounting, and telemetry. See Agent Harness Anatomy and Building effective agents.