Building on the Claude Messages API: max_tokens, Streaming, Tool Use Loops, Caching and Retries

One endpoint, a few rules

The Claude API is small on the surface. Text generation, tool calling, image input, structured output and caching all go through a single endpoint, POST /v1/messages, and the official SDKs (anthropic for Python, @anthropic-ai/sdk for TypeScript, plus Java, Go, Ruby, C# and PHP) wrap it. The API is stateless: every request carries the full conversation.

Most integration bugs I have seen do not come from exotic features. They come from a few rules that are easy to get wrong the first time: max_tokens is mandatory, a response is a list of typed blocks rather than a string, tool use is a loop your code drives, and the SDK already retries transient failures. This article walks through those rules with Python examples, and notes the TypeScript equivalent where it differs.

Examples use claude-opus-5. Other current model IDs include claude-opus-5-5 (the newest Opus), claude-sonnet-5 and claude-haiku-4-5. Use model IDs exactly as documented; a typo such as claude-sonnet-4.6 instead of claude-sonnet-4-6 returns a 404 not_found_error, the same response you get for a model your organization cannot access.

Authentication and the client

pip install anthropic          # Python
npm install @anthropic-ai/sdk  # TypeScript / JavaScript

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

Keep the key out of source code and pass it through the environment or a secret manager. One trap: the SDK takes the first credential it finds, and ANTHROPIC_API_KEY comes before ANTHROPIC_AUTH_TOKEN and any CLI login profile. A stale key left in your shell profile therefore wins over whatever you configured later, and the resulting 401 looks like a problem with the new credentials when it is really the old variable.

The Messages API request and response

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    system="You are a code reviewer. Point out bugs before style issues.",
    messages=[
        {"role": "user", "content": "Review this function:\n\ndef avg(xs): return sum(xs) / len(xs)"}
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)

print(response.stop_reason)   # e.g. "end_turn"
print(response.usage)         # input_tokens, output_tokens, cache fields

Three required fields: model, max_tokens and messages. The details that matter:

• max_tokens has no default. Leave it out and the API returns a 400. It is a hard ceiling on generated tokens, not a target, and the model is not aware of it: when output reaches the limit it simply stops, often mid-sentence, with stop_reason: "max_tokens". Setting it very low to “save money” mostly produces truncated answers you then have to retry. Set it generously and control length through the prompt.
• messages starts with a user turn. A conversation beginning with an assistant message is a 400.
• The system prompt is a top-level system field, not a {"role": "system"} entry at the start of messages. Some newer models accept system messages later in the conversation for operator instructions, but never as the first message.
• content in the response is a list of blocks. Each block has a type: text, tool_use, thinking, and others depending on the features you use.

That last point is the one that bites. A lot of older sample code does response.content[0].text. On models where thinking is on by default (Claude Opus 5 runs adaptive thinking unless you turn it off, and on Claude Opus 5.5 it cannot be turned off), the first block may be a thinking block with no .text, and with tools it may be a tool_use block. I have had exactly this break a script that worked for months: nothing in my code changed except the model ID, and suddenly it failed with an AttributeError on a thinking block. Iterating over the blocks and filtering by type avoids the whole class of bug. In TypeScript the compiler enforces this, since content[0].text does not type-check without narrowing on type.

Two other things that break when copying older code to current models: temperature, top_p and top_k are rejected with a 400 on the newest models, and so is prefilling the start of the assistant’s answer as the last message. Use the system prompt or structured outputs (output_config.format, or client.messages.parse() in the SDKs) to control format instead.

Thinking and effort

Current models decide for themselves how much to reason with adaptive thinking, and you steer depth and token spend with effort:

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    thinking={"type": "adaptive"},
    output_config={"effort": "medium"},  # low | medium | high | xhigh | max
    messages=[{"role": "user", "content": "Find the race condition in this code: ..."}],
)

The older thinking={"type": "enabled", "budget_tokens": N} form is rejected on the newest models. Lower effort for simple, high-volume tasks such as classification; keep it high for coding and multi-step agent work. By default the thinking text itself is omitted from the response (the block is present but its text is empty). If you want to show a reasoning summary, request "display": "summarized" in the thinking object.

Multi-turn conversations

Because the API is stateless, a chat is just a growing messages list that you resend each time:

history: list[dict] = []

def chat(user_text: str) -> str:
    history.append({"role": "user", "content": user_text})
    response = client.messages.create(
        model="claude-opus-5",
        max_tokens=16000,
        system="You are a patient programming tutor.",
        messages=history,
    )
    # Append the full content list, not just the text
    history.append({"role": "assistant", "content": response.content})
    return "".join(b.text for b in response.content if b.type == "text")

Appending response.content rather than a flattened string matters once you use tools, thinking or server-side compaction, because the next request needs those blocks intact. Every turn resends the entire history, so input tokens grow with the length of the conversation. That is the main reason prompt caching exists.

Streaming

For chat UIs, streaming turns a long silent wait into text that appears as it is generated. It is also a practical requirement for long outputs: a non-streaming request with a very large max_tokens risks hitting HTTP timeouts, and the SDKs steer you toward streaming for those.

with client.messages.stream(
    model="claude-opus-5",
    max_tokens=64000,
    messages=[{"role": "user", "content": "Explain Python's GIL with examples."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

    final = stream.get_final_message()   # the complete Message object

print("\nstop_reason:", final.stop_reason)
print("output tokens:", final.usage.output_tokens)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const stream = client.messages.stream({
  model: "claude-opus-5",
  max_tokens: 64000,
  messages: [{ role: "user", content: "Explain JavaScript's event loop." }],
});

stream.on("text", (text) => process.stdout.write(text));
const final = await stream.finalMessage();
console.log("\nstop_reason:", final.stop_reason);

Two things to plan for. First, the HTTP status is sent before generation finishes, so a stream that starts with 200 can still end in an error event (an overload, for example). Handle errors around the whole iteration, not just the initial call, and decide what your UI shows for a partial answer. Second, check stop_reason on the final message; a stream that ends with max_tokens looks complete in the UI unless you surface it. If you only need the final result and not the individual events, use get_final_message() / finalMessage() rather than accumulating deltas yourself.

Tool use is a loop you drive

With tool use, Claude does not execute anything. It returns a tool_use block naming a tool and its JSON input, your code runs the tool, and you send the output back as a tool_result. This repeats until Claude answers without requesting a tool.

import json

tools = [
    {
        "name": "get_order_status",
        "description": "Look up the shipping status of an order by its ID. "
                       "Use this whenever the user asks where an order is.",
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string", "description": "Order ID such as ORD-1234"}
            },
            "required": ["order_id"],
        },
    }
]

def run_tool(name: str, args: dict) -> str:
    if name == "get_order_status":
        return json.dumps({"order_id": args["order_id"], "status": "shipped"})
    raise ValueError(f"Unknown tool: {name}")

def answer(question: str, max_iterations: int = 10) -> str:
    messages = [{"role": "user", "content": question}]

    for _ in range(max_iterations):
        response = client.messages.create(
            model="claude-opus-5",
            max_tokens=16000,
            tools=tools,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason in ("max_tokens", "refusal"):
            # A truncated or declined turn: do not run its tool calls
            raise RuntimeError(f"Stopped early: {response.stop_reason}")

        if response.stop_reason == "pause_turn":
            continue  # server-side tool paused; resend to let it continue

        if response.stop_reason != "tool_use":
            return "".join(b.text for b in response.content if b.type == "text")

        results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            try:
                output = run_tool(block.name, block.input)
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": output})
            except Exception as exc:
                results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": f"Error: {exc}",
                    "is_error": True,
                })

        # All results for this turn go back in ONE user message
        messages.append({"role": "user", "content": results})

    raise RuntimeError("Tool loop did not finish")

The rules this code follows are the ones that produce errors when broken:

• Every tool_use needs a matching tool_result in the very next user message, linked by tool_use_id. Drop one (for example, because the tool threw and you skipped it) and the next request is rejected with a 400 saying tool_use ids were found without tool_result blocks immediately after. Report failures as a tool_result with is_error: true instead; Claude will usually try another approach or explain the problem.
• Claude can request several tools in one turn. Run them (concurrently if you like) and return all results together in a single user message. Splitting them across separate messages works against parallel tool calls.
• Check max_tokens and refusal before running tools. A turn cut off at the token ceiling can contain a tool_use whose input is incomplete. Executing it is how you end up calling an API with half an argument.
• Bound the loop. A model that keeps calling a failing tool will happily do so until you stop it. A maximum iteration count is cheap insurance.
• Parse block.input as data, never by string-matching serialized JSON; escaping details can differ between models.

Tool descriptions are prompts. “Get order” is a poor description; saying what the tool returns and when to use it changes how reliably Claude picks it. Setting "strict": True on a tool definition (with additionalProperties: false and a required list in the schema) guarantees the input matches the schema.

If you do not need custom control over each iteration, the SDKs include a tool runner (client.beta.messages.tool_runner with the @beta_tool decorator in Python, toolRunner with betaZodTool in TypeScript) that implements this loop for you. It is a beta helper. The manual loop above is still worth understanding, because it is exactly what the runner does and what you debug when something goes wrong.

Prompt caching

Long system prompts, tool definitions and reference documents are resent on every request. Prompt caching lets the API reuse the processed prefix instead of reprocessing it. Cache reads are billed at roughly a tenth of the normal input rate; writing the cache costs somewhat more than a normal request (about 1.25x for the default 5-minute lifetime, 2x for the 1-hour option).

REFERENCE_DOC = open("api_reference.md").read()

def ask(question: str):
    response = client.messages.create(
        model="claude-opus-5",
        max_tokens=16000,
        system=[
            {"type": "text", "text": "Answer questions using only the reference below."},
            {"type": "text", "text": REFERENCE_DOC, "cache_control": {"type": "ephemeral"}},
        ],
        messages=[{"role": "user", "content": question}],
    )
    u = response.usage
    print("written:", u.cache_creation_input_tokens, "read:", u.cache_read_input_tokens,
          "uncached:", u.input_tokens)
    return response

The mental model that makes caching work: it is an exact byte-for-byte prefix match over the request rendered in the order tools, then system, then messages. A cache_control breakpoint caches everything up to that point. Any change before the breakpoint invalidates it.

The first time I added caching to a service, cache_read_input_tokens stayed at zero on every request. The system prompt started with the current date and time, so each request had a unique prefix and wrote a brand-new cache entry that was never read. Moving the timestamp into the user message fixed it immediately. The other usual suspects are tool definitions built from a dictionary with unstable ordering, JSON serialized without sorted keys, and a tool list that varies per request.

Other details:

• A prefix shorter than the model’s minimum cacheable length is silently not cached. There is no error, just cache_creation_input_tokens: 0, and the minimum differs between models.
• A request can have at most four breakpoints. Put them at stability boundaries: after tools and system, and at the end of the shared part of the conversation.
• For the simplest case, a top-level cache_control={"type": "ephemeral"} on messages.create() places the breakpoint on the last cacheable block automatically.
• Always verify with the usage fields. Caching that is not hitting costs slightly more than no caching at all.

Errors, 429s, 529s and retries

Status	Type	Retry?	Typical cause
400	invalid_request_error	No	Missing max_tokens, bad message order, unsupported parameter
401	authentication_error	No	Missing, wrong or conflicting credentials
404	not_found_error	No	Wrong model ID or model not available to you
413	request_too_large	No	Request body too big
429	rate_limit_error	Yes	Request or token rate limit exceeded
500	api_error	Yes	Transient server issue
529	overloaded_error	Yes	API temporarily overloaded

A 429 is about your usage relative to your limits; the response includes a retry-after header and rate-limit headers showing what remains. A 529 is about capacity on Anthropic’s side and can happen even when you are well within your limits. Both are transient, and the SDK already treats them that way: by default it retries 408, 409, 429 and 5xx responses, plus connection errors, twice with exponential backoff.

That default is where I have seen retry logic go wrong most often, including in my own code. A hand-written loop that retries three times, wrapped around an SDK call that itself retries twice, makes up to nine attempts per logical request. Under a real rate limit this amplifies the load that caused the 429 in the first place. Timeouts are retried too, so wall-clock time for a failing call can reach roughly timeout × (max_retries + 1). Configure the SDK instead of stacking loops:

client = anthropic.Anthropic(max_retries=4, timeout=120.0)  # seconds in Python

try:
    response = client.messages.create(
        model="claude-opus-5",
        max_tokens=16000,
        messages=[{"role": "user", "content": "Summarize this incident report: ..."}],
    )
except anthropic.BadRequestError as e:        # 400: fix the request, retrying won't help
    log.error("Invalid request: %s", e.message)
    raise
except anthropic.RateLimitError:              # 429 after the SDK's own retries
    enqueue_for_later()
except anthropic.InternalServerError as e:    # 5xx, including 529 overloaded
    log.warning("Service unavailable (%s)", e.status_code)
    raise
except anthropic.APIConnectionError:          # network failure, no response
    raise

Note that the TypeScript SDK takes timeout in milliseconds, not seconds. Catch typed exceptions from most specific to least specific rather than inspecting error message strings. When a 429 persists after retries, the fix is usually architectural: queue work, smooth bursts, send non-urgent jobs through the Message Batches API, or use caching to reduce input tokens.

stop_reason values

Every response tells you why generation stopped. Treat it as part of the result, not metadata.

stop_reason	Meaning	What to do
end_turn	Finished naturally	Use the response
max_tokens	Hit your max_tokens ceiling	Output is truncated; raise the limit or ask for less
stop_sequence	Hit one of your stop_sequences	stop_sequence says which one
tool_use	Wants a tool run	Execute tools, send tool_result blocks back
pause_turn	A long server-side tool turn paused	Append the response and send it back to continue
refusal	Declined for safety reasons	Do not treat as an answer; stop_details has the category

Refusals arrive as normal HTTP 200 responses, so code that only checks for exceptions will happily display an empty or partial answer. Check stop_reason before reading content.

The official Claude API documentation has the full parameter reference and the current model list.

Related Articles

• Model Context Protocol (MCP): exposing tools to Claude through a standard protocol
• Prompt engineering for developers: writing system prompts and tool descriptions that behave predictably
• LangChain: when a framework over the raw API helps, and when it gets in the way
• OpenAI ChatGPT API Basics: First Request, Streaming, Function Calling and Prompt Patterns: comparing request shapes and tool calling with OpenAI’s API