Function Calling Isn't Magic: Treat Your LLM Like an Untrusted Intern With a Typewriter

Most write-ups treat function calling as a feature you switch on: register a few tools, pass a JSON schema, and the model suddenly "does things. " That framing is where production incidents come from. The model is not calling your code. It is filling out a form and sliding it under your door, and your job is to be the clerk who decides whether that form is valid, safe, and worth acting on. Everything that goes wrong in real agents traces back to skipping the clerk.

This post is the version I wish I had read before wiring tools into anything that touches a database or a payment. It is opinionated, it has a decision table, and it names the specific failures that only show up once real traffic hits — not the happy-path demo everyone reproduces from the docs.

The mental model: the model is an untrusted intern with a typewriter

Here is the frame that makes every design decision fall out cleanly. Imagine a brilliant but overconfident intern who cannot leave the room. They can read the conversation and they can type. They cannot press any buttons themselves. When they want something done, they write a note: "Please run charge_card with amount 4999 and customer cus_123. " They hand you the note. You — the trusted system — decide whether to honor it.

Once you internalize that the intern only ever produces notes, three things stop being mysterious. Why do you validate arguments? Because interns typo and occasionally invent numbers with total confidence. Why do you cap the loop? Because an anxious intern will keep writing notes forever if the last result confused them. Why do you never auto-run destructive actions? Because a note that says "delete the production table" is still just a note, and the whole point of having a clerk is that some notes get refused.

The model never calls your functions. It requests that you call them. That one word — requests — is the entire safety margin, and most brittle agents are brittle because their authors forgot it was there.

A worked example: the refund bot that looked fine in the demo

Let me walk through a concrete scenario, because the abstract advice only lands when you see it break. Say you are building a support agent that can issue refunds. You give it one tool, issue_refund, with a schema that takes an order_id and an amount. In the demo, a tester types "refund my broken headphones," the model calls issue_refund with the right order and the right price, and everyone claps.

Now ship it. Within a week you see three things the demo never surfaced. First, a customer writes "refund everything I ever bought" and the model happily emits a call with the sum of six orders — because nothing told it there is a per-refund ceiling. Second, the model calls issue_refund with an order_id that belongs to a different customer, because the ID appeared earlier in a long conversation and the model pattern-matched it. Third, when the payment API times out and you throw an exception, the whole agent turn crashes instead of the model saying "that didn't go through, let me retry. "

None of these are model "stupidity. " They are all missing clerk logic. The fix is not a better prompt — it is treating the tool call as untrusted input and putting the business rules on your side of the door.

Before: the tool everyone writes first

def issue_refund(order_id, amount):
    # trusts the model completely
    return payments.refund(order_id, amount)

# in the loop:
call = model_response.tool_call
result = issue_refund(call.args['order_id'], call.args['amount'])
# whatever the model asked for, we just did it

After: the clerk does its job

MAX_REFUND = 500_00  # cents; a business rule, not the model's call

def handle_refund(call, session):
    args = call.args
    order = orders.get(args.get('order_id'))

    # 1. validate existence and ownership on OUR side
    if order is None:
        return tool_error('unknown order_id')
    if order.customer_id != session.customer_id:
        return tool_error('order does not belong to this customer')

    # 2. enforce a ceiling the model cannot argue its way past
    amount = int(args.get('amount', 0))
    if amount <= 0 or amount > min(order.total, MAX_REFUND):
        return tool_error('amount out of allowed range')

    # 3. side effects need a human for anything real
    if amount > 50_00:
        return needs_confirmation(order.id, amount)

    try:
        receipt = payments.refund(order.id, amount)
        return tool_result({'status': 'ok', 'receipt': receipt.id})
    except PaymentError as e:
        # hand the failure back as data, not a crash
        return tool_error('payment failed: ' + str(e))

def tool_error(msg):
    return {'role': 'tool', 'content': {'error': msg}}

Notice what changed. The schema barely matters here; the model still just asks. What matters is that ownership checks, the ceiling, the confirmation gate, and the error-as-data handling all live in your code, where the model cannot reason around them. The After version is boring. Boring is the goal — boring code is what survives contact with real users.

When to use tool calling, and when it is the wrong hammer

Tool calling is genuinely transformative, which is exactly why people reach for it in situations where a plain function call would be simpler, cheaper, and more reliable. The question to ask is: does the decision of whether and how to act actually require language understanding? If the branching logic is deterministic, you do not need a model in the loop at all.

• USE it when the user's intent is fuzzy and must be mapped to one of many possible actions — "find me a flight next week that isn't a red-eye" into a structured search call.
• USE it when a task needs several dependent steps whose order depends on intermediate results the model has to read and react to.
• USE it to give the model a controlled escape hatch to real data (a database, a calendar) so it stops hallucinating facts it could have looked up.
• AVOID it when the trigger is deterministic — if a button click should always run the same function, wire the button to the function.
• AVOID it for a single, fixed transformation where structured output alone is enough; you want a shaped answer, not an action.
• AVOID it as a replacement for validation — a tool call that writes to your DB with no checks is a SQL injection with extra latency.

The trade-off table I actually keep

This is the decision matrix I mentally run before adding a tool. Treat the right column as the thing that bites you at 2 a. m. , not a footnote.

Design choice        | Cheap win                  | What breaks in production
---------------------+----------------------------+--------------------------------
Many granular tools  | model picks precisely      | it picks the wrong one; token
                     |                            | bloat; slow first call
Few fat tools        | small prompt, fast         | model crams intent into one
                     |                            | arg; ambiguous routing
Auto-run everything  | great demo                 | one bad call is irreversible
Confirm side effects | safe, auditable            | more turns, higher latency
Errors as exceptions | simple code                | agent crashes mid-plan
Errors as tool results| model self-corrects        | can loop retrying forever
No step cap          | 'it just works'            | runaway cost, infinite loop
Step cap + budget    | predictable spend          | may stop before finishing

There is no universally right column. The craft is choosing the failure you can live with for a given tool. A read-only search tool can auto-run all day; a money-moving tool earns every confirmation prompt it costs you.

Non-obvious gotchas that only reasoning (or scars) will teach you

These are the ones that do not show up in the quickstart and are not about "write good descriptions," which you already know.

1. Tool descriptions compete with each other. Two similarly worded tools make the model flip a coin. When routing gets flaky, the fix is usually to make descriptions contrast — say explicitly when NOT to use each — rather than adding words to one.
2. The model can call a tool with perfectly valid arguments that are semantically wrong. Schema validation passes; the order_id is a real ID for the wrong customer. Business validation is a separate layer, and no JSON schema will save you from it.
3. Tool results are attacker-controlled input. If a tool fetches a web page or a support ticket, its text can contain instructions. Treat every tool result as untrusted; never let a result silently escalate what the next tool is allowed to do.
4. Parallel tool calls break assumptions about order. If the model requests three calls at once, do not assume they run in sequence or that later ones saw earlier results. Design each call to be independent, or force sequential execution.
5. A well-meaning retry loop plus 'errors as tool results' is an infinite bill. The model gets an error, tries again, gets the same error, tries again. Your step cap is not optional; it is the only thing standing between a bad deploy and a surprise invoice.
6. Silent argument coercion hides bugs. If you cast the model's string '4999' to an int and it sends '4,999' or 'forty-nine ninety-nine', a naive cast throws or, worse, quietly produces garbage. Validate the shape before you coerce, and return a clear tool error when it is off.

The bottom line

Function calling is a small protocol wrapped around a big responsibility. The protocol — describe tools, let the model ask, run it yourself, feed the result back — you can learn in an afternoon. The responsibility is deciding, for every single tool, what you will let an overconfident intern trigger with a note, and what business rules live on your side of the door where the model cannot talk its way past them.

So the checklist is short but non-negotiable: validate existence and ownership, enforce ceilings and permissions in your code, gate real side effects behind confirmation, return failures as data so the model can recover, and cap the loop so a confused model cannot spin up a bill. Get that division of labor right and tool use is not just reliable — it is auditable, which is the property that actually lets you sleep. Blur it, and you have shipped a demo that works right up until the moment it does something you cannot undo.