keel
Harness

Failure and recovery

Make rejected operations visible and give the model bounded opportunities to correct them.

lookup({ id: 42 })

expected: string
→ tool-input-invalid
feedback → model

lookup({ id: "42" })
→ validate again
Another model step requires remaining budget
Feedback gives the model another opportunity. It does not guarantee a successful correction.

A failed operation should have an observable cause. When model-generated arguments fail a tool's schema, Keel refuses execution and returns the reason to the model. The next call can correct the input, but must pass the same checks as the original attempt.

Put the contract at the boundary

import { defineTool } from "@keel-dev/core";
import { z } from "zod";

const lookupInput = z.object({ id: z.string() });

const lookup = defineTool({
  id: "lookup",
  description: "Look up a record by ID",
  access: "public",
  input: lookupInput,
  retries: 1,
  onError: "feedback",
  execute: async (raw) => {
    const { id } = lookupInput.parse(raw);
    return { result: await lookupRecord(id) };
  },
});

lookupRecord is your application function. If the model supplies { id: 42 }, input parsing rejects the call before that function runs. The turn records tool-input-invalid, puts the rejection in the tool result, and can continue with another model step.

Distinguish the recovery paths

FailureHarness behavior
Unknown tool or denied accessRefuse execution and return feedback.
Invalid tool argumentsReturn schema feedback; a new model call can change the arguments.
Tool execution throwsRetry according to retries, then apply onError.
Refused state byproductAppend the refusal to the tool's result; escalate under onError: "fail".
Provider attempt failsLet the model chain apply its retry and fallback policy.
Required deliverable is missingRun bounded submission nudges, then fail if still missing.
Request no longer fitsFail with fit-overflow before the next provider call.

An execution retry repeats the tool with parsed input. A model correction asks the model to produce another call and can change the input. These consume different grants; see Budgets and cancellation.

Two failures precede the loop entirely and settle the turn directly: an ambient scope the app's schema rejects fails at the admit gate with ambient-invalid, and a throwing block reader, repair, or ambient section fails assembly with render-failure — a typed defect, never a raw exception.

Decide when failure ends the turn

After execution retries, onError: "feedback" returns the cause to the model so it can adapt or report the failure. onError: "fail" settles the turn as failed. Delegation has the same policy choice and returns a structured failure to its caller.

Feedback does not guarantee repair. The model might repeat its mistake or stop. Limits bound the attempt; schemas decide whether the next operation is valid. Chain exhaustion is a failure with chain-exhausted as its cause — never a fabricated successful answer, and never rebranded when it happens inside a continuation or nudge round: the recovery round's real cause survives to the outcome. See Retry and fallback.

Keep state repair separate

A block's repair function is application code that normalizes stored data on read. It does not invoke an LLM. A refused byproduct, by contrast, is never silent: when a tool's commit names an undeclared block or an unknown reducer, or the block's writer is already taken this turn, the refusal is appended to the tool's own result — STATE COMMIT REFUSED (block): reason — so the model reacts to it in the same loop. Under onError: "fail" the refused commit fails the whole turn instead.

The refusal also lands on the bus as a defect, so the correction loop and the trace agree. See Repairing state.

Inspect in Studio

Use a fake tier that issues one invalid call followed by a valid call. Inspect the refused operation and schema defect, then the corrected call and its result. Confirm whether the final turn recovered or failed; the presence of a defect alone does not answer that question.

Next: Completion and deliverables.