How Autonomous Should an AI Agent Be? Autonomy Levels From 0 to Full

The MCP post covered how an agent reaches tools that live outside your project. Once those tools are connected, a different question takes over: how much of the job should the agent do before it comes back to you?

Take the expense tracker from earlier in this series and add one tool, delete_expense. A user types, “clean up my old entries.” The model reads that as permission and starts calling the tool. Nothing malfunctioned. The message allowed that reading and the tool allowed the action. What was missing was a rule about when the tool may run.

How Autonomous Should an AI Agent Be?

An agent's autonomy is how much it may do on its own before a human has to approve, review or step in. It is a policy your code enforces, not a trait of the model. A sensible default is read-only tools first, human approval for writes, and one tool at a time promoted to run alone after you have watched it behave.

A Scale From Level 0 to Level 4

No standard defines these levels. This is the scale I use, and each step hands one more decision from the human to the agent.

• Level 0, no tools. The model only writes text and a human does everything. A plain chatbot. It needs no code: you just don't pass a tools array.
• Level 1, read-only tools. The agent can look things up, like an order status or a running total, but cannot change anything. The worst result is a wrong answer.
• Level 2, writes with approval. The agent proposes each change and a human accepts or declines it.
• Level 3, bounded autonomy. Routine writes run on their own inside limits you set, such as a step cap and a log of every action. Irreversible actions still ask.
• Level 4, full autonomy. Everything runs without a human in the loop. Few jobs justify this, and the ones that do need monitoring far beyond a prompt.

A level doesn't belong to a whole agent. The same agent can run reads at Level 4 and deletes at Level 2, so the next step is to classify tools, not agents.

Sort Tools by What a Mistake Costs

Three classes cover most agents. A read tool changes nothing. A write tool changes something you can undo. An irreversible tool can't be taken back: deleting data, sending a message, moving money. The class comes from the cost of a wrong call, not from the tool's name. update_record sounds harmless until it overwrites the only copy.

After that, the decision is a function of two inputs: the tool's class and the current level.

flowchart TD
    A[Model requests a tool] --> B{Tool risk class}
    B -->|read| C[Run it]
    B -->|write| D{Autonomy level}
    B -->|irreversible| E{Autonomy level}
    D -->|1| F[Refuse]
    D -->|2| G[Ask a human]
    D -->|3 or 4| C
    E -->|1| F
    E -->|2 or 3| G
    E -->|4| C
    G -->|approved| C
    G -->|declined| I[Send an error result to the model]
    F --> I
    C --> H[Send the result to the model]

Flow summary: Every tool request is checked against the tool's risk class and the current level. Reads always run. Writes run or wait for approval depending on the level, and irreversible actions wait for a human until Level 4. A refusal or a decline goes back to the model as an error result so it can respond sensibly.

The Policy in Code

Install the SDK with npm install @anthropic-ai/sdk, set ANTHROPIC_API_KEY, save this as agent/autonomy.ts and run AUTONOMY=2 npx tsx agent/autonomy.ts. Change the value to 1, 3 or 4 and send the same message to compare. It uses Node's built-in readline/promises, so you need a reasonably recent Node version.

// agent/autonomy.ts
import Anthropic from '@anthropic-ai/sdk';
import { createInterface } from 'node:readline/promises';

const client = new Anthropic();
const rl = createInterface({ input: process.stdin, output: process.stdout });

type Risk = 'read' | 'write' | 'irreversible';
type Decision = 'allow' | 'ask' | 'deny';

const LEVEL = Number(process.env.AUTONOMY ?? 2);

const expenses = [{ id: 1, item: 'tea', amount: 40 }];
let nextId = 2;

const risks: Record<string, Risk> = {
  get_total: 'read',
  add_expense: 'write',
  delete_expense: 'irreversible',
};

function decide(level: number, risk: Risk): Decision {
  if (risk === 'read') return 'allow';
  if (level <= 1) return 'deny';
  if (level >= 4) return 'allow';
  if (risk === 'write' && level === 3) return 'allow';
  return 'ask';
}

const tools: Anthropic.Tool[] = [
  {
    name: 'get_total',
    description: 'Return the exact sum of every expense recorded so far. Use this whenever the user asks for a total.',
    input_schema: { type: 'object', properties: {} },
  },
  {
    name: 'add_expense',
    description: 'Record one expense. Use it whenever the user says they spent money.',
    input_schema: {
      type: 'object',
      properties: { item: { type: 'string' }, amount: { type: 'number' } },
      required: ['item', 'amount'],
    },
  },
  {
    name: 'delete_expense',
    description: 'Permanently delete one expense by its id. Use it only when the user names the expense to remove.',
    input_schema: {
      type: 'object',
      properties: { id: { type: 'number' } },
      required: ['id'],
    },
  },
];

function execute(name: string, input: Record<string, unknown>): unknown {
  if (name === 'get_total') return { total: expenses.reduce((sum, e) => sum + e.amount, 0) };
  if (name === 'add_expense') {
    const expense = { id: nextId++, item: String(input.item), amount: Number(input.amount) };
    expenses.push(expense);
    return expense;
  }
  if (name === 'delete_expense') {
    const index = expenses.findIndex((e) => e.id === Number(input.id));
    if (index === -1) return { error: 'no such expense' };
    expenses.splice(index, 1);
    return { ok: true };
  }
  return { error: 'unknown tool' };
}

async function approve(name: string, input: Record<string, unknown>): Promise<boolean> {
  const answer = await rl.question(`Allow ${name} ${JSON.stringify(input)}? (y/n) `);
  return answer.trim().toLowerCase() === 'y';
}

async function guarded(name: string, input: Record<string, unknown>): Promise<unknown> {
  const decision = decide(LEVEL, risks[name] ?? 'irreversible');
  console.log(`[${decision}] ${name}`);
  if (decision === 'deny') return { error: 'not permitted at this autonomy level' };
  if (decision === 'ask' && !(await approve(name, input))) return { error: 'the user declined this action' };
  return execute(name, input);
}

async function runAgent(userText: string): Promise<string> {
  const messages: Anthropic.MessageParam[] = [{ role: 'user', content: userText }];

  for (let step = 0; step < 6; step++) {
    const response = await client.messages.create({
      model: 'claude-sonnet-5',
      max_tokens: 500,
      tools,
      messages,
    });

    messages.push({ role: 'assistant', content: response.content });

    if (response.stop_reason !== 'tool_use') {
      return response.content.flatMap((b) => (b.type === 'text' ? [b.text] : [])).join('');
    }

    const results: Anthropic.ToolResultBlockParam[] = [];
    for (const block of response.content) {
      if (block.type === 'tool_use') {
        const output = await guarded(block.name, block.input as Record<string, unknown>);
        results.push({ type: 'tool_result', tool_use_id: block.id, content: JSON.stringify(output) });
      }
    }
    messages.push({ role: 'user', content: results });
  }

  return 'Stopped: step limit reached';
}

runAgent('Add 300 for groceries, then delete expense 1.')
  .then(console.log)
  .finally(() => rl.close());

decide is the whole policy. A tool missing from risks falls back to irreversible, so anything you forget to classify gets the strictest treatment. A denial or a declined approval goes back as an error result rather than an exception, the same rule from the tool-calling post, so the model can tell the user what happened. guarded prints one line per decision, which is the smallest audit trail worth having. With the sample message, Level 2 should prompt you twice, once per write, and Level 3 should run the add on its own and prompt only for the delete.

What I Noticed Rereading My Own Series Code

Planning this post, I went back to the system prompt template from the prompt-engineering post. Its Boundaries section has the line “Never delete or change a past expense unless the user explicitly asks to.” The agent in that post has add_expense, get_total and save_note, and no delete tool at all. The line guards an action the agent can't take, and if a delete tool were ever added, that one sentence would be the only thing between the model and your data.

The tracker from the components post shows the same gap from the other side. add_expense and save_note write with no approval, and the six-step cap is the only limit. That was a Level 3 decision I never named. It's acceptable for an in-memory array and a notes file. It stops being acceptable the day a destructive tool joins the list, which is why the gate here sits in code next to execute and the prompt line stays as a second layer.

Common Mistakes With Autonomy

• Keeping the rule only in the prompt. A sentence can be misread. A function that never reaches delete without a yes can't be talked past.
• Classifying by name. Ask what a wrong call costs, then pick the class.
• No default for unclassified tools. Fail closed, as risks[name] ?? 'irreversible' does.
• Raising the level for the whole agent. Promote one tool at a time. A well-behaved add_expense says nothing about delete_expense.
• Running approvals in parallel. Reads can run together. Anything that waits on a person should run one at a time, which is why the loop uses a plain for.
• Treating a decline as a crash. Return it as a tool result so the model can offer an alternative.

What's Next

The level tells you how much supervision an agent needs. With tools, memory, prompts and autonomy covered, the next step is turning all of it into a first hands-on project, and that's worth planning properly.

Frequently Asked Questions

Is a more autonomous agent always a better one?

No. Autonomy removes human review, and review is worth most when an action is costly or hard to undo. Aim for the lowest level that still makes the agent useful, and set it per tool, not per agent.

Where should I start with a new agent?

At Level 1 with read-only tools. Move to Level 2 and approve every write by hand, then promote a single tool to run unattended once you have watched it behave correctly across a range of real requests.

Can the model choose its own autonomy level?

It shouldn't. The model is the thing being supervised, so the policy has to live in your code, where the model can neither edit it nor argue with it. A prompt line is a useful second layer, not the first.