What's Inside an AI Agent? Brain, Memory, Tools and the Action Loop

Open the minimal-agent.ts file from the last post on LLMs versus agents and try to point at the brain, the memory, the tools and the loop. The tools array and the while loop announce themselves. The memory is harder to see, because it is just an array called messages.

That is fine for a demo. It stops being fine the first time your agent forgets what you told it a minute ago, picks the wrong function, or keeps looping when it should have stopped. To debug any of those, you first have to know which part is misbehaving.

What Are the Components of an AI Agent?

An AI agent has four components: a brain (the language model that decides what to do next), memory (the conversation history and stored facts the model can read), tools (functions that let it act outside the model) and an action loop (the code that calls the model, runs the tools it asks for and feeds the results back). The brain is the only part that is a model. The other three are code and data that you write and control.

The Four Parts at a Glance

flowchart LR
    U[User message] --> M[(Memory)]
    M -->|reads| B[Brain: the model]
    B -->|asks for a tool| T[Tools]
    T -->|result is stored| M
    B -->|no tool needed| A[Final answer]

Flow summary: The user's message goes into memory and the brain reads it, then either asks for a tool or gives the final answer. When it asks for a tool, your code runs it and stores the result in memory, and that cycle is the action loop.

The Four Parts, One by One

The Brain: Where Decisions Happen

The brain is the language model. On each turn it receives the system prompt, the conversation so far, the list of available tools and any tool results. It produces one of two things: text for the user, or a request to call a tool. It never runs anything itself. It only asks.

The model is stateless, so it knows only what is inside the current request, and its instructions live in the system prompt, which works like a job description. In the code below, the brain is a single client.messages.create call. The last post used OpenAI's SDK and this one uses Anthropic's. The wire formats for tools and messages differ, but the four parts sit in the same places.

Memory: What the Agent Can See, and What It Keeps

Memory comes in two layers. Short-term memory is the message history you send with every call: user messages, model replies and tool results. Long-term memory is anything written to storage and read back in a later run, such as a notes file, a database row or a vector store. Last time I said that replaying chat history alone isn't real memory. That holds if it is all you have, but it is one half of the picture.

The model remembers nothing between calls, so both layers are your code's job. In the example below, the messages array is short-term memory and notes.json is long-term memory. The saved notes are loaded into the system prompt on every call, which is how a preference from one session shows up in the next.

Tools: How an Agent Touches the Real World

A tool is an ordinary function plus a description. The model sees three things: a name, a plain-language explanation of when to use it, and a JSON Schema for its inputs. It never sees your code. It decides from the description alone, so the description does the job of API documentation written for a reader who can't ask questions.

When the model wants a tool, it returns a structured request with the tool name and arguments. Your code runs the function and sends the output back. This is also where you set limits, because an agent can only do what its tools allow. A tool that reads is safer than one that writes, and one that asks for confirmation is safer than one that just acts.

The Action Loop: What Holds It Together

The loop is the code that drives the other three parts. Each pass sends memory to the brain and looks at what comes back. Plain text means the job is done. A tool request means the loop runs the tool, appends the result to memory and goes around again.

A loop needs exits: a final answer, a step cap, a tool failure you can't recover from, or a human who has to approve something. The while (true) in my last example had only the first. The version below adds a cap of six steps, a single line that stops a confused model from running up your bill.

All Four Parts in One Small File

This is last post's agent rebuilt as an expense tracker, so there is something to remember and something to do. Save it as agent/four-parts.ts, install the SDK with npm install @anthropic-ai/sdk, set ANTHROPIC_API_KEY and run npx tsx agent/four-parts.ts.

// agent/four-parts.ts
import Anthropic from '@anthropic-ai/sdk';
import { existsSync, readFileSync, writeFileSync } from 'node:fs';

const client = new Anthropic();
const NOTES_FILE = 'notes.json';

const messages: Anthropic.MessageParam[] = [];
const expenses: { item: string; amount: number }[] = [];

function loadNotes(): string[] {
  return existsSync(NOTES_FILE) ? JSON.parse(readFileSync(NOTES_FILE, 'utf8')) : [];
}

const tools: Anthropic.Tool[] = [
  {
    name: 'add_expense',
    description: 'Record one expense. Use it whenever the user says they spent money.',
    input_schema: {
      type: 'object',
      properties: {
        item: { type: 'string' },
        amount: { type: 'number', description: 'Amount in BDT' },
      },
      required: ['item', 'amount'],
    },
  },
  {
    name: 'get_total',
    description: 'Return the exact total of all recorded expenses in BDT.',
    input_schema: { type: 'object', properties: {} },
  },
  {
    name: 'save_note',
    description: 'Save a lasting fact about the user, such as a preference, for future sessions.',
    input_schema: {
      type: 'object',
      properties: { note: { type: 'string' } },
      required: ['note'],
    },
  },
];

const handlers: Record<string, (input: Record<string, unknown>) => unknown> = {
  add_expense: (input) => {
    expenses.push({ item: String(input.item), amount: Number(input.amount) });
    return { ok: true, recorded: expenses.length };
  },
  get_total: () => ({ total: expenses.reduce((sum, e) => sum + e.amount, 0) }),
  save_note: (input) => {
    writeFileSync(NOTES_FILE, JSON.stringify([...loadNotes(), String(input.note)]));
    return { ok: true };
  },
};

function execute(name: string, input: Record<string, unknown>): unknown {
  try {
    const handler = handlers[name];
    return handler ? handler(input) : { error: 'unknown tool' };
  } catch (err) {
    return { error: String(err) };
  }
}

async function runAgent(userText: string): Promise<string> {
  messages.push({ role: 'user', content: userText });

  for (let step = 0; step < 6; step++) {
    const response = await client.messages.create({
      model: 'claude-sonnet-5',
      max_tokens: 1024,
      system: `You track expenses. Known about the user: ${loadNotes().join('; ') || 'nothing yet'}`,
      tools,
      messages,
    });

    messages.push({ role: 'assistant', content: response.content });

    if (response.stop_reason !== 'tool_use') {
      return response.content.flatMap((b) => (b.type === 'text' ? [b.text] : [])).join('');
    }

    const results: Anthropic.ToolResultBlockParam[] = [];
    for (const block of response.content) {
      if (block.type === 'tool_use') {
        const output = execute(block.name, block.input as Record<string, unknown>);
        results.push({ type: 'tool_result', tool_use_id: block.id, content: JSON.stringify(output) });
      }
    }
    messages.push({ role: 'user', content: results });
  }

  return 'Stopped: step limit reached';
}

async function main() {
  console.log(await runAgent('I spent 250 on lunch and 80 on a rickshaw. I prefer short answers.'));
  console.log(await runAgent('What is my total so far?'));
}

main();

The brain is the client.messages.create call. Memory is messages plus notes.json. The tools are tools, handlers and execute. The action loop is the for block inside runAgent.

Follow one request through it:

1. Your message goes into messages.
2. The brain reads the notes, history and tool list, then asks for add_expense once per item.
3. The loop runs each request through execute and stores the results.
4. The brain reads the results, sees nothing else is needed and returns text.
5. If it also called save_note, the preference is now in notes.json for the next run.

The Mistake That Cost Me an Afternoon

The first version of this expense tracker forgot everything between messages. I had written const messages = [] inside the function that handled each user message, so every call started with a blank history. I told it I spent 250 on lunch, it logged the expense, and a minute later I asked for the total and it asked which expenses I meant. I blamed the system prompt and kept rewriting it until I noticed the array was being recreated on every call. Moving one line out of the function fixed it. The brain was fine. The memory had simply never been wired up.

Common Mistakes When Wiring the Four Parts

• Vague tool descriptions. The brain picks tools by reading their descriptions, so something like “handles expense stuff” forces a guess. Say what the tool does, what it needs and when to use it.
• Sending the whole history forever. Every step re-sends everything, so cost and latency climb and old details can crowd out the current task. Trim or summarize old turns and keep lasting facts in a separate store.
• Swallowing tool errors. If a tool fails and your code returns nothing, the brain can't correct course. Return the error text as the result, as execute does, and let the model try again.
• Making the brain do work that code can do. Adding up a list of amounts inside the prompt invites arithmetic slips. That is why get_total is a tool. Use code for exact work and the model for judgment.

What Comes Next

Each part deserves more than a section, and I'll take tool calling and memory apart properly in later posts. Until then, find these four things in the code of any agent framework you meet. If you can point at all four, you can usually find your way around it.

Frequently Asked Questions

Which part of an AI agent should I build first?

Start with the loop and a single tool. Get the model to request one real function, run it and return the result. Once that works, add memory, then more tools.

Does an AI agent need a database for memory?

Not to start. Short-term memory is just the message array, and a JSON file is enough to keep a handful of notes between runs. Move to a database or a vector store when the notes outgrow a file or you need to search them by meaning.

Do I need a framework like LangChain to build these four parts?

No. The file above is the whole pattern in about a hundred lines. Frameworks package the same four parts and add conveniences, and they are easier to judge once you have written the raw version.