In our previous guide on building a type-safe AI agent backend with Next.js 14 Server Actions, we wired up our Zod input schemas, execution trace logger, and a 5-step orchestration loop using a deterministic offline planner. That gave us a rock-solid server pipeline, but an agent with hardcoded if/else rules cannot understand natural developer questions or reason over unfamiliar bug reports. Now it is time to replace that mock planner with a live Large Language Model brain.

To launch a production-ready chatbot agent in Next.js using OpenAI or Anthropic Claude APIs, you should wrap both vendor SDKs behind a unified server-side provider adapter that normalizes system prompts, conversation history, and token usage telemetry. Calling this adapter inside a Next.js Server Action lets your agent switch between gpt-4o and claude-3-5-sonnet-latest through a single configuration flag while keeping API credentials, sliding-window message limits, and latency tracking strictly on the server.

Most beginner tutorials show you how to paste new OpenAI() directly inside an API route, send a single string, and print the reply. That works for a five-minute demo, but it falls apart the moment you need multi-turn conversation memory, strict system role boundaries, token cost visibility, or the flexibility to swap from OpenAI to Claude when one provider experiences elevated latency.

Why Build a Dual-Provider Brain (OpenAI + Claude) for Your Agent?

When you build a technical triage and documentation agent like our reference project DevPulse, relying on a single LLM provider creates an unnecessary single point of failure. In real engineering workflows, different models excel at different tasks:

• Anthropic Claude (claude-3-5-sonnet-latest / claude-3-5-haiku-latest): Tends to follow complex multi-clause system prompts, code analysis rules, and negative constraints ('Never invent error codes') with high precision.
• OpenAI (gpt-4o / gpt-4o-mini): Offers fast time-to-first-token and strict structured output schemas that are great for high-throughput triage turns.

However, OpenAI and Anthropic do not share the same API payload format. If you hardcode OpenAI's message structure directly into your React components and Server Actions, switching to Claude later requires rewriting your entire backend. Designing a thin, type-safe adapter layer on day one solves this permanently.

Understanding the Payload Mismatch Between OpenAI and Anthropic

Before writing code, look at the three concrete API differences our Next.js backend must normalize:

1. System Prompt Placement: OpenAI accepts { role: 'system', content: '...' } as the first element inside the messages array. Anthropic's Messages API rejects role: 'system' inside messages with an HTTP 400 error; it requires system to be passed as a separate top-level property.
2. Role Alternation Rules: Anthropic strictly requires the messages array to start with a user message and alternate between user and assistant. If a network retry accidentally pushes two consecutive user messages into history, Anthropic fails the request unless your backend merges or sanitizes them.
3. Response and Token Telemetry Shape: OpenAI returns text inside response.choices[0].message.content and token counts in usage.prompt_tokens / usage.completion_tokens. Anthropic returns an array of content blocks (response.content[0].text) and reports tokens as usage.input_tokens / usage.output_tokens.

Here is how our unified LLM adapter sits inside the Next.js Server Action flow to shield the rest of the application from these vendor quirks:

`mermaid flowchart TD UI([React Chatbot UI - DevPulseChat.tsx]) -->|messages + modelId| SA[Next.js Server Action - runDevPulseChat] SA --> Sanitize[Zod Validation + Sliding Window History Normalizer] Sanitize --> Adapter{Unified LLM Adapter - llm-client.ts}

Adapter -->|modelId = gpt-4o| OpenAI[OpenAI SDK: chat.completions.create]
Adapter -->|modelId = claude-3-5-sonnet| Claude[Anthropic SDK: messages.create]

OpenAI --> Normalize[Normalize Reply + Token Telemetry]
Claude --> Normalize

Normalize --> Trace[Append AgentStepTrace with Latency & Token Count]
Trace --> UI

`

Architecture Flow Summary: The React UI sends the conversation history and selected modelId to the Next.js Server Action, which validates the input with Zod and trims the message window. The unified LLM adapter (llm-client.ts) routes the request to either the OpenAI or Anthropic SDK with provider-specific formatting, normalizes the response text and token metrics into a single schema, logs the execution trace, and returns the state to the client.

Step 1: Installing the Official SDKs and Securing Environment Variables

Run the following command in your Next.js project root to install both official TypeScript SDKs alongside server-only (which triggers a build-time error if any client component ever accidentally imports our LLM module):

bash npm install openai @anthropic-ai/sdk zod server-only

Next, create or update your .env.local file. Never prefix these keys with NEXT_PUBLIC_, as doing so exposes your billing credentials in the browser bundle:

`ini

.env.local

OPENAI_API_KEY=sk-proj-your-openai-key-here ANTHROPIC_API_KEY=sk-ant-your-anthropic-key-here `

Step 2: Building the Unified LLM Adapter (src/lib/agent/llm-client.ts)

Let us create src/lib/agent/llm-client.ts. This module imports 'server-only' on line 1, initializes the SDK clients lazily so next build never crashes in CI environments without API keys, normalizes consecutive messages, and returns a clean NormalizedLLMResponse object.

`typescript // src/lib/agent/llm-client.ts import 'server-only'; import OpenAI from 'openai'; import Anthropic from '@anthropic-ai/sdk'; import { type AgentConfig } from './blueprint';

export interface ChatMessage { role: 'user' | 'assistant'; content: string; }

export interface NormalizedLLMResponse { reply: string; modelUsed: string; promptTokens: number; completionTokens: number; latencyMs: number; }

function getOpenAIClient(): OpenAI { const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error('Missing OPENAI_API_KEY in server environment variables.'); } return new OpenAI({ apiKey }); }

function getAnthropicClient(): Anthropic { const apiKey = process.env.ANTHROPIC_API_KEY; if (!apiKey) { throw new Error('Missing ANTHROPIC_API_KEY in server environment variables.'); } return new Anthropic({ apiKey }); }

export function sanitizeConversationHistory( messages: ChatMessage[], maxWindowSize = 10 ): ChatMessage[] { const trimmed = messages.slice(-maxWindowSize); const merged: ChatMessage[] = [];

for (const msg of trimmed) { const cleanContent = msg.content.trim(); if (!cleanContent) continue;

const previous = merged[merged.length - 1];
if (previous && previous.role === msg.role) {
  previous.content = `${previous.content}\n\n${cleanContent}`;
} else {
  merged.push({ role: msg.role, content: cleanContent });
}

}

if (merged.length > 0 && merged[0].role === 'assistant') { merged.shift(); }

return merged; }

export async function generateAgentReply( messages: ChatMessage[], config: AgentConfig ): Promise { const startTime = Date.now(); const safeMessages = sanitizeConversationHistory(messages, 10);

if (config.modelId === 'gpt-4o') { const openai = getOpenAIClient(); const response = await openai.chat.completions.create({ model: 'gpt-4o', temperature: config.temperature, max_tokens: 1024, messages: [ { role: 'system', content: config.systemPrompt }, ...safeMessages.map((m) => ({ role: m.role, content: m.content, })), ], });

const replyText =
  response.choices[0]?.message?.content ??
  'No response generated by OpenAI.';

return {
  reply: replyText,
  modelUsed: response.model,
  promptTokens: response.usage?.prompt_tokens ?? 0,
  completionTokens: response.usage?.completion_tokens ?? 0,
  latencyMs: Date.now() - startTime,
};

}

const anthropic = getAnthropicClient(); const response = await anthropic.messages.create({ model: 'claude-3-5-sonnet-latest', max_tokens: 1024, temperature: config.temperature, system: config.systemPrompt, messages: safeMessages.map((m) => ({ role: m.role, content: m.content, })), });

const firstTextBlock = response.content.find( (block) => block.type === 'text' ); const replyText = firstTextBlock && 'text' in firstTextBlock ? firstTextBlock.text : 'No text block returned by Claude.';

return { reply: replyText, modelUsed: response.model, promptTokens: response.usage.input_tokens, completionTokens: response.usage.output_tokens, latencyMs: Date.now() - startTime, }; } `

Take a close look at sanitizeConversationHistory() in the file above. It performs three critical jobs automatically before any API call leaves your server:

• Sliding Window Cap (slice(-10)): Keeps only the 10 most recent turns so a long chat session never blows up your token bill.
• Consecutive Role Merging: If a user double-submits or an earlier assistant reply failed, it merges adjacent same-role messages so Anthropic never throws a role-alternation error.
• Leading User Turn Guarantee: Ensures the conversation array always begins with { role: 'user' }.

Step 3: Upgrading the Next.js Server Action (src/app/actions/agent.ts)

Now let us update src/app/actions/agent.ts from Post 13 so our DevPulse backend uses generateAgentReply() to power a real multi-turn Chatbot Agent while recording token telemetry in AgentStepTrace.

`typescript // src/app/actions/agent.ts 'use server';

import { z } from 'zod'; import { defaultDevPulseConfig, type AgentStepTrace, } from '@/lib/agent/blueprint'; import { generateAgentReply, type ChatMessage, } from '@/lib/agent/llm-client';

const chatTurnSchema = z.object({ sessionId: z.string().min(8), modelId: z.enum(['gpt-4o', 'claude-3-5-sonnet-latest']).default('claude-3-5-sonnet-latest'), messages: z .array( z.object({ role: z.enum(['user', 'assistant']), content: z.string().trim().min(1).max(4000), }) ) .min(1), });

export interface ChatbotAgentResponse { success: boolean; reply: string; modelUsed: string; promptTokens: number; completionTokens: number; steps: AgentStepTrace[]; errorMessage?: string; }

export async function runDevPulseChatbot( rawInput: unknown ): Promise { const parsed = chatTurnSchema.safeParse(rawInput);

if (!parsed.success) { return { success: false, reply: '', modelUsed: 'none', promptTokens: 0, completionTokens: 0, steps: [], errorMessage: parsed.error.issues.map((i) => i.message).join('; '), }; }

const { messages, modelId } = parsed.data; const activeConfig = { ...defaultDevPulseConfig, modelId, };

try { const llmResult = await generateAgentReply( messages as ChatMessage[], activeConfig );

const traceStep: AgentStepTrace = {
  stepIndex: 1,
  thoughtSummary: `Invoked ${llmResult.modelUsed} (${llmResult.promptTokens} in / ${llmResult.completionTokens} out tokens)`,
  durationMs: llmResult.latencyMs,
};

return {
  success: true,
  reply: llmResult.reply,
  modelUsed: llmResult.modelUsed,
  promptTokens: llmResult.promptTokens,
  completionTokens: llmResult.completionTokens,
  steps: [traceStep],
};

} catch (err) { const message = err instanceof Error ? err.message : 'Unexpected LLM provider error'; return { success: false, reply: '', modelUsed: modelId, promptTokens: 0, completionTokens: 0, steps: [], errorMessage: message, }; } } `

Step 4: Building the Interactive Chatbot Agent UI (src/components/DevPulseChat.tsx)

Finally, let us build the client component that maintains the multi-turn conversation state, allows you to switch between Claude 3.5 Sonnet and GPT-4o on the fly, and displays real-time token and latency telemetry.

`tsx // src/components/DevPulseChat.tsx 'use client';

import { useState, useTransition } from 'react'; import { runDevPulseChatbot, type ChatbotAgentResponse, } from '@/app/actions/agent'; import { type ChatMessage } from '@/lib/agent/llm-client';

export default function DevPulseChat() { const [messages, setMessages] = useState<ChatMessage[]>([]); const [input, setInput] = useState(''); const [modelId, setModelId] = useState<'claude-3-5-sonnet-latest' | 'gpt-4o'>( 'claude-3-5-sonnet-latest' ); const [lastMetrics, setLastMetrics] = useState<ChatbotAgentResponse | null>(null); const [isPending, startTransition] = useTransition();

function handleSend(e: React.FormEvent) { e.preventDefault(); const trimmed = input.trim(); if (!trimmed || isPending) return;

const updatedMessages: ChatMessage[] = [
  ...messages,
  { role: 'user', content: trimmed },
];
setMessages(updatedMessages);
setInput('');

startTransition(async () => {
  const res = await runDevPulseChatbot({
    sessionId: 'devpulse-local-session-01',
    modelId,
    messages: updatedMessages,
  });

  setLastMetrics(res);
  if (res.success) {
    setMessages([
      ...updatedMessages,
      { role: 'assistant', content: res.reply },
    ]);
  }
});

}

return ( DevPulse Chatbot Agent System Contract Active | Sliding Window: 10 turns <select value={modelId} onChange={(e) => setModelId(e.target.value as 'claude-3-5-sonnet-latest' | 'gpt-4o') } className='bg-slate-950 border border-slate-700 rounded-lg px-3 py-1.5 text-xs text-slate-200' > Claude 3.5 Sonnet OpenAI GPT-4o

  <div className='h-80 overflow-y-auto space-y-3 p-3 bg-slate-950 rounded-lg border border-slate-800/80'>
    {messages.length === 0 && (
      <p className='text-xs text-slate-500 text-center py-12'>
        Ask DevPulse a technical question about API rate limits, auth, or bug triage.
      </p>
    )}
    {messages.map((m, idx) => (
      <div
        key={idx}
        className={`p-3 rounded-lg text-sm ${
          m.role === 'user'
            ? 'bg-cyan-950/50 border border-cyan-800/50 ml-12'
            : 'bg-slate-900 border border-slate-800 mr-12'
        }`}
      >
        <div className='text-[10px] uppercase tracking-wider text-slate-400 mb-1'>
          {m.role}
        </div>
        <div className='whitespace-pre-wrap'>{m.content}</div>
      </div>
    ))}
  </div>

  <form onSubmit={handleSend} className='flex gap-2'>
    <input
      value={input}
      onChange={(e) => setInput(e.target.value)}
      placeholder='Describe an issue or ask about API behavior...'
      className='flex-1 px-3 py-2 rounded-lg bg-slate-950 border border-slate-700 text-sm'
    />
    <button
      type='submit'
      disabled={isPending}
      className='px-4 py-2 rounded-lg bg-cyan-600 hover:bg-cyan-500 disabled:opacity-50 text-sm font-semibold'
    >
      {isPending ? 'Thinking...' : 'Send'}
    </button>
  </form>

  {lastMetrics && (
    <div className='text-xs text-slate-400 flex justify-between pt-2 border-t border-slate-800'>
      <span>Model: {lastMetrics.modelUsed}</span>
      <span>
        Tokens: {lastMetrics.promptTokens} in / {lastMetrics.completionTokens} out
      </span>
      <span>
        Latency: {lastMetrics.steps[0]?.durationMs ?? 0}ms
      </span>
    </div>
  )}
</div>

); } `

A Real Debugging Lesson from Multi-Provider Integration

When I first built a multi-provider chat interface in Next.js so I could compare OpenAI and Anthropic responses side by side during development, everything worked smoothly as long as every turn succeeded. The bug appeared the first time a request failed mid-flight.

A slow network connection caused an assistant call to time out, leaving the user's message appended at the end of the React messages state array without a corresponding 'assistant' reply. When I clicked the Send button again with a follow-up question, the state array now contained two consecutive { role: 'user' } objects at the tail. When gpt-4o was selected, OpenAI processed the two consecutive user messages without complaint. The moment I switched the dropdown to claude-3-5-sonnet-latest, the entire Server Action failed with an Anthropic 400 invalid_request_error: messages: roles must alternate between 'user' and 'assistant'.

Even worse, when I initially tried passing { role: 'system' } inside the shared message array, Anthropic rejected that too. Writing the sanitizeConversationHistory() utility you saw in llm-client.ts—which automatically merges consecutive same-role messages, strips empty strings, and keeps the system prompt separated in AgentConfig—completely eliminated those 400 errors across both SDKs.

Common Mistakes When Integrating OpenAI and Claude in Next.js

Before moving on to real-time token streaming in the next post, audit your setup against these four frequent mistakes:

1. Instantiating SDK Clients at the Top Level of the Module

If you write const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }) at the global top level of llm-client.ts, running next build in a Docker container or CI pipeline where runtime secrets are not injected at build time will immediately crash the build. Always instantiate the client lazily inside a helper function like getOpenAIClient() right when a request is executed.

2. Sending Unbounded Conversation Arrays on Every Turn

LLMs are stateless, which means every turn re-sends the conversation history. If a user exchanges 30 messages with code snippets and you send all 30 messages on turn 31, you are paying for tens of thousands of redundant input tokens on every single click. Always enforce a sliding window (like messages.slice(-10)) before calling the API.

3. Using NEXT_PUBLIC_ for LLM API Keys

Never name your environment variable NEXT_PUBLIC_OPENAI_API_KEY or import the OpenAI/Anthropic SDK inside a 'use client' component. Next.js inlines NEXT_PUBLIC_ variables directly into the public JavaScript bundle sent to every visitor's browser, allowing bots to scrape your key in seconds.

4. Ignoring the wait time on blocking responses

Right now, await generateAgentReply() waits until the entire paragraph is generated before returning the Server Action response. For short answers, that takes 1.5 to 2 seconds; for longer technical explanations, the user stares at 'Thinking...' for 5 to 8 seconds. That exact UX bottleneck is why we will implement real-time token streaming with the Vercel AI SDK in Post 15.

Frequently Asked Questions

Should I start with OpenAI (gpt-4o) or Claude (claude-3-5-sonnet-latest) for a Next.js chatbot agent?

Both models work exceptionally well, which is why our adapter supports both out of the box. In practice, many developers use claude-3-5-sonnet-latest when the agent needs to strictly adhere to technical documentation rules and complex code triage instructions, and use gpt-4o or gpt-4o-mini when optimizing for lower cost or high-speed classification turns.

Why not call fetch('[https://api.openai.com/v1/](https://api.openai.com/v1/)...') directly instead of installing the official SDKs?

The official openai and @anthropic-ai/sdk packages provide automatic TypeScript definitions for every request and response field, built-in exponential backoff retries on transient HTTP 429 or 5xx network errors, and clean timeout handling—features you would otherwise have to write and maintain manually around fetch.

How do we eliminate the 4-to-8 second wait time when the agent generates a long explanation?

In a blocking request-response call, the Server Action waits until the final token is produced before resolving. In the very next post (Post 15), we will upgrade this setup with response streaming using the Vercel AI SDK so tokens appear on the user's screen word by word within milliseconds.