How to Integrate AI Into Your Web App (Next.js + OpenAI + RAG, 2026 Guide)

Users do not just want apps that store and retrieve data anymore. They expect software that understands context, drafts text, summarizes documents, and answers questions in natural language. AI integration in your web app has moved from differentiator to baseline expectation.

This guide is what I actually do when a client asks me to add AI to their existing Next.js or MERN application. It is not "wrap ChatGPT in a textarea." It is the production patterns: orchestration layer, RAG over proprietary data, streaming, rate limiting, and cost controls.

The architecture that actually works

LLMs are stateless and have no idea what your database contains. Building an AI feature means building an orchestration layer between three things:

1. Your frontend (the UI that triggers the AI feature).
2. Your backend (Next.js API routes or a separate Node/Go service).
3. The LLM provider (OpenAI, Anthropic Claude, Google Gemini).

The frontend must never talk to the LLM directly. Every AI call goes through your backend so you can: authenticate the user, check rate limits, attach business context (RAG retrieved chunks), inject system prompts, and meter usage per tenant.

Pattern 1: Simple generative feature

The starting point is one focused generative feature. For example, a SaaS for real estate agents that turns 5 bullet points into a listing description.

The flow:

1. User submits bullets from the React form.
2. Frontend POSTs to your Next.js API route.
3. API route authenticates, builds the prompt, calls the LLM, streams back the response.

The examples below use AI SDK 7, the current major version as of October 2026. If you have older code that returns result.toDataStreamResponse(), it needs updating: AI SDK 5 replaced that helper with toUIMessageStreamResponse (5.0 migration guide), and AI SDK 7 deprecates the response methods on the streamText result in favor of standalone stream helpers (7.0 migration guide). Version 7 also needs Node.js 22 or later.

import { streamText, createTextStreamResponse, toTextStream } from "ai";
import { openai } from "@ai-sdk/openai";

export async function POST(req: Request) {
  const { propertyDetails } = await req.json();

  const result = streamText({
    model: openai("gpt-6.1-sol"),
    system:
      "You write luxurious, inviting real estate listing descriptions in 3 short paragraphs.",
    prompt: `Property bullet points: ${propertyDetails}`,
  });

  return createTextStreamResponse({
    stream: toTextStream({ stream: result.stream }),
  });
}

That is a working endpoint in under 20 lines. The SDK handles streaming and lets you switch providers by changing one line. If you are building a chat interface with the useChat hook from @ai-sdk/react, return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }) }) instead, as shown in the Next.js App Router quickstart. I recommend the AI SDK over calling a provider's raw SDK directly in most cases.

Pattern 2: Structured output (when prose is not enough)

Most AI features eventually need structured output, not just text. Tagging emails by intent, extracting line items from invoices, classifying tickets. For this use structured outputs via Zod schemas.

generateObject was deprecated in AI SDK 6. The current approach is generateText with an output setting, and the result is read from output instead of object (6.0 migration guide).

import { generateText, Output } from "ai";
import { z } from "zod";
import { openai } from "@ai-sdk/openai";

const ticketSchema = z.object({
  category: z.enum(["billing", "technical", "general"]),
  urgency: z.enum(["low", "medium", "high"]),
  confidence: z.number().min(0).max(1),
  summary: z.string().max(140),
});

const { output } = await generateText({
  model: openai("gpt-6-luna"),
  output: Output.object({ schema: ticketSchema }),
  prompt: `Classify this support ticket: ${ticket.body}`,
});

// output is now type safe and validated
if (output.confidence > 0.9 && output.category === "billing") {
  await routeToBilling(ticket.id, output.summary);
}

This pattern eliminates the "the AI returned malformed JSON" class of bugs that plague naive integrations.

Pattern 3: Retrieval Augmented Generation (RAG)

When your AI needs to answer questions about your data, simple prompts will not work. LLMs have context limits and hallucinate when asked about things they were not trained on.

Retrieval Augmented Generation (RAG) is the architecture that solves this. The AI does not memorize your data. It searches your data, retrieves the relevant pieces, and uses them as context for each answer.

The four phases of a RAG system

1. Ingestion and chunking. Take your data (PDFs, docs, FAQs, past emails) and split it into chunks of a few hundred tokens each. LangChain has good built in splitters.
2. Embedding. Convert each chunk into its own vector with an embedding model, so a long document becomes many vectors, one per chunk. The number of dimensions comes from the model, not from the length of the text. OpenAI's text-embedding-3-small returns 1,536 dimensions by default and text-embedding-3-large returns 3,072, and both accept a dimensions parameter to shorten them (OpenAI embeddings guide). Your vector column must match whatever size you choose.
3. Storage. Store vectors in a vector database. For most projects, Supabase pgvector is the right call because it lives next to your existing Postgres data and avoids a separate service. Pinecone or Weaviate make sense at scale.
4. Retrieval and generation. When a user asks a question, embed the query with the same model, find the top 5 nearest vectors, inject those chunks into the LLM prompt with instructions to answer only from the provided context.

Minimal RAG implementation with Supabase pgvector

import { openai } from "@ai-sdk/openai";
import { embed, generateText } from "ai";
import { createClient } from "@supabase/supabase-js";

const supabase = createClient(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_SERVICE_KEY!,
);

export async function answerQuestion(question: string, tenantId: string) {
  // 1. Embed the user's question with the same model used for the chunks
  const { embedding } = await embed({
    model: openai.embedding("text-embedding-3-small"),
    value: question,
  });

  // 2. Find the 5 most similar chunks for this tenant
  const { data: chunks } = await supabase.rpc("match_documents", {
    query_embedding: embedding,
    match_threshold: 0.78,
    match_count: 5,
    tenant_filter: tenantId,
  });

  // 3. Inject chunks into the prompt
  const context = (chunks ?? [])
    .map((c: { content: string }) => c.content)
    .join("\n---\n");

  const { text } = await generateText({
    model: openai("gpt-6.1-sol"),
    system:
      "Answer using only the provided context. If the answer is not in the context, say you don't know.",
    prompt: `Context:\n${context}\n\nQuestion: ${question}`,
  });

  return text;
}

This is roughly the architecture I shipped for VistaChat (hospitality inbox assistant) and other production grade AI integrations.

Five production grade practices (skip these and your AI feature will break)

1. Never expose API keys to the client

The number of side projects with OPENAI_API_KEY shipped in a public NEXT_PUBLIC_ variable is depressing. Every API call must go through your backend. Period.

2. Implement rate limiting and per tenant quotas

LLM APIs are billed by token. One malicious user looping a generative endpoint can run up a very large bill overnight. Add:

• IP based rate limiting before the model is called, either in the route handler or in a Next.js 16 proxy.ts file (the replacement for middleware.ts). Upstash Ratelimit is a common choice.
• Per user quotas tracked in your database.
• Hard kill switches if usage suddenly spikes far above normal.

3. Always stream

LLM responses can take many seconds to finish. A spinner for that long feels broken. Streaming responses word by word, like ChatGPT, makes the wait feel much shorter. The AI SDK handles this with the stream helpers shown above.

4. Pick the right model per task

In 2026 the smart move is model routing: cheap and fast for simple tasks, premium for complex reasoning. The current lineups as of October 2026, from the official OpenAI, Anthropic, and Google Gemini model pages:

• Classification, tagging, short summaries: GPT-6 Luna, Claude Haiku 4.5, Gemini 3.5 Flash-Lite.
• General chat, drafting, RAG answers: GPT-6.1 Sol, Claude Sonnet 5.5, Gemini 3.8 Flash.
• Long context, careful reasoning, agentic work: GPT-6 Astra, Claude Opus 5.5 or Claude Fable 5.1 (both with a 1M token context window), Gemini 3.1 Pro (in preview).

The price gap is large. On Anthropic's models overview, Claude Haiku 4.5 costs $1 per million input tokens and $5 per million output tokens, while Claude Fable 5.1 costs $10 and $50, as of October 2026. Sending simple tasks to the small model is where most of the savings come from.

5. Log everything

Log prompt, response, tokens used, latency, and user ID for every AI call. You will need this for debugging, billing reconciliation, and prompt iteration. Tools like Langfuse, Helicone, or a simple Supabase table all work.

What I would skip in 2026

• LangChain heavyweight chains. Use it for primitives (text splitters, document loaders) but the AI SDK plus structured outputs handle most production needs more cleanly.
• Vector only RAG for small datasets. If your whole knowledge base fits comfortably in the model's context window, just put it in the prompt and let the model figure it out. Vector DBs add complexity that is only worth it at scale.
• Building everything from scratch. If you need a chat UI, use assistant-ui or the AI SDK's useChat hook. Save build time for the proprietary parts.

Frequently asked questions

What is the easiest way to add AI to a Next.js app?

Install ai (the AI SDK) and @ai-sdk/openai, add your OPENAI_API_KEY to env, and use streamText in a Route Handler. With AI SDK 7, return the result through createTextStreamResponse or createUIMessageStreamResponse. You can ship a working AI feature in a few dozen lines.

What is RAG in simple terms?

RAG (Retrieval Augmented Generation) is how you make an AI answer questions about your data without retraining the model. You split your data into chunks, convert each chunk into its own vector, and at query time retrieve the most relevant chunks and feed them to the LLM as context.

Should I use OpenAI, Claude, or Gemini?

Use whichever gives the best price to quality ratio for your task, measured on your own data. All three offer a small, fast tier and a premium tier. Claude Opus 5.5, Claude Sonnet 5.5, and Claude Fable 5.1 all have a 1M token context window, which helps with long documents. Most production apps use 2 to 3 providers via the AI SDK and switch based on the task.

Is RAG better than fine tuning?

For most business use cases, yes. RAG is faster to build, cheaper, easier to update (just re-embed changed docs), and produces verifiable answers grounded in your data. Fine tuning is for narrow style or format adjustments, not knowledge.

How much does AI integration cost to run?

Inference costs scale with usage, so estimate from your own traffic. As an example, 10,000 calls a month with about 1,000 input tokens and 300 output tokens each is 10 million input and 3 million output tokens. On Claude Haiku 4.5 at $1 and $5 per million tokens, that is about 25 USD a month, as of October 2026 (Anthropic models overview). The same traffic on Claude Fable 5.1, at $10 and $50 per million tokens, is about 250 USD a month, which is why model routing matters.

Ready to add AI to your product?

If you want a secure, scalable AI integration built into your existing web app, reach out. I have shipped production RAG systems including VistaChat and DuChat. See my services for the full scope.