You Don't Need a Vector Database for AI Conversation Context

Someone on your team just said "we need conversation memory for our AI agents." By tomorrow, you'll have a Jira ticket for a RAG pipeline, a vector database, an embedding service, and a retrieval strategy. By next month, you'll have a system that costs more to run than the AI itself.

Here's the thing: your users aren't writing novels. They're sending WhatsApp messages. "Room 204, AC broken." Then two minutes later: "Actually 205, not 204." That's the conversation. That's the context you need to handle.

You don't need semantic search across 10,000 messages. You need to remember what happened two minutes ago.

The Overengineering Trap

The standard advice for AI conversation context goes something like this:

1. Store every message in a vector database
2. Generate embeddings for each message
3. On every new message, do a similarity search
4. Retrieve the top-k most relevant messages
5. Inject them into the prompt

For a customer support chatbot handling hour-long debugging sessions? Sure, maybe. For an AI agent that processes short, transactional message bursts? That's a rocket launcher for a nail.

The real cost isn't just the infrastructure. It's the failure modes: stale embeddings, irrelevant retrievals, token budget blown on context that doesn't help, and a retrieval pipeline that adds latency to every request.

The One-Line Summary Pattern

Here's what we do instead. After every agent run, the LLM generates a one-line summary of the conversation so far:

{
  "intent": "maintenance_request",
  "confidence": 0.95,
  "actions": [
    { "type": "operations.maintenance.create", "payload": { "room": 204, "issue": "AC broken" } }
  ],
  "context_summary": "Guest in room 204 reported broken AC, maintenance ticket created."
}

That context_summary field? That's your entire conversation memory. One sentence. Generated by the same LLM call that processes the message. No extra API call, no embedding, no database query.

When the next message comes in from the same contact, the agent receives:

{
  "last_message": "Actually it's room 205, not 204",
  "context_summary": "Guest in room 204 reported broken AC, maintenance ticket created.",
  "last_agent_output": {
    "intent": "maintenance_request",
    "actions": [{ "type": "operations.maintenance.create", "payload": { "room": 204 } }]
  }
}

The LLM reads the summary, understands the correction, and returns:

{
  "intent": "maintenance_request",
  "actions": [
    { "type": "operations.maintenance.update", "payload": { "room": 205, "issue": "AC broken" } }
  ],
  "context_summary": "Guest corrected room to 205. AC broken, maintenance ticket updated."
}

No retrieval. No embeddings. No vector search. The LLM summarizes its own context, and that summary chains forward.

Why the LLM Should Summarize Itself

You could build a separate summarization service. An extra API call after each run that compresses the conversation history. But why?

The agent already understands the conversation — it just processed it. Asking it to output a context_summary field alongside its actions costs almost nothing: a few extra tokens in an output it's already generating.

And the summary is better. A generic summarizer would say "user mentioned room 204 and AC." The agent writes "Guest in room 204 reported broken AC, maintenance ticket created." It captures the state, not just the content.

The prompt instruction is minimal:

The input may contain context_summary and last_agent_output from a previous exchange.
- Interpret the new message in light of the previous context
- Carry forward fields unless the customer explicitly changes them
- If the customer corrects a previous value, use the corrected value

Three rules. That's the entire context management system.

TTL: Context That Expires

Not all context is useful forever. If someone reports a broken AC at 2pm and sends "hello" at 9pm, that's probably a new conversation. Treating it as a continuation would be confusing.

Every agent has a context_ttl — a time-to-live in minutes. Default: 60. If the last interaction was within the TTL window, context is injected. If not, the agent starts fresh.

context_ttl = 60 minutes (default)

14:00 — "AC broken in 204"        → no context (first message)
14:02 — "Actually 205"            → context injected (2 min ago)
14:15 — "Can I also get towels?"  → context injected (15 min ago)
21:00 — "Hello"                   → no context (TTL expired)

Set it to 0 to disable context entirely. Set it to 480 for an agent handling all-day conversations. The right value depends on your use case, not on a one-size-fits-all architecture.

The Carry-Forward Problem

There's one subtlety that trips people up: what happens when a message is relevant to the conversation but doesn't require an action?

Customer: "Room 204, AC broken."
Agent:    → operations.maintenance.create (room: 204)

Customer: "Thanks"
Agent:    → system.noop.skip (no action needed)

Customer: "Oh and the TV remote doesn't work either"
Agent:    → ???

If noop resets the context, the agent doesn't know which room the customer is in. The third message becomes "someone somewhere has a TV issue."

The fix: noop doesn't clear context. The summary carries forward even when the agent takes no action. The noop run still saves its context_summary: "Guest in room 204, AC maintenance created, guest acknowledged."

When the TV message arrives, the agent has full context: room 204, existing maintenance request, and a new issue to add.

What This Covers (and What It Doesn't)

This pattern handles short, transactional conversations. The kind where:

• A customer sends 2-5 messages about the same topic
• Each message builds on the previous one
• The conversation resolves within minutes or hours
• Context means "what just happened," not "what happened last month"

This is 95% of messaging interactions. Room service requests, support tickets, booking modifications, order status checks. The user sends a request, maybe corrects a detail, confirms, done.

What it doesn't handle:

• Long-running sessions — a 50-message debugging thread where message #47 references something from message #3
• Cross-conversation memory — "remember I'm allergic to shellfish" across multiple stays
• Multi-topic threads — a single conversation that covers three unrelated requests

For those, yes, you probably need something more sophisticated. A retrieval system, maybe a knowledge graph. But don't build that infrastructure for the 95% case. Build it when the 5% case actually shows up in your logs.

The Full Flow

Message arrives from +33612345678
  ↓
Lookup conversation_key: "business_account_42:+33612345678"
  ↓
Find last completed run for this agent + contact
  ↓
Within TTL? ──No──→ Fresh run (no context)
  │
  Yes
  ↓
Inject into input:
  - context_summary (previous run's summary)
  - last_agent_output (previous intent + actions)
  ↓
LLM processes message + context
  ↓
Output includes new context_summary
  ↓
Save context_summary → ready for next message

No database for history. No embedding pipeline. No retrieval. Just a text field that chains from one run to the next.

The Cost Comparison

Let's be honest about what the alternatives cost:

The summary approach adds zero latency, zero infrastructure, and zero API calls. The "cost" is a few extra output tokens per run — negligible compared to the base cost of the LLM call itself.

When to Graduate

You'll know it's time to move beyond this pattern when:

• Users explicitly reference messages from days ago ("like I said last Tuesday")
• Your average conversation length exceeds 10 messages
• You need cross-agent context (agent A's output feeds agent B's context — a different pattern entirely)
• Compliance requires full conversation audit trails

When that happens, add the retrieval layer on top of the summary pattern, not instead of it. The summary is still useful as a fast-path: check the summary first, fall back to retrieval only when the summary isn't enough.

Start simple. Add complexity when your logs tell you to.

This is how we handle conversation context in WhatsRB Cloud — a one-line summary, a configurable TTL, and zero extra infrastructure. For 95% of WhatsApp conversations, it's all you need. Try it out.