Does Your Small Business Need RAG? The Cost Math Before You Build

RAG (retrieval-augmented generation) means the AI searches your documents first and answers from the few passages it finds. It is the right tool when your documents are too big or too restricted for the simpler option. For most small businesses they are neither. If your handbook, SOPs, product sheets, and support FAQ add up to less than about 160,000 tokens, you can put all of it in every prompt, cache it, and skip the pipeline. In our worked example below, that costs about $70 a month to run on Claude Sonnet 5.5, against $19 to $44 for RAG, but RAG costs roughly $5,000 more to build. At that gap, the pipeline takes 8 to 16 years to pay for itself.

• Measure before you build. Count your documents in tokens. On current Claude models, 1M tokens is roughly 555,000 words, so 150,000 tokens is about 83,000 words.
• Under about 160,000 tokens: full context with prompt caching is cheaper or close to it per question, and cheaper to build.
• RAG earns its cost when the documents exceed the 1M-token context window, when different staff may only see some documents, or when a large knowledge base gets steady all-day traffic.
• All prices are Anthropic and OpenAI list prices as of 2026-10-20.

What is the option most small businesses skip?

The option is to send the whole knowledge base with every question and let prompt caching absorb the cost. Anthropic says it directly in its contextual retrieval write-up: if the knowledge base is under 200,000 tokens, you can include all of it in the prompt and not need RAG at all. That advice is older than the current models, and it has only become more true. Claude Sonnet 5.5 has a 1M-token context window, billed at the same per-token rate across the whole window.

Caching is what makes this affordable. Per Anthropic's prompt caching docs, the first request writes your documents to a cache, and later requests that start with the identical text read them back at a fraction of the input price. A 1-hour cache costs 2x the input price to write, and every hit refreshes the hour at no charge. On Sonnet 5.5, a cache read is 0.05x the input price: $0.10 per million tokens instead of $2.

What does each option cost to run each month?

Here is a worked example, not a client case. A 12-person company with a 150,000-token knowledge base. Staff ask 40 questions a day, 22 working days a month, so 880 questions. Each question is about 300 tokens with instructions, and each answer about 500 tokens. Questions come in bursts, so we assume the 1-hour cache goes cold and gets rewritten 4 times a day (88 writes a month). Prices are from Anthropic's pricing page as of 2026-10-20: Sonnet 5.5 at $2 input, $4 for 1-hour cache writes, $0.10 for cache reads, and $10 output, all per million tokens.

• Full context, cached: a cache hit costs $0.015 for the documents plus about $0.006 for the question and answer, so $0.021 per question. A cache write costs $0.61. That is 792 hits and 88 writes, about $70 a month.
• Full context, no caching: $0.31 per question, about $269 a month. Caching is the whole trick; without it, this option is the expensive one.
• RAG: ten retrieved chunks of 800 tokens plus the question is about 8,300 input tokens and the same 500-token answer, so $0.022 per question, about $19 a month in model costs. Embeddings are close to free: OpenAI's text-embedding-3-small is $0.02 per million tokens on its pricing page, so embedding the whole knowledge base costs under a cent. Storage is free if you already run Postgres, since pgvector on Supabase is an extension, or $25 a month for a new Supabase Pro project. Total: $19 to $44 a month.

Per question, the two options are almost the same price: $0.021 against $0.022. The full-context bill is higher only because of the cache rewrites when nobody asks anything for an hour. If questions arrive all day and the cache never goes cold, the writes nearly disappear, and the break-even sits at 160,000 tokens: below it, a cached full prompt is cheaper per question than the RAG prompt. That figure uses Sonnet 5.5's 0.05x cache-read rate. On a model with the standard 0.1x rate, such as Sonnet 5, it halves to about 80,000 tokens.

What does each option cost to build?

The build gap is far larger than the running gap. A full-context assistant is a prompt, a cache setting, a small app or Slack bot, and a way to update the documents. Our basic Claude integrations, such as a custom assistant or document analysis, start at $5,000 to $15,000. A production RAG pipeline adds ingestion, chunking, embeddings, a vector store, retrieval tuning, and evaluation of whether the right passages come back. We put that at $10,000 to $30,000 in our RAG vs fine-tuning comparison.

Take the low ends of both ranges: RAG costs at least $5,000 more. In the example, it saves $26 to $51 a month. $5,000 divided by $51 is 98 months; divided by $26, it is 195 months. That is 8 to 16 years, longer than most documents stay current and much longer than most model generations last. You can plug your own numbers into our AI cost estimator to see where your project lands.

When does a small business actually need RAG?

You need RAG when the full-context option breaks, not when it is merely less fashionable. Go through these checks in order and stop at the first yes.

• The documents do not fit. Past the 1M-token context window there is no choice. In practice, switch well before that, since a 500,000-token prompt costs about $220 a month in the example even with caching.
• Not everyone may see everything. If the sales team must not read HR files, one shared cached prompt leaks them. RAG with access filters, where each search only covers documents the user is allowed to see, is the clean fix. Separate cached prompts per role also work when there are only two or three roles.
• A large knowledge base with steady traffic. Above about 160,000 tokens with questions all day, RAG is cheaper per question, and at thousands of questions a month the saving can cover the build cost in a reasonable time.
• The documents change hourly. Caching needs an identical prefix, so every edit to the cached text forces a new write. Inventory or pricing that changes all day belongs in a database lookup, not in a cached document.

What goes wrong with each approach?

RAG fails quietly when the search step misses. In Anthropic's own tests, a standard embedding setup failed to retrieve the right passage in the top 20 results 5.7% of the time. Their best combination of techniques brought that to 1.9%, at the cost of a more complex pipeline. When retrieval misses, the model answers from whatever it did get, and the answer often sounds just as confident. Full context has no search step to miss, so the model always has every document in front of it.

Full context fails on cost if someone forgets the cache, as the $269 line shows, and on staleness if nobody owns the documents. Both are cheap to prevent. Log cache hits from the API usage data and alert when the hit rate drops. Give one person the job of updating the source documents, the same way someone owns the price list.

What should you do this week?

Collect the documents your team actually asks about and count the tokens with Anthropic's token counting endpoint or any tokenizer. If the total is under 160,000 tokens and everyone may see everything, build the cached full-context version first. It is the cheaper project, and it gives you real questions to test a RAG pipeline against later if you outgrow it. Track cost per question from day one; we explain how to judge whether an automation pays in our AI automation ROI worksheet. If you fall into one of the RAG cases, or you are not sure, our Claude integration team will tell you which side of the line you are on before you pay for either.