The Vapi knowledge base: how it works, and how to load your knowledge base into it

On Vapi, the knowledge base is a tool your assistant has to decide to use. You upload files, and when you publish the assistant, Vapi creates a query tool that searches them. Retrieval only happens when the assistant calls that tool, so an assistant that isn't told when to call it answers from general knowledge instead, confidently.

This page is written from Vapi's own documentation, checked on 25 September 2026. KB Builder is not affiliated with Vapi. Our own live calls so far ran on a different platform, so the advice below applies what we learned there; it has not been tested on Vapi calls yet.

How the Vapi knowledge base works

	What the docs say
Where it lives	Files in the dashboard; selected in the assistant's Files or Knowledge Base section
How the assistant uses it	A query tool, created when you publish. The assistant calls it when it decides to
What you can add	.txt, .pdf, .docx, .doc, .csv, .md, .tsv, .yaml, .json, .xml, .log
Size	Vapi advises keeping each file under 300 KB; no hard limit is stated
Retrieval engine	Google, with Gemini models
Chunk size, chunks per search	Not stated in the docs
Updating a file	Only the name can be changed; new content means a new file
Extra charge	None stated

Vapi's API reference also lists a newer Knowledge Bases V2, with files attached to a knowledge base and a knowledgeBase tool type. The guides don't describe it yet, so the steps below follow the documented dashboard flow.

Where it goes wrong on calls

The assistant doesn't search. This is the most common Vapi failure. If the system prompt doesn't name the tool and say when to use it, the assistant often answers a price question from what it thinks is likely. Vapi's docs say so plainly: the knowledge base description alone is not sufficient.

The search returns a price without its product. When the assistant does search, it gets back pieces of your file. If a piece carries "₹12,000 per cycle" and the treatment's name sits in another piece, the assistant can quote the IUI price to a caller asking about IVF. This is the failure that cost us two wrong prices on live calls.

Old and new files both answer. Because a file's content can't be replaced in place, it's easy to add a new version and forget to remove the old one from the tool. Then two prices for one product come back.

How to load a KB Builder file into Vapi

A KB Builder knowledge base is a Markdown file written in short sections that each name the product or service they are about, so a piece still makes sense when it arrives alone. A small business's file is usually well under Vapi's 300 KB advice.

1. In the dashboard, go to Files and upload the .md file.
2. Open the assistant, find the Files (or Knowledge Base) section and select the file.
3. Add the tool instruction below to the system prompt, using your tool's name.
4. Click Publish. Vapi creates the query tool.
5. Make test calls that ask the price of two similar products, and check in the call log that the tool was called.

When the business's website changes, rebuild the file in KB Builder, upload the new file, swap it in on the assistant, publish, and delete the old file.

Lines to add to the assistant's prompt

First, the line Vapi needs so the tool is used at all. Replace the tool name with yours:

When the caller asks about the business's products, services, prices, timings, policies or contact details, always use the knowledge-base query tool before answering.

Then the lines from our own live calls:

Answer questions about the business only from the knowledge base. If it doesn't have the answer, say you will check, and give the business's phone number.
Give a price, time or rule only from a sentence that names the product or service the caller asked about.
Whenever you give a price, say the name of the product or service with it.
If the caller hasn't said which product or service they mean, ask before answering.
When the knowledge base says something has not been confirmed, say exactly that. Never guess.
Don't promise messages, callbacks or bookings unless the knowledge base says the business offers them.

A note for API users

If you attach the tool through the API, Vapi's docs warn that a PATCH to the assistant must include the whole model object, not just toolIds, or it overwrites the rest of the model's settings.