Mastra RAG: Answering From Your Own Docs

Learn how to ingest support documentation, generate embeddings, and give your Mastra support agent direct search access to your knowledge base using Retrieval-Augmented Generation (RAG).

In Lesson 8, you connected agents and tools inside structured workflows. In this lesson, you will give your agent access to domain-specific knowledge by indexing policy documents with Mastra RAG and querying them with a built-in vector tool.

Previous lesson: Using Agents and Tools Inside Mastra Workflows | Next lesson: Mastra MCP, Evals, and Deployment

Understanding Retrieval-Augmented Generation in Mastra

Large language models possess general knowledge, but they know nothing about your company return policy, private pricing sheets, or internal documentation. Retrieval-Augmented Generation (RAG) solves this limitation by fetching relevant passages from your own files and feeding them to the model alongside the user prompt.

Think of RAG like an open-book exam. Instead of forcing the model to memorize every detail of your store policies during training, you allow it to glance at a specific page in your handbook right before it answers a question.

Mastra provides first-class primitives for RAG through the @mastra/rag package:

1. MDocument: Loads and parses unstructured text, markdown, or HTML.
2. Chunking: Splits large documents into smaller, overlapping segments so the model receives focused context.
3. Embeddings: Converts text chunks into mathematical vector representations using an embedding model.
4. Vector Stores: Stores vector representations and metadata for fast similarity search using packages like @mastra/libsql.
5. createVectorQueryTool: Generates an agent-ready tool that handles vector lookups automatically.

To install the packages required for this lesson, run this command in your project terminal:

bash
Copy

Quick Insight: RAG reduces hallucinations because the model grounds its answers in real excerpts returned from your database.

Chunking Text and Generating Embeddings

A raw document is usually too large to pass into a prompt all at once. Even when context windows allow it, stuffing entire manuals degrades answer quality and increases token costs. You need to divide your documentation into bite-sized chunks and convert those chunks into numerical embeddings.

Mastra handles chunking with the MDocument class. After chunking, you convert each text piece into an array of floating-point numbers using embedMany from the ai package paired with Mastra's ModelRouterEmbeddingModel.

Create a script file at src/mastra/rag/seed-kb.ts to chunk and embed a sample support policy:

ts
Copy

Common Mistake: Notice the chunk configuration property is maxSize. Older tutorials or draft documentation sometimes use size, which causes a TypeScript compilation error in current versions of @mastra/rag.

Storing and Querying Vectors with LibSQLVector

Vectors must be stored in an index that supports approximate nearest neighbor search. In this local setup, you use LibSQLVector, which keeps your vector database local in a file named vectors.db.

When creating the index with store.createIndex, the dimension property must match the exact output dimension of your embedding model. The openai/text-embedding-3-small model outputs vectors with exactly 1536 dimensions. If you set the dimension to any other number, index creation or query operations will fail.

Here is how you can perform a standalone similarity search against the vector index:

ts
Copy

When storing vectors, you passed metadata: chunks.map((c) => ({ text: c.text })). Storing the original text inside the metadata field is essential. The vector search matches mathematical distances, but the LLM requires the underlying plain text stored in that metadata to read the answer.

Creating the Vector Query Tool for Your Agent

You do not need to manually write search code inside your agent instructions. Mastra provides createVectorQueryTool, which wraps a registered vector index into a standard tool that your agent can invoke whenever it needs factual policy information.

Create the tool file at src/mastra/tools/help-docs-tool.ts:

ts
Copy

The property vectorStoreName refers to the key under which the vector database is registered in your root Mastra configuration.

Now update src/mastra/index.ts to register both the persistent storage for threads and the vector database:

ts
Copy

Pro Tip: The name passed to vectorStoreName in createVectorQueryTool must match the key used in the vectors object of the Mastra class instance. Here, both use kbVector.

Putting It All Together in the Support Assistant

Now attach helpDocsTool to supportAgent alongside the orderStatusTool created in Lesson 4. The agent will have access to order tracking and the store knowledge base. Keep the memory configuration from Lesson 5 so the agent still remembers customers.

Update src/mastra/agents/support-agent.ts:

ts
Copy

Seeding only needs to happen once. If you seed inside your test script, every run inserts the same chunks again and your index fills up with duplicates. Create a one-time seed script at src/seed.ts:

ts
Copy

Run it once:

bash
Copy

Now create src/test-rag.ts to ask the agent a policy question:

ts
Copy

Run this script with tsx:

bash
Copy

The agent reads the question, sees that it is about the return policy, calls helpDocsTool, reads the retrieved text, and replies that a flat return shipping fee of $5.99 is deducted from the refund.

Try This: Add another section to policyText in seed-kb.ts detailing international shipping rules, delete the vectors.db file, run src/seed.ts again, and ask the agent whether the store ships to Canada.

In the final lesson, Mastra MCP, Evals, and Deployment, you will complete the course by connecting external tools with the Model Context Protocol (MCP), evaluating agent responses with built-in scorers, and deploying your Mastra application to production.