RAG over the Product Catalog

Back in lesson 2 the assistant answered "Do you have running shoes under 3000?" by cheerfully making things up — it had no idea what ShopKart actually stocks. In lesson 5 you built semantic search over the real catalog. This lesson connects the two: retrieve the relevant products, hand them to the model as context, and let it answer from real inventory. That pattern is Retrieval-Augmented Generation (RAG), and it is the single most important technique for a trustworthy shopping assistant.

Why a bare LLM is not enough

A language model only knows what was in its training data. It has never seen ShopKart's catalog, does not know today's prices or stock, and when it does not know something it tends to guess confidently. For an e-commerce assistant that is dangerous: promising a product you do not sell, or a price that does not exist, erodes trust and creates support tickets.

RAG fixes this by changing where the answer comes from. Instead of "answer from memory", the instruction becomes "here are the relevant ShopKart products; answer using only these." The model's language ability is still used to phrase a helpful reply, but the facts come from your data, retrieved fresh at query time. Update a price in your database and the very next answer reflects it — no retraining.

RAG the hard way (to see what the advisor does)

Before the one-liner, it helps to see RAG assembled by hand, using the pieces you already have. Retrieve, format, augment, generate:

@Service
public class ManualRagService {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;

    public ManualRagService(ChatClient shopkartChatClient, VectorStore vectorStore) {
        this.chatClient = shopkartChatClient;
        this.vectorStore = vectorStore;
    }

    public String answer(String question) {
        // 1 + 2: embed + retrieve
        var docs = vectorStore.similaritySearch(SearchRequest.builder()
                .query(question).topK(4).similarityThreshold(0.5).build());

        // 3: build a context block from retrieved products
        String context = docs.stream()
                .map(Document::getText)
                .collect(java.util.stream.Collectors.joining("\n---\n"));

        // 4: generate, instructing the model to use ONLY this context
        return chatClient.prompt()
                .system("""
                    Answer the shopper using ONLY the ShopKart products in the
                    context below. If nothing there fits, say we don't seem to
                    carry a match and offer to help differently. Never invent
                    products, prices, or stock.
                    """)
                .user(u -> u
                    .text("Context:\n{context}\n\nQuestion: {question}")
                    .param("context", context)
                    .param("question", question))
                .call()
                .content();
    }
}

That works, and understanding it means you can debug RAG when it misbehaves. But Spring AI packages exactly this flow into an advisor so you do not hand-roll it every time.

RAG the easy way: QuestionAnswerAdvisor

An advisor is a component that hooks into the ChatClient request pipeline. QuestionAnswerAdvisor does the retrieve-and-augment steps for you: it runs the similarity search against your VectorStore and injects the retrieved documents into the prompt as context, all before the model sees the request. You just attach it.

Configure it as a default advisor so every call is grounded:

package com.shopkart.assistant;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class RagAssistantConfig {

    @Bean
    public ChatClient ragChatClient(ChatClient.Builder builder, VectorStore vectorStore) {
        return builder
                .defaultSystem("""
                    You are Kartik, the ShopKart shopping assistant. Answer using
                    the retrieved ShopKart products provided to you. If none fit,
                    say so honestly. Prices are in Indian Rupees. Never invent
                    products, prices, or stock levels.
                    """)
                .defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore).build())
                .build();
    }
}

The service now looks exactly like plain chat — the advisor makes it RAG invisibly:

@Service
public class ShoppingAssistant {

    private final ChatClient chatClient;

    public ShoppingAssistant(ChatClient ragChatClient) {
        this.chatClient = ragChatClient;
    }

    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

Ask the question that started this course:

Shopper: Do you have running shoes under 3000?
Kartik:  Yes! We have the Asics Gel-Contend 8 Running Shoes at Rs 2,799 -
         a lightweight daily trainer with a cushioned midsole that works
         well even for flat feet, rated 4.3 out of 5. Would you like more
         options in this range?

That answer is grounded: the product, price, and rating all come from the retrieved Document, not from the model's imagination. Change the price in your catalog and re-index, and the next answer updates automatically.

Tuning retrieval

The quality of a RAG answer is mostly the quality of what you retrieve. Garbage context yields a garbage answer. Control retrieval by giving the advisor a custom SearchRequest:

QuestionAnswerAdvisor.builder(vectorStore)
        .searchRequest(SearchRequest.builder()
                .topK(6)                    // pull a few more candidates
                .similarityThreshold(0.6)   // but only reasonably close ones
                .build())
        .build();

• topK too low and you may miss the right product. Too high and you flood the prompt with weak matches, costing tokens and diluting the answer. Start around 4 to 6.
• similarityThreshold too low lets irrelevant products in. Too high and valid but loosely worded matches get dropped. Tune against real queries from your logs.

You can also override the filter or search at runtime per request. For example, restrict retrieval to one category using metadata you stored on the documents:

String answer = chatClient.prompt()
        .user("gift under 1000 rupees")
        .advisors(a -> a.param(
                QuestionAnswerAdvisor.FILTER_EXPRESSION, "category == 'Sports'"))
        .call()
        .content();

When nothing relevant is found

An honest assistant says "we don't carry that" instead of inventing a product. Two defences work together:

• Set a similarityThreshold so genuinely irrelevant queries retrieve nothing.
• Instruct the model in the system prompt to admit when the context is empty (as we did above).

Shopper: Do you sell live goldfish?
Kartik:  I'm sorry, it doesn't look like ShopKart carries live pets or
         goldfish. I can help you find aquarium supplies or something else -
         what are you shopping for?

If you need stricter behaviour, the more advanced RetrievalAugmentationAdvisor (from the spring-ai-rag module) lets you configure an empty-context policy explicitly and add query rewriting. QuestionAnswerAdvisor covers the majority of catalog Q&A cases and is the right default.

Keeping the index fresh

RAG answers are only as current as your vector store. When products change, the index must change:

• On catalog updates, re-index changed products. Because Documents carry a productId in metadata, you can delete and re-add just the affected ones rather than rebuilding everything.
• Schedule a periodic reconciliation for safety, or index on the same event that updates the product in your primary database.
• Never let stock or price live only in the embedded text. Keep them in metadata (and ideally verify critical facts like stock via a tool call, which is exactly what the next lesson adds).

Key takeaways

• RAG grounds the assistant in your real catalog: retrieve relevant products at query time and instruct the model to answer only from them.
• The flow is embed the query, retrieve top-K from the VectorStore, augment the prompt, generate.
• QuestionAnswerAdvisor.builder(vectorStore).build() added via defaultAdvisors/advisors does retrieve-and-augment for you; the calling code stays like plain chat.
• Tune topK and similarityThreshold on the advisor's SearchRequest; retrieval quality drives answer quality.
• Use a threshold plus a "say when you don't know" system instruction so the assistant admits when ShopKart has no match.
• Keep the index fresh on catalog changes; prefer tool calls for volatile facts like live stock and price.

Next up: Tool Calling — letting the assistant act, not just answer: check stock, look up orders, place an order.