RAG Agents Assessment | RAG Agents for E-commerce

1

An e-commerce site has 10 million products that change daily (prices, stock). Should they use RAG or fine-tuning for their product search assistant?

Fine-tuning - it's more accurate RAG - it can access current data without retraining Neither - use traditional keyword search Fine-tuning - it's cheaper to run

2

A customer searches for 'something to keep drinks cold at the beach'. Which search type would work best?

Keyword search for 'cold' and 'beach' Exact match on product titles Semantic search - understands intent without exact keywords SQL query on the database

3

You're embedding products for a vector database. Which approach creates better embeddings?

Embed only the product title Embed only the description Combine title, description, features, and category into one embedding Create separate embeddings for each field and average them

4

What is the purpose of metadata filtering in vector search for e-commerce?

To make embeddings smaller To filter results by price, category, or stock without affecting semantic similarity To speed up the embedding process To improve LLM response quality

5

A re-ranking model is added after initial retrieval. Why?

To reduce the number of API calls Initial bi-encoder retrieval is fast but approximate; cross-encoder re-ranking is slower but more accurate To convert vectors back to text To add more products to results

6

A Product Q&A Agent is asked 'Does this laptop support 4K external monitors?' but the specs don't mention it. What should the agent do?

Say 'Yes' since most modern laptops do Say 'No' since it's not in the specs Admit the information isn't available and suggest checking the manufacturer's website or contacting support Ignore the question

7

When chunking policy documents for a FAQ chatbot, what's the ideal strategy?

Store the entire policy as one document Split by sentence Chunk by logical sections (e.g., 'Return Window', 'Non-Returnable Items') with some overlap Random 100-character chunks

8

An Order History Agent needs to find 'the headphones I bought last month'. How should it identify the correct order?

Ask for the order number Use time filter + semantic search on product names to find likely matches Show all orders and let customer scroll Assume it's the most recent order

9

What should a Policy FAQ chatbot do when a customer asks for an exception to the return policy?

Automatically grant the exception Refuse - policies are policies Explain the standard policy, acknowledge the request, and escalate to human support for special consideration Ignore the request

10

A comparison agent is comparing products from different categories (laptop vs tablet). What should it do?

Refuse to compare Compare anyway without mentioning the category difference Warn the user that these serve different purposes, then offer helpful comparison based on their use case Only compare price

11

Hybrid search combines keyword and semantic search. For the query 'Sony WH-1000XM5', what alpha value (0=keyword, 1=semantic) would work best?

alpha=1.0 (pure semantic) alpha=0.5 (balanced) alpha=0.2-0.3 (mostly keyword) alpha=0.8 (mostly semantic)

12

You're using Reciprocal Rank Fusion (RRF) to combine results. Why is RRF preferable to weighted score averaging?

RRF is faster RRF uses rank positions instead of scores, avoiding normalization issues between different search systems RRF produces more results RRF only works with vectors

13

A shopping assistant agent needs multiple tools. Which combination makes the most sense for e-commerce?

Search products, Check inventory, Compare products, Get reviews Send emails, Create documents, Schedule meetings Only search products Only LLM generation, no tools

14

Your vector database has 1 million products. A customer searches 'birthday gift for 10 year old who likes dinosaurs'. What's the optimal retrieval strategy?

Retrieve top 5 results only Retrieve top 50, filter by category/age, re-rank, return top 5 Retrieve all 1 million and filter Use only keyword search

15

Final question: What's the most important principle when building RAG agents for e-commerce?

Use the most expensive models Maximize the amount of retrieved context Ground responses in real data, be honest about limitations, and always serve the customer's actual need Respond as quickly as possible, even if less accurate

Launch Offer