Embeddings, Vector Search & RAG - Interview-Ready Framework for MBA Students
Why does a chatbot confidently answer a generic question, but struggle when the answer is hidden in page 47 of a policy PDF? The issue is not “AI intelligence” - it is whether the system can find the right knowledge before it speaks.
- Embeddings convert text, images or other data into numeric vectors so similar meanings sit close together.
- Vector search retrieves items by semantic similarity, not just exact keyword match.
- RAG - retrieval-augmented generation - retrieves relevant context first, then asks an LLM to answer using that context.
- RAG is useful when answers must be current, private, cited or domain-specific - policies, legal documents, product manuals, research reports.
- A good RAG system is not just “LLM + vector DB”; it needs clean documents, chunking, metadata, retrieval, reranking, prompting, evaluation and guardrails.
- Measure retrieval and answer quality separately: Recall@K, Precision@K, MRR, nDCG, groundedness and latency.
- The biggest trap: saying “RAG removes hallucinations.” It reduces hallucinations only if retrieval, grounding and evaluation are strong.
Big Picture
Think of RAG as an open-book exam for an LLM. The embedding model creates a searchable map of meaning; vector search finds the most relevant pages; the LLM writes the final answer using those pages as evidence.
Core Explanation
The central idea is simple: LLMs are good at language, but weak at controlled knowledge access. They may not know your company’s latest HR policy, SEBI compliance note, underwriting rule or product manual. RAG fixes this by giving the model the right context at answer time.
1. Embeddings: turning meaning into coordinates
An embedding is a dense vector representation that places semantically similar items close together in mathematical space.
For example, “home loan prepayment charges” and “can I close my housing loan early?” may share few exact words, but their embeddings should be close because the intent is similar.
2. Vector search: finding meaning, not just matching words
Vector search retrieves nearest vectors to a query vector using a similarity measure such as cosine similarity.
Traditional search asks, “Do these words match?” Vector search asks, “Is this meaning close?” This is why it helps with synonyms, paraphrases, multilingual queries and messy user language.
3. RAG: grounding generation in retrieved evidence
Retrieval-augmented generation is a pattern that retrieves external evidence and supplies it to a generative model before it answers.
The “retrieval” part improves relevance and factuality; the “generation” part makes the answer readable, summarized and conversational.
The moving parts of a RAG system
Worked example: cosine similarity in vector search
Suppose a user asks about “loan prepayment.” The query embedding and two document embeddings are simplified into three dimensions:
- Query vector q = [0.6, 0.8, 0]
- Document 1 d1 = [0.5, 0.86, 0] - about prepayment charges
- Document 2 d2 = [0.9, 0.1, 0.4] - about credit card rewards
Cosine similarity = dot product divided by vector lengths.
- cos(q, d1) = (0.6×0.5 + 0.8×0.86 + 0×0) / (1 × 0.995) ≈ 0.993
- cos(q, d2) = (0.6×0.9 + 0.8×0.1 + 0×0.4) / (1 × 0.990) ≈ 0.626
So the vector search engine ranks Document 1 higher, even if the user did not use the exact document wording.
What to measure in a RAG system
Do not evaluate RAG as one black box. Retrieval can fail even if the LLM is strong; generation can fail even if retrieval is perfect.
The RAG quality stack
Most weak RAG prototypes fail at the bottom layers: poor source quality, messy chunking or weak retrieval. A better prompt cannot compensate for missing evidence.
Definitions
- Embedding: A dense vector representation that places semantically similar items close together in mathematical space.
- Vector search: Retrieving nearest vectors to a query vector using a similarity measure such as cosine similarity.
- RAG: A pattern that retrieves external evidence and supplies it to a generative model before it answers.
- Chunk: A small passage of source content stored and retrieved as one unit of evidence.
- Grounding: Constraining an AI answer to information present in retrieved or otherwise trusted sources.
IRCTC’s AskDISHA virtual assistant shows why retrieval matters in India-scale service journeys: rail passengers ask messy questions about refunds, cancellations, PNR status and booking rules. In such a setting, a RAG design principle is critical - answers must be grounded in official, current railway rules and user-specific context, not the model’s memory. The so what: in regulated or high-volume Indian services, retrieval quality is not a technical luxury; it is customer trust.
Case Study - Thomson Reuters: RAG for High-Stakes Legal Research
Thomson Reuters built generative AI into legal research by grounding answers in trusted legal content, proving why RAG matters when users need citations, not confident guesses.
Situation: Legal professionals do not just need a fluent summary. They need to know which case, statute or commentary supports it. A general chatbot may produce a plausible legal answer, but plausibility is not enough when professional risk is high.
The move: Thomson Reuters integrated generative AI into products such as legal research and professional workflow tools by combining large language models with retrieval from trusted legal and tax content. The primary driver was grounded access to proprietary, curated content. Supporting drivers included legal-domain workflows, source citations, expert review, structured product interfaces and enterprise-grade controls.
The lesson: RAG is strongest where three conditions meet: knowledge is specialized, errors are costly and users need traceability. The answer is not just “use an LLM”; the answer is “connect the LLM to a reliable evidence system.”
The strategic takeaway: RAG creates competitive advantage when a company owns trusted content and embeds it into a workflow where accuracy, speed and verification all matter.
How AI Changes Embeddings, Vector Search & Retrieval-Augmented Generation
By 2026, RAG is moving from “simple vector search plus prompt” to more adaptive retrieval systems.
For interview prep, load a company annual report, product FAQ and 2-3 recent articles into NotebookLM. Ask: “Create 10 customer or analyst questions that require retrieval across these sources, then show which source supports each answer.” This trains you to think like a RAG designer: source, retrieve, cite, answer.
Interview Relevance
“Suppose a bank wants an internal AI assistant for relationship managers to answer product, compliance and process questions. How would you design a RAG system, and how would you evaluate whether it works?”
Use this sentence in interviews: “I would not evaluate the chatbot only by whether the final answer sounds good; I would separately test whether it retrieved the right evidence and whether every claim in the answer is grounded in that evidence.”
Common Mistake
The mistake that costs candidates: saying “RAG solves hallucination” as if adding a vector database automatically makes an LLM factual. It costs you because it ignores the real failure points - bad documents, poor chunking, weak retrieval, missing citations and no evaluation. One-line fix: say “RAG reduces hallucination only when retrieval quality, grounding instructions and evidence-based evaluation are designed properly.”
What to Revise Next
Now move from retrieval to reliable execution. Revise Prompting for Analysis: Patterns That Produce Reliable Output to learn how to structure the LLM’s reasoning and answer format, then study AI-Assisted SQL & Python: Getting Code You Can Trust to connect AI outputs with verifiable data work.