Advanced RAG Architectures: Rewriting, Routing, Corrective RAG

The help desk at a college in Indore starts with one clerk and one filing cabinet. A student asks a question, the clerk pulls the first file that looks right and reads it out. It works for "When are fees due?".

Then the questions get harder. "What happens to my fees and hostel room if I take a semester break?" needs three files. "Can I appeal?" needs the clerk to ask "appeal what?". And some questions have no file at all.

So the desk improves. The clerk rephrases unclear questions, splits big ones, pulls the whole folder instead of one page, checks whether the file actually answers the question, and says "I'll find out" instead of guessing.

What are advanced RAG architectures?

Mapping the help desk: the first clerk is naive RAG. Rephrasing and splitting are query transformations. Pulling the whole folder is small-to-big retrieval. Checking the file is a relevance grader, and "I'll find out" is a corrective fallback.

Advanced RAG architectures are retrieval-augmented generation pipelines that add pre-retrieval, retrieval, post-retrieval or control-flow steps to basic RAG, so the system can handle complex questions and recover from poor retrieval.

If basic RAG is new to you, start with What is RAG?. This lesson assumes you know chunking and hybrid search with re-ranking, and uses the same fictional KIT college handbook.

Where naive RAG breaks

Naive RAG retrieves once, with the user's exact words, and trusts whatever comes back. That single pass fails in predictable ways:

Where each technique plugs into the pipeline

Every advanced pattern attaches to one of four points in the pipeline. Thinking in these stages keeps a complex system understandable.

Advanced RAG is basic RAG with extra steps at four points. Pick the stage where your measured failure happens.

1. Pre-retrieval: improve the query before searching (rewrite, decompose, route, HyDE).
2. Retrieval: search better (hybrid search, metadata filters, small-to-big).
3. Post-retrieval: clean the results (re-rank, grade relevance, compress, deduplicate).
4. Generation: check the output (cite sources, verify claims against context, retry).

Query transformation: rewriting and decomposition

Users do not write search queries; they write messages. Query rewriting asks an LLM to turn a chat message, plus recent conversation, into a clean standalone search query. "ok so what about the money thing?" after a talk about withdrawal becomes "KIT fee refund rules after withdrawal".

Query decomposition splits a multi-part question into sub-queries, retrieves for each, and merges the results. With a fixed budget of three chunks, compare one search with three focused ones:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

chunks = [
    "Semester fees are due by 15 July and a late fee of 100 rupees per day applies.",
    "Approved refunds are credited to your bank account within 10 working days.",
    "Hostel residents must be inside by 10 pm; late entry needs warden approval.",
    "Late return of library books costs 2 rupees per day per book.",
    "Each subject needs 75 percent attendance to sit the end-semester exam.",
    "Hostel mess fees are paid per semester along with the room rent.",
    "A late fee of 50 rupees per day applies to unpaid hostel mess fees.",
]
vec = TfidfVectorizer(stop_words="english").fit(chunks)
M = vec.transform(chunks)

def retrieve(q, k=3):
    s = cosine_similarity(vec.transform([q]), M)[0]
    return [int(i) for i in s.argsort()[::-1][:k] if s[i] > 0]

question = "What is the late fee per day for semester fees, the hostel curfew and the attendance needed?"
single = retrieve(question)
print("Single query:", [chunks[i][:38] for i in single])

# Decomposition: an LLM would write these; we hard-code its output
sub_queries = ["semester late fee per day", "hostel night entry time",
               "attendance needed per subject"]
merged = []
for sq in sub_queries:
    for i in retrieve(sq, k=1):
        if i not in merged:
            merged.append(i)
print("Sub-queries :", [chunks[i][:38] for i in merged])

The single query spends all three slots on fee chunks, because "late", "fee", "per day" and "semester" dominate the question. The curfew and attendance rules never reach the LLM. Three focused sub-queries return one chunk for each part.

A cousin of decomposition is multi-query retrieval: generate three or four paraphrases of the same question, retrieve for each, and fuse the lists with Reciprocal Rank Fusion. It costs one extra LLM call and catches wording mismatches. HyDE goes further and searches with a generated answer instead of the question; it has its own lesson, HyDE explained.

Routing and adaptive RAG

Not every message needs retrieval. "Thanks!" needs none, "When are fees due?" needs one lookup, and the three-part question above needs decomposition. Adaptive RAG puts a small classifier, often a cheap LLM call, in front of the pipeline to pick the route.

Routing also chooses where to search: the fee policy collection, the hostel collection, or a SQL table of exam dates. A good router saves money on easy questions and spends it where it helps.

Small-to-big (parent-child) retrieval

Small chunks match precisely; big chunks give the LLM enough context. Small-to-big retrieval gets both: you search over small child chunks, such as single sentences, but return the parent section they belong to.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

# Small-to-big: search small child chunks, return their whole parent section
sections = {
    "Fee refunds": ["If you withdraw before classes start, the full fee is refunded minus 1,000 rupees.",
                    "Withdrawal in the first 15 days of classes earns a 50 percent refund.",
                    "No refund is given after the first 15 days.",
                    "Approved refunds are credited within 10 working days."],
    "Library": ["Students may borrow 4 books for 14 days.",
                "Late return costs 2 rupees per day per book."],
}
children = [(title, s) for title, sents in sections.items() for s in sents]
vec = TfidfVectorizer(stop_words="english").fit([c for _, c in children])
scores = cosine_similarity(vec.transform(["Do I get money back if I quit in week 2 of classes?"]),
                           vec.transform([c for _, c in children]))[0]
best_title, best_child = children[scores.argmax()]

print("Matched child :", best_child)
print("Sent to LLM   :", best_title, "->", " ".join(sections[best_title]))

The search matches the single sentence about the first 15 days, but the LLM receives the whole refund section. It can then answer properly: 50 percent back, credited within 10 working days. A variant called sentence-window retrieval returns the matched sentence plus two neighbours on each side instead of a whole section.

Corrective RAG: grade, then fall back

Naive RAG always uses whatever it retrieved, even when nothing relevant exists. Corrective RAG (CRAG), proposed in a 2024 research paper, adds a grader after retrieval. If the retrieved chunks look relevant, it continues.

If they look weak, it rewrites the query and tries again, or falls back to another source such as a trusted website. If nothing works, it says so.

Corrective RAG checks retrieval before generating. Weak results trigger a rewrite, then a fallback, instead of a confident wrong answer.

In production the grader is usually a small LLM call or a re-ranker score. Here it is a similarity threshold, which shows the control flow:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

chunks = ["Approved refunds are credited within 10 working days.",
          "Hostel residents must be inside by 10 pm; late entry needs warden approval.",
          "Revaluation costs 300 rupees per paper and must be requested within 7 days.",
          "Each subject needs 75 percent attendance to sit the end-semester exam."]
vec = TfidfVectorizer(stop_words="english").fit(chunks)
M = vec.transform(chunks)

def retrieve(q):
    s = cosine_similarity(vec.transform([q]), M)[0]
    return int(s.argmax()), float(s.max())

def corrective_rag(question, rewrites, good=0.3):
    for attempt, q in enumerate([question] + rewrites):
        i, score = retrieve(q)
        verdict = "relevant" if score >= good else "weak"
        print(f"  try {attempt}: {q!r} -> score {score:.2f} ({verdict})")
        if score >= good:
            return f"Answer from: {chunks[i]}"
    return "Fallback: say 'I don't know' or ask a human (or search a trusted site)"

print(corrective_rag("When is my money returned?", ["refund credited days"]))
print(corrective_rag("Who is the college principal?", ["principal name"]))

The refund question fails at first (score 0.00) and succeeds after the rewrite (0.56). The principal question fails both times, because the handbook simply does not say, so the system falls back honestly. Self-RAG takes the idea into generation: the model is trained to decide when to retrieve and to critique its own output against the passages.

GraphRAG, multimodal RAG and agentic RAG

Three bigger architectures change the knowledge source or the control loop:

• GraphRAG extracts entities and relationships (student types, rules, departments) into a knowledge graph, often with community summaries. It helps with questions that connect many documents or ask for themes across a whole collection. It costs a lot of LLM calls at indexing time.
• Multimodal RAG embeds and retrieves images, tables, scanned forms or audio alongside text, so a question can be answered from a fee chart or a campus map.
• Agentic RAG gives an LLM agent retrieval as one tool among several. The agent plans, searches several times, calls APIs and decides when it has enough. Agentic RAG covers it in depth.

How to choose: add one layer at a time

1. Build naive RAG with good chunking and a test set of real questions.
2. Measure: is the right chunk retrieved? Is the answer faithful? See Evaluating RAG systems.
3. Look at failures and group them by type, using the failure table above.
4. Add the single pattern that targets the largest group, then measure again.
5. Stop when quality meets your target. Every layer adds latency and something else to debug.

Common mistakes with advanced RAG

• Starting with agents. An agentic system built on poor chunking is a slow, expensive way to retrieve the wrong chunk.
• No fallback path. Without a grader and an "I don't know" route, every pattern still ends in confident guessing.
• Unbounded retries. Cap rewrites at one or two, and log each attempt.
• Ignoring latency. Three LLM calls before retrieval can push a 1-second answer to 5 seconds. Route easy questions to the fast path.

Key takeaways

• Advanced RAG adds steps at four points: pre-retrieval, retrieval, post-retrieval and generation.
• Query rewriting and decomposition fix vague and multi-part questions cheaply.
• Small-to-big retrieval matches on small chunks but sends whole sections.
• Corrective RAG grades retrieval and falls back instead of guessing.
• GraphRAG and agentic RAG are powerful but costly; add them only when measurements justify it.

Patterns like these are easiest to learn by breaking and fixing a real pipeline. Build RAG pipelines step by step with mentors at Debugshala, and work through problem-decomposition puzzles on Debugshala Test.