03.1
Parametric Knowledge and External Evidence
A language model's parameters encode statistical regularities acquired during training, whereas external material supplies task information that can be updated and located. RAG adds retrieved material to the generation context so that the model can answer with reference to it. The foundational work combines parametric generation with non-parametric memory for knowledge-intensive tasks. This does not imply that retrieved knowledge is permanently written into the model's weights, nor does it guarantee that the generator will use only the supplied evidence. [R06]
An application must distinguish what a model might know from what the system is permitted to use as the basis of an answer. Question answering over institutional documents often requires conclusions to follow a specified version of internal records, even when general knowledge would allow a plausible response. When the records are insufficient, the desired behavior may be to explain that insufficiency rather than fill the gap with common knowledge. We recommend expressing evidential requirements as testable rules. For example, a statement about an institution's procedure should cite a location in a currently applicable document. Compliance should be examined using missing material, obsolete documents, and contradictory records, not only demonstrated with ordinary questions. The central control problem in RAG is therefore the selection of evidence and the manner in which it is used. This framing also explains why a stronger generator alone cannot repair every retrieval failure.
03.2
Retrieval Units and Segmentation Strategies
Segmentation determines which objects a retriever can return. Fixed-length chunks are easy to implement, but they can separate a condition from its conclusion, a table heading from its values, or an image from its explanation. Segmentation by headings, paragraphs, or layout regions retains more structure, but still has to handle tables continuing across pages and references spanning paragraphs. Smaller chunks can provide more precise localization while lacking the context needed for independent interpretation. Larger chunks preserve more information but consume additional context and can introduce irrelevant material. We treat segmentation as the design of evidence boundaries rather than as a preprocessing step controlled only by length.
Retrievable objects should retain their parent document, neighboring passages, and structural location. When retrieval returns a number, the system also needs its metric, unit, period, and population. When it returns an image region, it needs the source page and version. Graph structure can connect these objects only if the identifiers required for those connections were preserved during ingestion. For procedures, clauses, and experimental material, a useful design is to combine fine-grained localization with retrieval of the surrounding structure. The system first identifies specific evidence and then adds the local context required to interpret it. The effectiveness of this design should be evaluated jointly through the support it provides for answers and the context resources it consumes. A segmentation method that improves one retrieval metric while repeatedly removing essential conditions may be unsuitable for the actual task.
03.3
Candidate Retrieval and Ranking Combinations
Different retrieval signals can complement one another. Lexical matching preserves names, identifiers, and explicit expressions. Vector similarity helps locate material phrased differently but related in meaning. Graph neighborhoods can expand retrieval to associated objects. Time and permissions constrain whether a candidate is usable. A hybrid design should specify the responsibility of each signal. Simply adding scores does not make them commensurable, because their scales, distributions, and calibration can differ. Fusion should be examined on a development set instead of manually tuned to make a small demonstration look favorable.
Reranking passes a smaller candidate set through a more detailed relevance model or rule. It can improve ordering, but it cannot restore necessary evidence that was never retrieved. If an answer depends on two dispersed records, scoring each passage against the question independently can also miss the value of their combination. We recommend recording where candidates came from, why they were filtered, and how their order changed. Evaluation should examine whether the selected set covers the complete evidence chain. It is insufficient to optimize only the similarity of the first result. Some questions require one explicit record, others require a relationship to be completed, and still others require coverage across several themes. Distinguishing these requirements allows retrieval design to remain aligned with the research question. Comparative GraphRAG studies and graph-guided expansion methods provide reasons to examine this alignment, without establishing one universally optimal fusion rule. [R09][R13]
03.4
Context Organization and Generation Constraints
Retrieval leaves a separate problem unresolved: how should material be organized so that the model uses it appropriately? Evidence order, duplication, conflicting sources, and long-distance dependencies can all affect the context presented to the generator. Research has observed that the position of relevant information changes performance for the models and tasks examined. The ability to fit material into a context window should therefore be evaluated separately from the ability to use it effectively. This observation is not a fixed performance claim about every later model, but it motivates controlled experiments involving position and length. [R29]
We recommend packaging context as a set of evidence objects with explicit sources and statuses. Original excerpts, machine-generated summaries, and speculative explanations should remain distinguishable. Questions requiring precise attribution should retain the passages directly supporting the conclusion. Questions requiring synthesis can use summaries while preserving routes back to their sources. Generation instructions should require appropriate qualification when evidence is missing, conditions do not match, or records conflict. The existence of such an instruction does not establish that it is effective; evaluation must inspect the actual answers. A particularly problematic practice is to reintroduce the model's previous answer as if it were independent evidence, creating a self-supporting loop. If earlier generated material is reused, its derived status and original basis must be retained. Otherwise repetition can make a claim appear corroborated when every instance ultimately comes from the same unsupported generation.
03.5
Multi Hop Questions and Query Routing
A multi-hop question connects several evidential steps, but the number of steps is not identical to reasoning difficulty. A two-step question may require difficult entity disambiguation, whereas a longer path may involve an explicit hierarchy. The system should identify the required relationships, object types, and conditions rather than mechanically increase traversal depth. Asking which counters serve dishes containing chicken requires connections among counters, dishes, and ingredients. Adding a date also requires the relevant menu version. If one condition is missing, a connected path is still an incomplete answer.
Query routing can assign a request to factual lookup, relational expansion, global summarization, cross-modal verification, or a combination of these operations. The router can itself fail, so fallback behavior and execution records should be preserved. A useful study compares fixed routing, rule-based routing, and model-based routing while holding the corpus and generation budget constant. An advantage obtained by using test answers or by manually selecting the best route after seeing the outcome does not represent online performance. We recommend classifying routing mistakes separately. A system can possess adequate evidence but choose an unsuitable way of organizing it for the question. Recognizing that failure prevents unnecessary reconstruction of the whole graph. It also supports a more precise experiment: improve route selection while keeping extraction and generation unchanged, then determine whether the affected question class improves without an unacceptable loss elsewhere.
Original design
03.6
Decomposing End to End Failure
An incorrect answer can originate in the data, extraction, retrieval, organization, generation, or presentation stage. If the source lacks the required information, even an excellent retriever cannot supply genuine support. If entities were incorrectly merged, graph retrieval may accurately return an invalid path. If correct evidence was retrieved but excluded from the final context, organization or budgeting is responsible. If the model disregarded clear evidence, generation behavior requires examination. An end-to-end score alone cannot distinguish these causes. Evaluation should retain intermediate objects at each stage so that an outcome can be traced back to its inputs. [R25][R26]
We propose attributing errors to the earliest failure that can be located. First check whether the collection contains sufficient evidence, then whether the representation preserves it, then inspect candidate retrieval and ranking, and finally examine the conclusion and its citations. A condition that supplies the correct evidence can reveal whether the generator still fails under ideal retrieval. This condition is diagnostic rather than a measure of deployment performance, but it helps estimate the potential benefit of improving retrieval. Error analysis should also include answers that happen to be correct while relying on the wrong evidence. Text-only scoring can overlook such fragile successes. A dependable system must address both whether the answer is correct and why the available material authorizes that answer. This decomposition makes improvement more economical, because each intervention can target a demonstrated failure rather than changing several components at once and obscuring the cause of any gain.