08.1
Preserve Original Resources and Versions at Ingestion
This chapter proposes an implementable reference architecture as an original design synthesis, not a claim of deployment in an enterprise. Ingestion accepts text, page images, tables, and business records and first creates a resource inventory rather than immediately extracting knowledge. Each resource needs a stable identifier, content checksum, origin, collection time, applicable time, permissions, and processing status. Changed content at one URL should create a new version. Repeated uploads of one file should be recognized while retaining upload events. These foundations determine whether evidence can later be located, errors repaired, and derived data removed.
Parsing should preserve mappings between original resources and normalized outputs. PDF pages, image coordinates, table cells, and video timestamps need different locators but a consistent evidence interface. Failures should record their type and affected scope instead of silently skipping content, and partial success should be allowed. OCR text is a derived observation; the scanned page remains the verification reference. Evolving business systems also require receipt time to be distinguished from effective time so late arriving records do not appear to describe newly occurring events. Ingestion quality limits the system: later models cannot recover pages, units, or version conditions already discarded at the entrance.
08.2
Maintain Observation and Entity Layers Separately
Extraction should produce typed observations instead of directly overwriting entity properties. An observation needs at least a subject candidate, relation or attribute, value, qualifiers, evidence locator, extraction method, and review status. The entity layer manages stable objects and identity resolution; the observation layer records what different sources state. Separation permits multiple supporting or conflicting observations without forcing one truth value at write time. Confirmed application attributes can be derived from reviewed observations while retaining their support sets.
Ontology and extractor versions need an explicit contract. If an ontology divides cooking method into preparation and heating method, old outputs cannot automatically be interpreted under the new structure. Preserve ontology versions, model versions, and mapping rules, then recompute affected data or provide explicit compatibility. Entity resolution should also avoid a single irreversible batch operation. Generate candidates, record merge decisions and evidence, and update query views afterward. A discovered error should permit reversal of one merge without losing original observations. This design adds storage and management work but provides repairability. Automatic graph construction without a repair route lets early errors propagate through later community reports and answers.
08.3
Coordinate Graph Vector and Resource Indexes
A unified architecture does not require every object to reside in one database. A resource store holds original files and images, a graph store represents entities, events, and evidence relations, a vector index supplies similar candidates, and a keyword index supports exact names and identifiers. Stable identifiers connect them. Graph databases serve relationship constraints and neighborhood access, while vector indexes serve approximate similarity. Deployment can start with a simple combination and introduce sharding or dedicated services when measured latency, scale, or operational requirements justify them.
Consistency across indexes is the central engineering issue. If a withdrawn resource loses its graph edges but its old vectors remain retrievable, invalid content can still appear in answers. Maintain index status by resource version with explicit states for pending, queryable, invalid, and rebuilding, and check evidence versions at query time. Updates can build a new index before switching a version pointer, avoiding partially built views. Deletion should first block retrieval, then clean derived stores asynchronously while recording completion. Unification primarily concerns identity, version, and evidence contracts, rather than one storage technology. This interpretation keeps architecture understandable and allows individual components to be replaced.
08.4
Query Planning Produces an Inspectable Evidence Package
The query service identifies objects, time, scope, and answer type before choosing retrieval actions. Exact identifiers may use direct lookup, relational questions constrained neighborhoods, thematic questions community reports, and visual questions regions or pages. Routing should record triggering conditions and fallback behavior when uncertain. A request with several subquestions can use an explicit plan stating which evidence each part requires instead of asking a model to guess the entire answer at once.
The evidence package supplied to generation should contain source passages, locators, relation paths, temporal conditions, permission checks, and conflict notices. Original material and summaries need different labels, as do facts and inferences. Claim level verification after generation checks whether each sentence has sufficient support and whether its citation points to relevant content. If evidence supports only part of a request, answer that part and explain the gap. The package can be an independent interface for different language models or for human reviewers. The durable asset becomes a reviewable, comparable, and regenerable evidence collection rather than one fluent answer. Model upgrades can then be assessed for changes in expression separately from changes in retrieval.
Original design
08.5
Incremental Updates Summary Invalidation and Rollback
Updating a document can affect more than one chunk. It may change entity identity, relation weights, community membership, multiple summaries, and cached answers. Derived dependencies should therefore form a graph of their own: which observations generated a summary, which resources generated an observation, and which summaries a cached answer used. Changes can then invalidate dependent objects rather than triggering either an indiscriminate rebuild or an insufficient file only update. Community recomputation may affect a broad region, so local index updates should be distinguished from structural repartitioning.
Rollback should be an ordinary capability. If a model upgrade doubles extracted statements, recall may have improved, or negations and hypotheses may have become facts. Compare changes on fixed samples, inspect high impact differences, and switch gradually. Preserve configuration, versions, input manifests, and quality summaries for each release so the previous usable state can be restored. Historical answers may retain evidence snapshots explaining their original output, while clearly indicating whether that evidence remains current. Correctness in an evolving system is temporal. One offline evaluation cannot prove it permanently. Operations must manage the entry of new knowledge, invalidation of old knowledge, and reversal of errors.
Original design
08.6
Minimum Delivery Contracts from Prototype to Service
A demonstration may only upload documents and return answers. A usable service must define failure behavior. Unsupported formats, insufficient permissions, absent evidence, conflicting evidence, and temporary model unavailability should produce distinct states instead of one blank response or an invented answer. Interfaces should include a request identifier, knowledge version, citation list, and observable processing stages. Users can then decide whether to retry, add material, or seek human verification. Internal logs may be detailed, while user explanations should focus on the task rather than implementation terms that do not help decisions.
Acceptance testing should cover an ordinary query, an unanswerable query, a version conflict, permission isolation, and resource deletion. Each case examines what the user ultimately sees rather than only whether one module succeeds. Operational measures include error types, evidence coverage, latency, and cost, with attention to slow requests hidden by averages. Service contracts also define who owns source data, reviews entity merges, and handles corrections. Architecture becomes sustainable only when connected to these responsibilities. The reference diagram therefore includes both data flow and feedback: the system transforms documents into answers and returns discovered problems to source and knowledge maintenance processes.