AI Context & Retrieval

AI Context & Retrieval is the advanced governance and diagnostics surface for AI knowledge use. It is where operators review approved context sources, indexing evidence, retrieval policies, chunks, embeddings, vector posture, OCR posture, and retrieval preview evidence.

For the normal user/admin workflow, start from Knowledge Bases. A Knowledge Base is the named reusable collection that Copilot users select. AI Context is the lower-level evidence behind those collections.

The page manages the governance ledger around context. It does not make AI the owner of documents, catalog records, external repositories, register data, or other source truth. Source modules still own the original content, access rights, retention, classification, lifecycle state, and deletion behavior.

What You Can Manage

Record	Use it for
Context source	Register an approved source reference, its source type, rights summary, classification summary, retention summary, and current governance status.
Context chunk	Review indexed context evidence as safe references and summaries. Chunks are evidence of what was prepared for retrieval, not source truth.
Index run	Review indexing attempts, outcomes, quality notes, failure evidence, and provenance.
Embedding	Review searchable-index lineage for prepared chunks, including backend kind, model, dimension, source lineage, and lifecycle status. Embedding records do not expose vector values.
Vector status	Review whether the deployment has a semantic retrieval backend, chunk embedding provider, query embedding provider, and vector index posture configured. The status does not expose endpoints, secrets, vector payloads, or backend table details.
Retrieval policy	Define whether a source or source class may be used for indexing, retrieval, prompt assembly, quotation, drafting, or generation.
Retrieval preview dossier	Create a draft decision dossier from retrieval-authorized source refs so reviewers can inspect citations, coverage, and decision posture before any owning workflow publishes final truth.

For day-to-day document work, start from the document. Upload or select the DMS document version, run Content Extraction, then use the document-side Index for AI retrieval action to hand the extracted source to AI Context. Build or update the user-facing collection from Knowledge Bases. AI Context remains the governance and diagnostics console for context sources, chunks, index runs, embeddings, policies, and retrieval preview.

AI Context can also add AI handoff-ready extracted sources and create retrieval policies through guided forms. Choose the governed AI capability, governed purpose, classification, access decision, and allowed knowledge uses from the available lists; Novantra generates the required governance snapshots behind the scenes. Manual source registration and raw policy details remain advanced administration options. Chunks, index runs, and embeddings are evidence because they are produced by approved indexing workflows and source connectors. Retrieval previews can also create draft decision dossier candidates from allowed sources. Those dossiers cite source refs and policy decisions; they do not copy raw source text.

Content Extraction is the source-preparation foundation that can provide safe source refs, snippets, anchors, extraction posture, preview posture, OCR posture, and AI handoff readiness to retrieval workflows. AI Context & Retrieval governs whether those prepared refs may be indexed, retrieved, cited, or assembled into prompts; it does not become the owner of the original source content.

How To Use It

Upload or select a document version in DMS.
Run Content Extraction and confirm the source is ready and available for AI handoff.
Choose Index for AI retrieval from the document’s Content Extraction panel.
Select the governed AI capability when more than one active capability exists.
Record the reason for making this version available to governed retrieval.
Review the resulting context source, chunks, and index-run evidence in AI Context.
Create or review retrieval policies before relying on the source for prompt assembly, citations, drafting, or generation. The normal policy form asks for a policy name, governed AI capability, governed purpose, default access decision, and allowed knowledge uses.
Review embedding lineage where semantic retrieval is enabled. Missing or stale embeddings mean semantic retrieval is not ready even if keyword search is available.
Review vector status before relying on semantic retrieval or RAG behavior in a cloud or on-prem profile.
Use retrieval preview to confirm which sources are allowed or blocked for a question, governed AI capability, retrieval policy, and use kind.
Create a draft decision dossier from allowed preview results when reviewers need a governed citation set for decision review, coverage analysis, or drafting follow-up.
Archive sources and policies that should no longer be used.

If a source is blocked, retired, out of scope, expired, reclassified, or no longer accessible to the actor, it should not be used for embeddings, ranking, prompt assembly, dossiers, suggestions, exports, or downstream AI actions.

Source Ownership Boundaries

Context records should contain references and sanitized summaries only. Do not paste raw document text, source extracts, credentials, tokens, storage paths, private comments, or provider payloads into context source notes or policy descriptions.

The owning source module remains responsible for:

document or record content;
permissions and resource access;
classification and retention;
source version and lifecycle status;
source-rights and export restrictions;
deletion, reclassification, and source retirement.

AI retrieval must consume those decisions. It should not create a second local copy of source truth.

Retrieval Policy Checks

Before content is used, the retrieval path should verify the actor and purpose against the relevant authorization and source policy. In practice this means checking:

the AI use authorization and action policy;
provider and profile posture;
actor access to the source;
classification, retention, export, and source-rights limits;
source lifecycle, version, and retirement state;
whether the requested use kind is allowed.

Only content that passes these checks should be eligible for ranking, prompt assembly, citations, dossiers, or suggestions.

Embedding And Vector Posture

Embedding records are governed lineage records. They show which context chunk was prepared for semantic retrieval, which approved backend kind was used, which model and dimension were recorded, and whether the embedding is active, stale, blocked, retired, failed, or archived.

They are not a source-text store and they do not expose vector values, pgvector row identifiers, backend table names, prompts, provider payloads, or raw extracted content. pgvector or another approved vector backend is required only for semantic retrieval or RAG-enabled profiles. Keyword search, extraction readiness, and no-AI deployments do not require vector support.

If a source changes, is reclassified, becomes legally held, loses access rights, is deleted, or moves to a restricted retrieval policy, the embedding lifecycle should move to stale, blocked, or retired through source-owner and AI retrieval lifecycle handling before it is used again.

Content Extraction lifecycle events are one source of those transitions. AI subscribes to them through its feature runtime and updates embedding lineage through AI-owned repositories, keeping vector lifecycle ownership inside AI while source modules continue to own source truth and access.

When Content Extraction reports that a source has been extracted and is AI-handoff ready, AI can create or update the related context source, context chunks, index-run evidence, and chunk embeddings through the same governed indexing path. Corpus indexing requires a tenant-approved provider model with the embeddings capability, allowed context_embedding use, source-classification permission, a validated vector size, and a vector backend. Retrieval preview also requires a tenant-approved query embedding provider model with the embeddings capability, allowed context_retrieval use, compatible vector size, and classification permission for the candidate sources, so a source can be indexed while the preview still shows a degraded retrieval posture if query-time governance or configuration is missing.

Vector payloads, where a deployment stores them for retrieval, remain behind AI-owned backend seams. Small controlled deployments may use a local exact-scan backend; larger or RAG-heavy deployments may use pgvector or another approved backend. Source modules, Document Intelligence, and docs users should see only capability states, source refs, chunk refs, scores, and sanitized evidence.

The Vector status view reports semantic retrieval posture: configured backend kind, index strategy, pgvector extension and index posture where relevant, tenant-approved chunk and query embedding model readiness, dimension compatibility, and whether the profile is currently configured, unavailable, or misconfigured. It is a readiness view, not a vector store browser.

For pgvector profiles, operators should check all readiness rows together: the database extension must be known and installed, HNSW or IVFFlat indexes must be ready when that strategy is selected, tenant admins must approve and validate embedding-capable models for corpus/index builds and retrieval preview, dimensions must be compatible, and either the built-in native pgvector store for the selected profile or an approved external ranker must be available. Built-in semantic-rag profiles inspect this readiness through sanitized AI-owned catalog probes, but the UI does not expose table names, index names, vector rows, or database connection details. If any readiness item is missing, semantic retrieval should show a degraded capability state instead of returning ungrounded results.

Cloud and on-prem semantic-rag deployment profiles apply the optional AI pgvector tenant migration ledger. Default AI profiles, keyword-only profiles, and no-AI profiles do not receive pgvector DDL. The native table and indexes remain AI-owned backend implementation details; source modules and public contracts continue to see only source refs, chunk refs, capability states, and sanitized retrieval evidence.

Retrieval preview can evaluate a semantic query when a vector backend is available and tenant admins have approved a compatible query embedding model in AI Providers. Building or refreshing the vector corpus from governed chunks also requires an approved chunk embedding model; if the model is missing, suspended, lacks embeddings, has no validated vector size, is blocked for the source classification, or the data-boundary policy blocks the handoff, indexing records blocked evidence and does not send chunk text to the provider. Query-time retrieval follows the same fail-closed pattern before query text reaches the embedding provider or vector backend ranking. Operators still configure vector infrastructure, but tenant model choice is managed in the app. The preview still applies source-owner access, AI retrieval policy, and the data-boundary decision before ranking any candidate chunks. If no semantic query is supplied, no eligible vectors exist, a source is blocked, an embedding provider model is not approved, a data-boundary decision blocks the query, dimensions are incompatible, or the vector backend is unavailable, the page shows a capability state instead of silently falling back to ungoverned ranking.

The preview also includes an internal evaluation summary for reviewers and operators. It shows source coverage, ranking coverage, score bands, source-module coverage, language coverage, and tuning hints such as missing policy coverage, blocked sources, stale embeddings, unavailable vector capability, or ranked results below the requested semantic limit. The summary uses counts, scores, capability states, and safe refs only. It does not return the semantic query text, vector values, backend row identifiers, provider payloads, or raw source text.

Ranked preview results expose source refs, chunk refs, source labels, scores, and sanitized evidence keys only. They do not expose embedding values, pgvector row identifiers, backend table names, storage keys, provider payloads, or private source text.

Retrieval-Backed Dossiers

The retrieval preview can create a draft AI decision dossier from the currently evaluated sources. The dossier includes allowed context-source refs, retrieval policy refs, decision counts, source-module counts, blocker-code counts, and evidence-snapshot key names. It is a candidate review artifact.

The dossier does not contain raw chunk text, source extracts, prompts, provider responses, private comments, or final policy/control/evidence decisions. If the dossier leads to a policy, procedure, control, evidence requirement, lifecycle review, or other operational record, that final record must still be created by the owning module and its approval workflow.

Cloud, Sovereign, And No-AI Profiles

Cloud and Sovereign AI-enabled deployments include the Context & Retrieval page as part of the optional AI feature bundle. Provider selection, embedding behavior, source connectors, and vector capability depend on the deployment configuration and customer-approved providers.

Cloud and on-prem AI-enabled profiles use the same product contracts and capability states. The configured backend can differ by deployment, but missing, disabled, stale, or blocked vector capability must be visible before users rely on semantic retrieval.

Both deployment modes use the same extraction-to-AI indexing contract. Cloud can use managed runtime configuration and on-prem can use install-local or customer-hosted configuration, but neither mode should bypass source-owner access, AI retrieval policy, embedding lineage, or sanitized capability reporting.

The same Vector status posture is available in both deployment modes for AI-enabled profiles. Cloud may report managed service configuration, while on-prem may report install-local or customer-hosted configuration, but both must surface missing chunk embedding providers, missing query embedding providers, missing pgvector support where relevant, and unavailable semantic retrieval as visible readiness states.

For small controlled corpora, an AI-enabled deployment may use the local exact-scan backend with encrypted AI-owned vector payload storage. That backend is usable only when the deployment has also registered approved chunk and query embedding providers. Larger semantic retrieval or RAG profiles should configure pgvector or another approved backend through the same capability model.

When a deployment selects the built-in semantic-rag profile, cloud and on-prem use the same pgvector capability key, migration contribution, native AI-owned pgvector ranking path, and sanitized extension/index readiness probe. Operators still need to configure approved chunk and query embedding providers before semantic retrieval becomes available.

No-AI Sovereign profiles physically exclude the AI feature bundle, including Context & Retrieval routes and runtime code. Customers moving from no-AI to AI-enabled deployments should use the supported upgrade path, configure providers and authorizations, and register sources before production use.