Most RAG demos index a folder once and call it done. Real source data does not sit still: policies get revised, pages get deleted, products are renamed. The failure that follows is subtle and embarrassing, a chatbot confidently citing a document that was withdrawn two months ago. Index freshness is a pipeline problem, not a database feature.

Why Indexes Go Stale

There are three distinct ways your index drifts from the truth. Content gets edited, so stored chunks no longer match the source. Content gets deleted, and orphaned vectors keep being retrieved. And content gets added without anyone triggering ingestion. Teams usually handle the third case and forget the first two, because additions are visible and the others fail silently.

Permissions drift too. If a document becomes restricted, its chunks in the index may still be returned to people who should no longer see it. That is a security concern as well as a quality one.

Start With Stable Identifiers

Everything else depends on this. Give each source document a stable id, and derive chunk ids from it, for example the document id plus a chunk position. Store the document id, a content hash, a source URL or path, and a last-indexed timestamp as metadata on every chunk. Without a stable id, you cannot find the old chunks to replace, so you end up with duplicates of old and new versions living side by side.

Detecting What Changed

You have three common options, and many systems combine them.

Source-provided change signals. Webhooks, change feeds, or “modified since” queries from the system holding the content. These are efficient and near real time when available.

Hash comparison. On a schedule, fetch or list documents, compute a hash of the normalized content, and compare it to the stored hash. This is reliable and does not depend on the source’s cooperation, but it requires reading everything periodically.

Periodic full reconciliation. Compare the complete list of source ids to the complete list in the index. This is the only approach that reliably finds deletions, so run it even if you use change feeds for everything else.

The Update Procedure

For a changed document, the safe sequence is: re-chunk the new version, embed the new chunks, write them, then remove old chunks for that document id that are not in the new set. Doing delete-then-insert risks a window where the document is missing. Writing the new version first and then cleaning up avoids gaps, though check your database’s current documentation for transactional or atomic-swap support. Some systems let you tag chunks with a version number and filter queries to the latest version, which makes the swap instant.

For deletions, remove all chunks by document id and confirm with a count. For moved or renamed documents, rely on stable ids rather than paths, or treat it as delete plus add.

Sync Strategy Comparison

Approach	Handles edits	Handles deletes	Cost	Notes
Source webhooks or change feed	Yes	Often	Low	Needs source support, can miss events
Scheduled hash comparison	Yes	Only with listing	Medium	Robust, works with any source
Full reconciliation	Yes	Yes	Higher	Run on a slower cadence as a safety net
Full rebuild	Yes	Yes	Highest	Simple, useful after model or chunking changes

Embedding Model Changes Are Their Own Event

If you switch embedding models, old and new vectors are not comparable, so the whole index must be rebuilt. Plan for this by building the new index alongside the old one, validating retrieval quality on your test questions, and then switching traffic. Record the embedding model name and chunking configuration as index metadata so you always know what produced the vectors.

Freshness You Can Measure

Pick a freshness target that matches how the content is used. A pricing FAQ may need updates within minutes; an archive of old reports may be fine weekly. Then measure it: log the time between a source modification and the corresponding index update, and alert when it exceeds your target. Also track the count of source documents versus indexed document ids, because a growing mismatch is an early warning of ingestion failures.

A Hypothetical Example

Consider an internal assistant built on a company’s policy documents. A travel policy is revised: the daily allowance changes and a section is removed. The nightly job uses a content hash, notices the document changed, re-chunks it, and writes the new chunks under the same document id. A weekly reconciliation job discovers that an old policy page was deleted from the wiki three weeks ago; the index still had its chunks, and the job removes them. A user asking about the removed section now gets “I couldn’t find that,” rather than an outdated answer presented as fact.

Operational Details Worth Getting Right

Make ingestion idempotent, so rerunning a failed job does not create duplicates. Keep a dead-letter list of documents that failed to parse, and review it rather than letting failures vanish. Rate-limit embedding calls and batch them, but check the provider’s current documentation for limits. Finally, expose the source link and last-updated date in answers where possible; users then have a way to notice staleness themselves.

Frequently Asked Questions

How often should I re-index? Match it to how fast the content changes and how costly stale answers are. Event-driven updates for volatile content, nightly or weekly jobs for stable content.

Do I have to re-embed the whole document when one paragraph changes? Often you can re-embed only the chunks whose content hash changed, if chunk boundaries are stable. If an edit shifts every boundary, re-chunking the whole document is simpler.

How do I find orphaned vectors? List distinct document ids in the index and compare them against the source system’s current ids. Anything in the index but not the source is an orphan.

What about access control changes? Store permission metadata on chunks and filter at query time, and make sure permission changes trigger a metadata update, not just content changes.

Is a full rebuild ever the right answer? Yes, after changing the embedding model, chunking strategy, or when the index has accumulated unexplained inconsistencies and the corpus is small enough to rebuild affordably.

Next Step

Add a document id, content hash, and last-indexed timestamp to every chunk if they are not already there, then write a small script that counts source ids missing from the index and index ids missing from the source. Run it once and see what it finds.