RAG cache¶

The cache sub-package eliminates redundant work in the RAG pipeline: re-parsing the same document, re-embedding the same chunks, and especially re-summarising the same cluster of text with the same LLM call. The most consequential consumer is RAPTOR (which summarises overlapping clusters across runs) and GraphRAG (which extracts entities from documents that often recur).

All implementations satisfy:

type Store interface {
    Get(ctx, hash) (*CacheEntry, bool)
    Put(ctx, hash, *CacheEntry) error
    Delete(ctx, hash) error
    Size() int
    Clear()
}

A TransformationCache namespaces keys by a transform name so two caches can share a store without colliding.

Stores¶

in-memory — process-local LRU¶

rag:
  cache:
    type: in-memory
    max_size: 10000

Concurrency-safe via sync.RWMutex. Use for single-process runs, tests, and as the default fallback. Source: pkg/rag/cache/cache.go (InMemoryStore).

redis — Redis-backed shared cache¶

rag:
  cache:
    type: redis
    addr: localhost:6379
    password: ${REDIS_PASSWORD}
    db: 0
    prefix: "helix:cache:"
    ttl: 24h

Native TTL (uses SET EX per key). Size() uses SCAN (so it scans the keyspace each call — call sparingly). Use across multiple processes or to share cached LLM responses with other tools. Source: pkg/rag/cache/redis.go.

Processors¶

CacheProcessor — mark a hit, don't compute¶

rag:
  cache:
    type: probe
    store: ${CACHE_REF}
    transform: chunk

Sets HelixCacheHit and HelixCacheKey on the exchange but does not run the inner pipeline. Use as a probe followed by a choice that routes hits around the rest of the pipeline. Source: pkg/rag/cache/cache.go (CacheProcessor).

SubPipelineCacheProcessor — full wrap-and-replay¶

rag:
  cache:
    type: wrap
    store: ${CACHE_REF}
    transform: summarise
    inner: ${INNER_PROCESSOR_REF}
    key_header: ""        # default: hash full body
    ttl: 1h

Wraps an inner processor so identical inputs replay cached outputs. On hit, the inner pipeline is skipped, the cached body replaces ex.In().Body(), and HelixCacheHit=true plus HelixCacheCreatedAt are set. On miss, the inner runs and the result is cached under the computed key. TTL is enforced at Get time (entries past TTL are deleted in-line). Source: pkg/rag/cache/cache.go (SubPipelineCacheProcessor).

CachingChatModel — memoise LLM Generate calls¶

rag:
  cache:
    type: llm
    store: ${CACHE_REF}
    inner_model: ${LLM_REF}

Wraps any llm.ChatModel so identical Generate calls (same provider + model + messages + resolved options) replay from cache instead of hitting the API again. Cache keys are namespaced by provider:model so swapping models doesn't return another model's cached answers. Only Generate is memoised — Stream is delegated unchanged (streaming responses aren't cached). Tracks Hits() and Misses() for metrics.

This is the cache wrapper that makes RAPTOR cluster summarisation practical: the same cluster (with the same prompt) hits the cache on re-builds, dropping LLM cost to near-zero. Source: pkg/rag/cache/chatmodel.go.

Helpers¶

Source: pkg/rag/cache/cache.go.

Choosing between them¶

In production RAPTOR + GraphRAG pipelines, the typical wiring is:

redis store -> CachingChatModel wraps the LLM -> RAPTOR & GraphRAG see the cached model

This single change can drop a re-run's LLM cost by 90%+.

See also¶

• raptor — the canonical consumer of CachingChatModel.
• graph — entity extraction also benefits from CachingChatModel.
• generate — refine / tree-summarize synthesizers make many LLM calls; cache them.
• ingest — wrap with SubPipelineCacheProcessor to avoid re-parsing unchanged documents.