RAG generate¶

Generation takes the retrieved (and postprocessed) []*rag.SearchResult, assembles a prompt, and calls a ChatModel to produce the final answer. The package defines several context-assembly strategies (synthesis modes), a citation-aware variant, and a Helix processor that wraps it all up. Every synthesizer implements:

type Synthesizer interface {
    Synthesize(ctx, query string, results []*rag.SearchResult) (string, error)
}

The DSL processor rag-generate looks up a synthesizer by mode via generate.NewSynthesizer(mode, model). There is also an AugmentProcessor that just adds a context block to the prompt without calling the LLM (PromptTemplate, ContextAssembler).

Synthesis modes¶

compact — stuff as much context as fits, single call¶

rag:
  generate:
    type: compact
    model: ${LLM_REF}
    max_context_chars: 8000

The default. Concatenates retrieved chunks up to the budget, calls the LLM once. There are two implementations: CompactSynthesizer (synthesizer.go) and the configurable ConfigurableCompactSynthesizer (compact.go) that exposes the budget. Source: pkg/rag/generate/synthesizer.go, pkg/rag/generate/compact.go.

refine — iterative refinement¶

rag:
  generate:
    type: refine
    model: ${LLM_REF}

For each chunk in turn: ask the LLM to refine its previous answer given the new chunk. O(N) LLM calls. Use when every chunk contains some additional context worth integrating. Source: pkg/rag/generate/synthesizer.go (RefineSynthesizer), pkg/rag/generate/refine.go (ConfigurableRefineSynthesizer).

tree-summarize — hierarchical summarisation¶

rag:
  generate:
    type: tree-summarize
    model: ${LLM_REF}

Recursively summarises groups of chunks pairwise until one answer remains. O(N log N) LLM calls but parallelisable. Use when retrieval returns many roughly-equal-relevance chunks (summarisation tasks). Source: pkg/rag/generate/synthesizer.go (TreeSummarizeSynthesizer), pkg/rag/generate/tree_summarize.go (ConfigurableTreeSummarizer).

simple-summarize — one-shot summary call¶

rag:
  generate:
    type: simple-summarize
    model: ${LLM_REF}

Concatenate all context, ask for a single summary. Cheaper than compact for pure summarisation tasks. Source: pkg/rag/generate/synthesizer.go (SimpleSummarizeSynthesizer).

accumulate — answer per chunk, then concatenate¶

rag:
  generate:
    type: accumulate
    model: ${LLM_REF}

For each chunk, generate a per-chunk answer; concatenate them as the final response. Useful when chunks come from distinct sources and you want one answer per source. Source: pkg/rag/generate/synthesizer.go (AccumulateSynthesizer), pkg/rag/generate/accumulate.go (ConfigurableAccumulateSynthesizer).

compact-refine — compact, then refine on remaining chunks¶

rag:
  generate:
    type: compact-refine
    model: ${LLM_REF}

compact for the first batch (up to budget), then refine over the rest. Use for large result sets where you want a strong first answer plus integration of overflow chunks. Source: pkg/rag/generate/synthesizer.go (CompactRefineSynthesizer).

context-only — emit raw context, no LLM call¶

rag:
  generate:
    type: context-only

No model required. Returns the assembled context as the answer. Used for testing or pipelines that hand off to a downstream LLM step. Source: pkg/rag/generate/synthesizer.go (ContextOnlySynthesizer).

generation-only — LLM without retrieved context¶

rag:
  generate:
    type: generation-only
    model: ${LLM_REF}

Calls the LLM on the query alone. The control for A/B testing whether retrieval helps — same generation pipeline, no RAG. Source: pkg/rag/generate/synthesizer.go (GenerationOnlySynthesizer).

citation — compact + numbered sources + inline [N] markers¶

rag:
  generate:
    type: citation
    model: ${LLM_REF}
    style: numeric

Like compact but each chunk is prefixed [N] in the prompt and the system message instructs the model to cite using those markers. After generation, the citations are parsed into structured Citation records (sentence text + source chunk ID). Use for any answer the user must be able to verify. Source: pkg/rag/generate/citation.go.

Prompt / context utilities¶

AugmentProcessor — add context to an existing prompt¶

rag:
  generate:
    type: augment
    template: |
      Use the following context to answer:
      {context}
      Question: {query}

Doesn't call the LLM itself — wires the assembled context into the exchange so a downstream chat/agent step sees the full prompt. Source: pkg/rag/generate/generate.go (AugmentProcessor, PromptTemplate, ContextAssembler).

Choosing between them¶

Watch for: refine and accumulate make N LLM calls. On 50 results that's 50 sequential generations. Pair with a small top_k postprocessor.

See also¶

• retrieve — produces the []*rag.SearchResult consumed here.
• postprocess — filter / rerank before generation; pair long-context-reorder with compact for long-context LLMs.
• cache — CachingChatModel deduplicates identical Generate calls (huge for refine / tree-summarize).
• eval — measure answer quality (faithfulness, relevancy, semantic similarity).