AI Agents Guide¶

Build intelligent, tool-using AI agents with Helix. This guide covers all 7 agent types, 17 LLM providers, tool calling, 8 memory backends, multi-agent orchestration, and agent flows — all configurable via declarative YAML or Go code.

Table of Contents¶

1. Overview
2. Agent Types (Deep Dive)
3. LLM Providers
4. Memory Configuration
5. Tools — incl. middleware, custom orchestration, checkpointing, async interrupts
6. Governance — kill switches, rings, policy engine, audit chain
7. Agent Flows
8. Exposing Agents via Protocols
9. RAG-Augmented Agents
10. Advanced Patterns
11. Production Deployment
12. Complete Examples
13. Strict YAML Validation
14. Human-in-the-Loop
15. Agent Memory with MemX Backend
16. Multimodal Agents
17. Built-in Function Registry
18. SmartMemory -- Token-Managed Memory
19. Playwright MCP Browser Tools

1. Overview¶

What Are Helix Agents?¶

Helix agents are AI-powered processing units that live inside your integration pipelines. Unlike standalone agent frameworks, Helix agents are first-class integration components: they receive messages from Kafka, respond to HTTP requests, process database events, and route results to downstream systems -- all within the same workflow definition.

An agent in Helix combines four capabilities:

• LLM reasoning via pluggable providers (OpenAI, Anthropic, Ollama)
• Tool use for interacting with external systems during reasoning
• Memory for maintaining conversational and long-term context
• Orchestration patterns for multi-agent coordination

Top-Level agents: Primitive¶

Agents can be defined as top-level agents: primitives in YAML, not just as inline steps inside workflows:. The top-level syntax is simpler and AI-first:

# Top-level agents: primitive (recommended for standalone agents)
agents:
  - id: assistant
    type: react
    trigger: http-server://0.0.0.0:8080/chat
    model:
      provider: openai
      model: gpt-4o
    system_prompt: "You are a helpful assistant."
    tools:
      - name: search
        type: function
        description: "Search the knowledge base"
    memory:
      enabled: true
      type: sliding-window
      window_size: 20

Compare with the equivalent inline syntax inside a route:

# Inline agent step inside workflows: (use when the agent is part of a larger pipeline)
workflows:
  - id: chat-route
    trigger:
      uri: http-server://0.0.0.0:8080/chat
    steps:
      - agent:
          id: assistant
          type: react
          model:
            provider: openai
            model: gpt-4o
          system_prompt: "You are a helpful assistant."

Use agents: when the agent IS the endpoint. Use inline agent: steps when the agent is one step in a larger integration pipeline (e.g., consuming from Kafka, then processing with an agent, then routing to another system).

Architecture¶

                    +------------------+
                    |   Exchange       |
                    |  (Message Body)  |
                    +--------+---------+
                             |
                    +--------v---------+
                    |   Agent Step     |
                    |  (agent://id)    |
                    +--------+---------+
                             |
              +--------------+--------------+
              |              |              |
     +--------v---+  +------v-----+  +-----v------+
     | LLM Provider|  |   Tools    |  |   Memory   |
     | (OpenAI,   |  | (FuncTool, |  | (Sliding,  |
     |  Anthropic,|  |  MCP,      |  |  Vector,   |
     |  Ollama)   |  |  SimpleTool)|  |  SQL, ...) |
     +--------+---+  +------+-----+  +-----+------+
              |              |              |
              +--------------+--------------+
                             |
                    +--------v---------+
                    |   Agent Output   |
                    |  (set on Exchange|
                    |   body/headers)  |
                    +------------------+
                             |
              +--------------+--------------+-----------+
              |              |              |           |
     +--------v---+  +------v-----+  +-----v----+ +---v-----+
     |   AG-UI    |  |    A2A     |  |   A2UI   | |   MCP   |
     | (Frontend  |  | (Agent-to- |  | (Rich UI)| | (Tool   |
     |  Streaming)|  |  Agent)    |  |          | |  Export) |
     +------------+  +------------+  +----------+ +---------+

When to Use Which Agent Type¶

2. Agent Types (Deep Dive)¶

2.1 ReAct Agent¶

What it does: Implements the ReAct (Reason + Act) loop -- the most widely used agent pattern. The agent reasons about what to do, optionally calls tools, observes the results, and repeats until it has a final answer.

How it works internally: 1. Loads conversation history from memory 2. Appends the user message 3. Sends messages + tool definitions to the LLM 4. If the LLM returns tool calls, executes them and appends results 5. Loops back to step 3 (up to max_iterations) 6. If no tool calls, the LLM response is the final answer

YAML example:

agents:
  - id: my-assistant
    type: react
    trigger: direct://assistant
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    system_prompt: |
      You are a helpful assistant with access to tools.
      Always verify information before responding.
    memory:
      enabled: true
      type: sliding-window
      window_size: 20
    max_iterations: 15
    tools:
      - name: search_web
        type: function
        description: "Search the web for current information"
        parameters:
          type: object
          properties:
            query:
              type: string
              description: "Search query"
          required: ["query"]

Key parameters: | Parameter | Default | Description | |---|---|---| | max_iterations | 10 | Maximum ReAct loop iterations before failing | | system_prompt | (empty) | System message prepended to all conversations | | memory.type | conversation | Memory backend to use |

Use cases: Customer support chatbots, research assistants, code generation agents, data analysis assistants.

2.2 Supervisor Agent¶

What it does: Routes incoming requests to the most appropriate specialist sub-agent. The supervisor uses the LLM to analyze the request and decide which sub-agent should handle it.

How it works internally: 1. Presents the LLM with available sub-agent IDs and the user request 2. Asks the LLM to respond with JSON: {"agent": "<id>", "message": "<refined message>"} 3. Parses the routing decision and delegates to the chosen sub-agent 4. Falls back to the first sub-agent if JSON parsing fails

YAML example:

agents:
  - id: support-supervisor
    type: supervisor
    trigger: direct://support
    model:
      provider: anthropic
      model: claude-sonnet-4-20250514
      api_key: "${ANTHROPIC_API_KEY}"
    system_prompt: |
      You are a customer support supervisor. Route requests to the
      appropriate specialist agent based on the customer's needs.
      - billing-agent: payment issues, invoices, refunds
      - technical-agent: bugs, errors, configuration problems
      - sales-agent: pricing, plans, upgrades
    sub_agents:
      - id: billing-agent
        type: react
        model:
          provider: anthropic
          model: claude-sonnet-4-20250514
          api_key: "${ANTHROPIC_API_KEY}"
        system_prompt: "You handle billing and payment inquiries."
        max_iterations: 5
      - id: technical-agent
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "You are a technical support specialist."
        max_iterations: 10
        tools:
          - name: query_logs
            type: function
            description: "Query application logs"
            parameters:
              type: object
              properties:
                service:
                  type: string
                timerange:
                  type: string
              required: ["service"]
      - id: sales-agent
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "You are a friendly sales specialist."
        max_iterations: 5

Key parameters: | Parameter | Default | Description | |---|---|---| | sub_agents | (required) | List of specialist sub-agents | | system_prompt | (empty) | Instructions for the routing decision |

Use cases: Customer support triage, multi-department helpdesk, domain-specific routing.

2.3 Plan-Execute Agent¶

What it does: Separates planning from execution. First asks the LLM to create a step-by-step plan (JSON array), then executes each step sequentially using the ReAct pattern, passing accumulated context forward.

How it works internally: 1. Asks the LLM to create a plan: [{"step": 1, "action": "description"}, ...] 2. For each step, runs a ReAct loop with the step description + accumulated results 3. Returns the combined results from all steps

YAML example:

agents:
  - id: research-agent
    type: plan-execute
    trigger: direct://research
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    system_prompt: |
      You are a research analyst. When given a research topic,
      create a structured plan and execute each step thoroughly.
    max_iterations: 20
    tools:
      - name: search_papers
        type: function
        description: "Search academic papers"
        parameters:
          type: object
          properties:
            query:
              type: string
            limit:
              type: integer
          required: ["query"]
      - name: summarize_paper
        type: function
        description: "Get and summarize a specific paper"
        parameters:
          type: object
          properties:
            paper_id:
              type: string
          required: ["paper_id"]

Key parameters: | Parameter | Default | Description | |---|---|---| | max_iterations | 10 | Max ReAct iterations per step (not total) | | system_prompt | (empty) | Instructions for both planning and execution |

Use cases: Research tasks, report generation, multi-step data analysis, complex problem solving.

2.4 Deep Agent¶

What it does: Performs hierarchical task decomposition. Recursively breaks complex tasks into subtasks, solves leaf tasks via ReAct, and synthesizes results back up the tree.

How it works internally: 1. Asks the LLM whether the task can be solved directly or needs decomposition 2. If direct: solves with ReAct 3. If decomposition needed: LLM returns {"direct": false, "subtasks": ["task1", "task2", ...]} 4. Recursively decomposes each subtask (up to deep_max_depth) 5. At max depth, solves directly with ReAct 6. After all subtasks complete, synthesizes a final answer from combined results

YAML example:

agents:
  - id: deep-analyst
    type: deep
    trigger: direct://analyze
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    system_prompt: |
      You are a deep analysis agent. Break complex problems into
      manageable subtasks and solve them systematically.
    max_iterations: 10
    deep_max_depth: 4
    tools:
      - name: query_database
        type: function
        description: "Run a SQL query against the analytics database"
        parameters:
          type: object
          properties:
            sql:
              type: string
          required: ["sql"]

Key parameters: | Parameter | Default | Description | |---|---|---| | deep_max_depth | 3 | Maximum recursion depth for task decomposition | | max_iterations | 10 | Max ReAct iterations at each leaf node |

Use cases: Complex data analysis, competitive landscape assessment, architectural reviews, multi-faceted research.

2.5 Sequential Agent¶

What it does: Runs sub-agents in sequence, passing each agent's output as the next agent's input. This creates a processing pipeline where each agent refines or transforms the previous result.

How it works internally: 1. Runs the first sub-agent with the original user message 2. Takes the response content and passes it as the input to the next sub-agent 3. Repeats until all sub-agents have processed 4. Returns the last sub-agent's response

YAML example:

agents:
  - id: content-pipeline
    type: sequential
    trigger: direct://create-content
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    sub_agents:
      - id: researcher
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "Research the given topic and provide key facts and data points."
        max_iterations: 5
      - id: writer
        type: react
        model:
          provider: anthropic
          model: claude-sonnet-4-20250514
          api_key: "${ANTHROPIC_API_KEY}"
        system_prompt: "Write a well-structured article based on the research provided."
        max_iterations: 3
      - id: editor
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "Edit the article for clarity, grammar, and style. Output the final version."
        max_iterations: 3

Use cases: Content creation pipelines, code review chains, translation + localization, data enrichment.

2.6 Parallel Agent¶

What it does: Runs all sub-agents concurrently with the same input, collects results, and combines them. Each sub-agent processes independently in its own goroutine.

How it works internally: 1. Launches all sub-agents concurrently (one goroutine per sub-agent) 2. Each sub-agent receives the same user message 3. Collects all responses (errors are logged but do not block other agents) 4. Combines results as [agent-id]: response\n\n for each agent 5. Returns the combined response

YAML example:

agents:
  - id: perspectives
    type: parallel
    trigger: direct://analyze
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    sub_agents:
      - id: technical-reviewer
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "Analyze this from a technical feasibility perspective."
      - id: business-reviewer
        type: react
        model:
          provider: anthropic
          model: claude-sonnet-4-20250514
          api_key: "${ANTHROPIC_API_KEY}"
        system_prompt: "Analyze this from a business impact perspective."
      - id: risk-reviewer
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "Analyze this from a risk and compliance perspective."

Use cases: Multi-perspective analysis, ensemble responses, parallel data processing, A/B response testing.

parallel_error_policy controls what happens when one sub-agent fails:

agents:
  - id: perspectives
    type: parallel
    parallel_error_policy: fail_fast
    sub_agents: [ ... ]

Failures always surface as delegate_error events on OnEvent regardless of policy, so dashboards see every miss.

2.7 Loop Agent¶

What it does: Iteratively refines a response until the agent signals completion (by including "DONE", "COMPLETE", or "FINISHED" in its output) or reaches max_iterations.

How it works internally: 1. Runs a ReAct loop with the user message 2. Checks the response for completion keywords (case-insensitive) 3. If not complete, prepends "Previous iteration result:" and asks the agent to continue refining 4. Repeats up to max_iterations

YAML example:

agents:
  - id: refinement-agent
    type: loop
    trigger: direct://refine
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    system_prompt: |
      You are a code quality agent. Review and iteratively improve
      the given code. When you are satisfied with the quality,
      include "DONE" in your response along with the final code.
    max_iterations: 5

Completion signals: The agent must include one of these keywords (case-insensitive) in its response to stop iterating: DONE, COMPLETE, FINISHED.

Key parameters: | Parameter | Default | Description | |---|---|---| | max_iterations | 10 | Maximum refinement iterations |

Use cases: Code quality improvement, essay refinement, iterative problem solving, self-correcting agents.

3. LLM Providers¶

Helix supports 17 LLM providers (plus a gemini alias for Vertex), all implementing the ChatModel interface with both synchronous and streaming modes. The three quick-start examples below cover the most common cases; the full provider list with auth and endpoint details lives in the dedicated LLM & Embedding Providers guide.

Quick reference:

Reasoning, finish-reason, and audio I/O are uniform across the chat-completions providers — see Reasoning, FinishReason & Audio I/O for WithReasoningEffort, WithEnableThinking, WithAudioOutput, and the Message.FinishReason / Message.ReasoningContent surfaces.

3.1 OpenAI¶

API endpoint: POST {base_url}/chat/completions Authentication: Authorization: Bearer {api_key} Default base URL: https://api.openai.com/v1 HTTP timeout: 120 seconds

model:
  provider: openai
  model: gpt-4o                          # Required: model identifier
  api_key: "${OPENAI_API_KEY}"           # Required: API key (env var recommended)
  base_url: "https://api.openai.com/v1"  # Optional: custom endpoint (Azure, vLLM, etc.)
  temperature: 0.7                        # Optional: 0.0 - 2.0 (default: provider default)
  max_tokens: 4096                        # Optional: max output tokens
  top_p: 0.9                             # Optional: nucleus sampling
  stop: ["END", "STOP"]                  # Optional: stop sequences
  response_format: "json_object"          # Optional: "text" or "json_object"

Recommended models: | Model | Best For | Cost | |---|---|---| | gpt-4o | General purpose, tool use | Medium | | gpt-4o-mini | Fast responses, simple tasks | Low | | o3 | Complex reasoning | High |

Compatible services (via base_url): Azure OpenAI, vLLM, LiteLLM, Groq, Together AI, any OpenAI-compatible API.

3.2 Anthropic¶

API endpoint: POST {base_url}/v1/messages Authentication: x-api-key: {api_key} + anthropic-version: 2023-06-01 Default base URL: https://api.anthropic.com HTTP timeout: 120 seconds Default max_tokens: 4096

model:
  provider: anthropic
  model: claude-sonnet-4-20250514                      # Required: model identifier
  api_key: "${ANTHROPIC_API_KEY}"         # Required: API key
  base_url: "https://api.anthropic.com"   # Optional: custom endpoint
  max_tokens: 8192                        # Optional: default 4096
  temperature: 0.7                        # Optional
  top_p: 0.9                             # Optional

Recommended models: | Model | Best For | Cost | |---|---|---| | claude-sonnet-4-20250514 | General purpose, tool use | Medium | | claude-opus-4-20250514 | Complex reasoning, analysis | High | | claude-haiku-3-20250414 | Fast responses, simple tasks | Low |

Note: Anthropic uses a different message format internally. The system prompt is sent as a top-level system field (not a message). Tool results are sent as user messages with tool_result content blocks. Helix handles all format translation automatically.

3.3 Ollama (Local)¶

API endpoint: POST {base_url}/api/chat Authentication: None (local inference) Default base URL: http://localhost:11434 HTTP timeout: 300 seconds (local models can be slow on first load) Streaming format: Newline-delimited JSON (not SSE)

model:
  provider: ollama
  model: llama3.1:8b                       # Required: model name:tag
  base_url: "http://localhost:11434"        # Optional: custom Ollama endpoint
  temperature: 0.7                          # Optional
  max_tokens: 2048                          # Optional: maps to num_predict
  top_p: 0.9                               # Optional
  stop: ["\n\n"]                            # Optional

Recommended models: | Model | Parameters | Best For | |---|---|---| | llama3.1:8b | 8B | General purpose, fast | | llama3.1:70b | 70B | Complex reasoning | | qwen2.5:7b | 7B | Coding, reasoning | | mistral:7b | 7B | Fast, multilingual |

Prerequisites: Install and run Ollama (ollama serve), then pull models (ollama pull llama3.1:8b).

3.4 API Key Configuration¶

Environment variables (recommended):

model:
  api_key: "${OPENAI_API_KEY}"     # Resolved at load time

Vault integration (for production):

helix:
  config:
    vault:
      addr: "https://vault.example.com"
      token: "${VAULT_TOKEN}"
      path: "secret/data/helix/openai"
      key: "api_key"

Direct (development only -- never commit to source control):

model:
  api_key: "sk-..."

3.5 Streaming vs Non-Streaming¶

All three providers support both modes:

• Non-streaming (default): agent.Run(ctx, message) -- waits for the full response. Used when the exchange does not set HelixStream=true.
• Streaming: runner.QueryStream(ctx, message) -- returns a *llm.StreamReader that yields StreamChunk values. Enabled by setting the exchange header HelixStream to "true". Used by AG-UI, A2A, and A2UI protocol servers.

3.6 Token Usage Tracking¶

All LLM components set token usage on exchange headers after each call:

Use these headers for cost tracking, rate limiting, and monitoring.

4. Memory Configuration¶

Memory determines what conversational context the agent sees. All memory types implement the Memory interface:

type Memory interface {
    Add(ctx context.Context, message *llm.Message) error
    Get(ctx context.Context) ([]*llm.Message, error)
    Search(ctx context.Context, query string, limit int) ([]*llm.Message, error)
    Clear(ctx context.Context) error
}

4.1 Conversation Memory¶

What it does: Stores the full, unabridged conversation history. Every message ever added is returned by Get(). Simple and unbounded. In-memory by default, or durable via a store.

When to use: Short conversations, debugging, when you never want to lose context.

steps:
  - agent:
      id: my-agent
      memory:
        enabled: true
        type: conversation

This is the default if no memory is configured. With no store it lives in process memory (lost on restart). To persist the full history, add a store:

      memory:
        enabled: true
        type: conversation
        store: mongodb                       # redis | file | memx | mongodb
        connection: "mongodb://localhost:27017/helix"   # db in the path (default: helix)
        collection: agent_memory             # optional (default: agent_memory)
        session_id: "session-abc-123"

For a SQL database (PostgreSQL/Oracle/SQL Server/MySQL) with the same full-history semantics, use type: sql-long-term.

Caution: Unbounded growth. For long conversations, token limits will eventually be exceeded.

4.2 Sliding Window Memory¶

What it does: Keeps only the last N messages. The system message (first message with role system) is always preserved and does not count toward the window size.

When to use: Chat applications where recent context matters most.

steps:
  - agent:
      id: chat-agent
      memory:
        enabled: true
        type: sliding-window
        window_size: 20

Behavior: When the window is exceeded, the oldest non-system messages are dropped. The system message always occupies one slot.

4.3 Summary Memory¶

What it does: When message count exceeds max_messages, uses the LLM to summarize the oldest half of messages into a running summary. The summary is prepended as a system message on Get(), followed by the most recent messages.

When to use: Long-running conversations where you need both context compression and recent detail.

steps:
  - agent:
      id: long-chat
      memory:
        enabled: true
        type: summary
        max_messages: 30

Note: Summary memory requires an LLM call each time summarization triggers. When configured via YAML, the agent's own model is used for summarization. The prompt is: "Summarize the following conversation concisely, preserving key facts and decisions."

4.4 Token Window Memory¶

What it does: Keeps messages within an estimated token budget. Token count is estimated as len(content) / 4. The system message is always preserved.

When to use: When you want precise control over context window usage to avoid exceeding model limits.

steps:
  - agent:
      id: budget-agent
      memory:
        enabled: true
        type: token-window
        max_tokens: 8000

Behavior: Drops the oldest non-system messages until the total estimated token count fits within the budget.

4.5 Long-Term Memory¶

What it does: Key-value memory with categories, tags, and search. Backed by a Store interface.

When to use: Agents that need to remember facts across sessions (with a durable store).

steps:
  - agent:
      id: persistent-agent
      memory:
        enabled: true
        type: long-term
        store: file           # memory (ephemeral) | redis | file | memx
        file_path: .helix/memory/user-123.json
        namespace: "user-123"

Each stored entry has: ID, Namespace, Content, Category (fact/preference/decision/procedure), Role, Tags, CreatedAt.

4.6 SQL Database Long-Term Memory¶

What it does: Stores all messages persistently in a PostgreSQL (or any database/sql compatible) database. Messages are scoped by session ID. Search uses ILIKE for content matching.

When to use: Production agents that need durable, queryable conversation history.

steps:
  - agent:
      id: production-agent
      memory:
        enabled: true
        type: sql-long-term
        dsn: "${DATABASE_URL}"
        session_id: "session-abc-123"

Table schema (auto-created via EnsureTable):

CREATE TABLE IF NOT EXISTS agent_memory (
    id         SERIAL PRIMARY KEY,
    session_id TEXT NOT NULL,
    role       TEXT NOT NULL,
    content    TEXT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

4.7 Vector Memory¶

What it does: Provides semantic search over past messages using vector embeddings. Messages are embedded and stored in a vector store. Search() finds semantically similar messages. Get() returns an empty slice (vector memory is search-only, not sequential).

When to use: When the agent needs to recall relevant past interactions by meaning rather than recency.

steps:
  - agent:
      id: semantic-agent
      memory:
        enabled: true
        type: vector
        top_k: 5
        store: qdrant                       # durable — persists across restarts
        connection: "http://localhost:6333"
        collection: agent-semantic
        embed:
          provider: openai
          model: text-embedding-3-small
          api_key: "${OPENAI_API_KEY}"

Supported vector stores (dimension is derived from the embedding model, and the collection/table is created automatically):

• memory — in-process, brute-force cosine (default, ephemeral)
• memx — durable, via memx_url (see §14)
• External vector databases (durable): qdrant, pinecone, milvus, pgvector, postgis, chromadb, weaviate, opensearch, elasticsearch, redis — the same backends the RAG pipeline uses, bridged into agent memory. pinecone takes its index host in connection plus api_key (and reuses namespace).

4.8 Composite Memory¶

What it does: Combines multiple memory backends. Add() writes to all backends. Get() and Search() merge results from all backends, deduplicating by content.

When to use: When you need both sequential history (sliding window) and semantic search (vector) simultaneously.

steps:
  - agent:
      id: hybrid-agent
      memory:
        enabled: true
        type: composite
        backends:
          - type: sliding-window
            window_size: 20
          - type: vector
            top_k: 5
            embed:
              provider: openai
              model: text-embedding-3-small
              api_key: "${OPENAI_API_KEY}"

Deduplication: Messages are deduplicated by role:content key across all backends.

4.9 Per-Session Memory Isolation¶

By default, all requests to an agent share a single memory instance. This means User A's conversation leaks into User B's context — a critical problem for any multi-user deployment.

The session_id_header field solves this. When set, Helix extracts the session ID from the specified request header and creates an isolated memory instance per session. Each unique session ID gets its own memory, created on first access and cached for subsequent requests.

Basic Usage¶

agents:
  - id: support-bot
    type: react
    trigger: http-server://0.0.0.0:8080/api/chat
    model:
      provider: openai
      model: gpt-4o
    system_prompt: "You are a helpful support agent."
    memory:
      enabled: true
      type: sliding-window
      window_size: 30
      session_id_header: X-Session-ID

Clients include the session header with each request:

# User Alice
curl -X POST http://localhost:8080/api/chat \
  -H "X-Session-ID: alice-session-001" \
  -d "What's your return policy?"

# User Bob (completely isolated memory)
curl -X POST http://localhost:8080/api/chat \
  -H "X-Session-ID: bob-session-042" \
  -d "How do I reset my password?"

Alice and Bob each have their own conversation history. Neither can see the other's messages.

How It Works¶

1. Request arrives — the agentProcessor reads the header specified by session_id_header
2. Session lookup — the SessionMemoryManager checks its cache (sync.Map) for that session ID
3. First access — if not found, the MemoryFactory creates a new memory instance using the same config (type, window size, DSN, etc.) but scoped to that session
4. Subsequent requests — the cached memory is returned instantly
5. Agent runs — the agent's memory is swapped to the session-scoped instance before Run() executes

If the header is missing from a request, the exchange ID is used as a fallback — effectively giving that request a one-shot memory with no persistence.

The session ID is also set as an exchange header agent.session.id for downstream processors to reference.

With SQL Database Persistence¶

For production deployments where memory must survive restarts, combine session_id_header with sql-long-term:

steps:
  - agent:
      id: production-agent
      type: react
      model:
        provider: anthropic
        model: claude-sonnet-4-20250514
      memory:
        enabled: true
        type: sql-long-term
        dsn: "${DATABASE_URL}"
        session_id_header: X-Session-ID

Each session ID becomes the session_id column value in the agent_memory table. This means:

• Memory persists across server restarts
• Sessions are queryable: SELECT * FROM agent_memory WHERE session_id = 'alice-session-001'
• Cleanup is straightforward: DELETE FROM agent_memory WHERE session_id = '...'

With Vector Memory¶

Semantic search scoped per session:

steps:
  - agent:
      id: research-agent
      memory:
        enabled: true
        type: vector
        top_k: 10
        session_id_header: X-Session-ID
        embed:
          provider: openai
          model: text-embedding-3-small
          api_key: "${OPENAI_API_KEY}"

Each session gets its own vector namespace, so similarity search only returns results from that user's conversation history.

With Composite Memory¶

Combine per-session sliding window with per-session vector search:

steps:
  - agent:
      id: advanced-agent
      memory:
        enabled: true
        type: composite
        session_id_header: X-Session-ID
        backends:
          - type: sliding-window
            window_size: 30
          - type: vector
            top_k: 5
            embed:
              provider: openai
              model: text-embedding-3-small
              api_key: "${OPENAI_API_KEY}"

Both sub-backends are isolated per session.

Session ID Sources¶

The session_id_header can reference any header set by upstream processors or the transport layer:

Configuration Reference¶

When session_id_header is not set, the agent uses a single shared memory instance (legacy behavior). This is fine for single-user CLIs, batch processing, or agents that intentionally share context across all requests.

5. Tools¶

Tools give agents the ability to interact with external systems during their reasoning loop.

5.1 Tool Interface¶

All tools implement:

type Tool interface {
    Info() *ToolInfo      // Name, description, JSON Schema parameters
    Invoke(ctx context.Context, arguments string) (string, error)
}

The arguments string is raw JSON matching the tool's parameter schema. The return value is a string that the LLM sees as the tool result.

5.2 Defining Tools in YAML¶

steps:
  - agent:
      id: my-agent
      tools:
        - name: get_weather
          type: function
          description: "Get current weather for a city"
          parameters:
            type: object
            properties:
              city:
                type: string
                description: "City name"
              units:
                type: string
                enum: ["celsius", "fahrenheit"]
                description: "Temperature units"
            required: ["city"]

5.3 Tool Types Reference¶

Helix agents support 8 tool types. Each type lets an agent interact with a different kind of resource during its reasoning loop.

type: tool -- Reference Top-Level Tools¶

Reference a tool defined in the top-level tools: block. The agent inherits the tool's description, input schema, and output schema.

tools:
  - id: search
    description: "Search knowledge base"
    input:
      query: { type: string, required: true }
    output:
      results: { type: array }
    steps:
      - transform:
          starlang: '"[]"'

agents:
  - id: assistant
    type: react
    trigger: http-server://0.0.0.0:8080/chat
    model:
      provider: openai
      model: gpt-4o
    tools:
      - name: search
        type: tool
        tool_id: search            # references tools.[].id above

RAG Pipelines as Tools (via type: tool)¶

Expose a RAG pipeline to an agent by giving it an id you can reference. A pipeline defaults to direct://<id> as its trigger, so a type: tool reference (which resolves tool:// then direct://) can address it directly.

pipelines:
  - id: knowledge-base
    source: direct://knowledge-base
    steps:
      - rag-chunk: { strategy: semantic }
      - rag-embed: { provider: openai, model: text-embedding-3-small }
      - rag-store: { store: pgvector, connection: "${env:PG_URL}" }
      - rag-retrieve: { strategy: hybrid, top_k: 10 }

agents:
  - id: support-bot
    type: react
    trigger: http-server://0.0.0.0:8080/support
    model:
      provider: anthropic
      model: claude-sonnet-4-20250514
    tools:
      - name: query_docs
        type: tool
        tool_id: knowledge-base       # references the pipeline above
        description: "Search the knowledge base for relevant documents"

type: a2a -- Call External A2A Agents¶

Delegate work to a remote agent that speaks the A2A protocol. This is the correct choice for calling remote agents. For local agents within the same flow, use sub_agents instead.

agents:
  - id: coordinator
    type: supervisor
    trigger: http-server://0.0.0.0:8080/coordinate
    model:
      provider: openai
      model: gpt-4o
    tools:
      - name: research_agent
        type: a2a
        description: "Delegate research tasks to a remote research agent"
        a2a_server:
          url: "http://research-agent.internal:9000"
        auth:
          type: bearer
          token: "${env:A2A_TOKEN}"
      - name: finance_agent
        type: a2a
        description: "Get financial analysis from the finance team agent"
        a2a_server:
          url: "http://finance-agent.internal:9000"
        auth:
          type: oauth2
          token_url: "https://auth.example.com/token"
          client_id: "${env:OAUTH_CLIENT_ID}"
          client_secret: "${env:OAUTH_CLIENT_SECRET}"
          scopes: ["agent:invoke"]

type: mcp -- Call External MCP Tools (with Auth)¶

agents:
  - id: coder
    type: react
    trigger: http-server://0.0.0.0:8080/code
    model:
      provider: anthropic
      model: claude-sonnet-4-20250514
    tools:
      - name: code-tools
        type: mcp
        description: "Code analysis and formatting tools"
        mcp_server:
          url: "http://mcp-tools.internal:3100"
          transport: http
          tools: ["format-code", "lint"]
        auth:
          type: api-key
          header: "X-API-Key"
          key: "${env:MCP_API_KEY}"

type: connector -- Use Any Registered Connector¶

Bind any of the 439 built-in connectors (HTTP REST APIs, SaaS, databases, queues, etc.) as a tool by URI.

agents:
  - id: lookup-bot
    type: react
    trigger: http-server://0.0.0.0:8080/lookup
    model:
      provider: openai
      model: gpt-4o
    tools:
      - name: get_customer
        type: connector
        connector: "http://customer-api.internal:8080/v1/customers"
        description: "Look up a customer by ID"
        parameters:
          type: object
          properties:
            id: { type: string, description: "Customer ID" }
          required: ["id"]

For inline expression logic, define a type: tool referencing a top-level tools: entry whose steps: use a transform: { starlang: ... } step — the canonical [[feedback_tools_route_split]] pattern.

auth: on Tools¶

The auth: block can be added to type: a2a, type: mcp, and type: connector tools to secure outbound calls. Four auth types are supported:

# Bearer token
auth:
  type: bearer
  token: "${env:API_TOKEN}"

# API key in custom header
auth:
  type: api-key
  header: "X-API-Key"
  key: "${env:SERVICE_KEY}"

# OAuth2 client credentials
auth:
  type: oauth2
  token_url: "https://auth.example.com/token"
  client_id: "${env:CLIENT_ID}"
  client_secret: "${env:CLIENT_SECRET}"
  scopes: ["read", "write"]

# Mutual TLS
auth:
  type: mtls
  cert_file: "/certs/client.crt"
  key_file: "/certs/client.key"
  ca_file: "/certs/ca.crt"

All ${env:...} placeholders are resolved at load time via Helix's secret resolution system (environment variables, Vault, AWS Secrets Manager).

The Clear Rule: Local vs Remote¶

5.4 MCP Server Tools¶

Connect an external MCP server as a tool source. All tools from the MCP server are automatically registered with the agent.

steps:
  - agent:
      id: my-agent
      tools:
        - name: external-tools
          type: mcp
          description: "Tools from external MCP server"
          mcp_server:
            url: "http://localhost:3000"
            transport: http            # http, sse, or stdio
            tools: ["search", "fetch"] # Optional: filter specific tools (empty = all)
        - name: local-tools
          type: mcp
          mcp_server:
            transport: stdio
            command: "npx"
            args: ["-y", "@modelcontextprotocol/server-filesystem", "/data"]

MCP server tool parameters:

5.5 FuncTool (Go API)¶

Type-safe tool with automatic JSON deserialization:

type SearchArgs struct {
    Query string `json:"query"`
    Limit int    `json:"limit"`
}

searchTool := tool.NewFuncTool[SearchArgs](
    "search_docs",
    "Search the documentation",
    map[string]any{
        "type": "object",
        "properties": map[string]any{
            "query": map[string]any{"type": "string"},
            "limit": map[string]any{"type": "integer"},
        },
        "required": []string{"query"},
    },
    func(ctx context.Context, args SearchArgs) (string, error) {
        results := doSearch(args.Query, args.Limit)
        return fmt.Sprintf("Found %d results: %v", len(results), results), nil
    },
)

5.6 SimpleTool (Go API)¶

For tools that process raw JSON arguments:

echoTool := tool.NewSimpleTool(
    "echo",
    "Echo the input back",
    map[string]any{"type": "object"},
    func(ctx context.Context, args string) (string, error) {
        return "Echo: " + args, nil
    },
)

5.7 Tool Registry and Middleware¶

registry := tool.NewRegistry()
registry.Register(searchTool)
registry.Register(echoTool)

// Apply middleware to a tool
logged := tool.WithLogging()(searchTool)
registry.Register(logged)

// Use in agent config
agent := adk.New(adk.AgentConfig{
    Tools: registry,
})

5.8 Model & Memory Middleware¶

Both the LLM model and the memory backend can be wrapped with named middleware that the DSL resolves at load time. Use this to add structured logging, redaction, caching, rate-limiting, or tracing without forking the agent core.

agents:
  - id: traced-agent
    type: react
    model: { provider: openai, name: gpt-4o-mini }
    model_middleware: [logging]      # wraps the ChatModel
    memory: { enabled: true, type: sliding-window, max_messages: 20 }
    memory_middleware: [logging]     # wraps the Memory backend

Built-in logging middleware emits structured slog records for every model call (request + response, tokens, finish reason) and every memory op (add / get / search / clear). Register custom middleware from Go via llm.RegisterModelMiddleware(name, factory) and memory.RegisterMiddleware(name, factory); an unknown name in YAML fails the load with the registered set in the error message — no silent fallback.

Middleware list order is outermost-first (the first entry sees raw input, the last entry is closest to the underlying model/memory).

5.9 Custom Agent Orchestration¶

The seven built-in types (react, supervisor, plan-execute, deep, sequential, parallel, loop) cover most patterns. To add your own:

adk.RegisterAgentType("my-orchestrator",
    func(ctx context.Context, a *adk.Agent, input string) (string, error) {
        // … your orchestration logic …
    })

Reference it from YAML as type: my-orchestrator. The agent dispatcher rejects unknown types with the full list of built-ins plus the registered set, so configuration drift surfaces at load time, not at first invocation.

5.10 Checkpointing & Resumable Runs¶

adk.Runner persists interrupt/resume checkpoints. The default in-process MemoryCheckpointStore is fast but loses state on restart; supply a durable backend for production:

import (
    "github.com/arvenza/helix/pkg/agent/adk"
    "github.com/arvenza/helix/pkg/agent/kvstore"
)

kv := kvstore.NewRedis(redisClient)            // or memx, SQL, etc.
runner := adk.NewRunner(adk.RunnerConfig{
    Agent:           agent,
    CheckpointStore: adk.NewKVCheckpointStore(kv, "helix:agent:checkpoint:"),
})

// Run; if a tool calls tool.Interrupt, a checkpoint is saved
result, err := runner.Run(ctx, runID, input)
if ie, ok := adk.AsInterrupt(err); ok {
    // ie.Tool / ie.Info — surface to human; persist the runID
}

// Later, after the human responds:
result, err = runner.Resume(ctx, runID, humanResponse)

The checkpoint shape is JSON-tagged so any KV backend that can hold opaque bytes works.

5.11 Human Input & Async Interrupts¶

Two ways for a tool to ask a human:

Synchronous (RequestHumanInput) — the tool blocks in place until a human responds via the approval queue. Use when the conversation is interactive.

Asynchronous (tool.Interrupt) — the tool returns a typed *tool.InterruptError, the agent aborts cleanly and surfaces a *adk.InterruptError from Run, plus an interrupt event on the OnEvent stream. The caller persists a checkpoint (via CheckpointStore) and resumes later with the human's reply. Use for long-running workflows, ticket-based approvals, or queue-driven flows.

import "github.com/arvenza/helix/pkg/agent/tool"

func (t *escalateTool) Invoke(ctx context.Context, args string) (string, error) {
    if needsHuman(args) {
        return "", tool.Interrupt("escalate to compliance — needs human reviewer")
    }
    return process(args)
}

// Caller side
result, err := agent.Run(ctx, input)
if ie, ok := adk.AsInterrupt(err); ok {
    persistTicket(ie.Tool, ie.Info)   // surface to human queue
    return // resume later via Runner with a CheckpointStore
}

Governance (cross-cuts every tool call)¶

The runtime-governance layer wires deterministic, app-layer guardrails around every tool call: scoped kill switches, 4-tier privilege rings, a rule-based policy engine with Starlang predicates, and a hash-chained tamper-evident audit log. Configure under agent.governance:.

agents:
  - id: finance-agent
    type: react
    model: { provider: openai, name: gpt-4o-mini }
    tools: [transfer, get_balance]

    governance:
      audit_chain: true
      rings:
        assignment: standard
        permissions:
          standard: [get_balance, transfer]
          admin: ["*"]
      policy:
        - action: transfer
          effect: review
          starlang_when: |
            ctx["args"]["amount"] > 10000
        - action: get_balance
          effect: allow
          max_calls: 100
          window: 1m
        - action: "*"
          effect: allow

Evaluation order per tool call: kill switches → privilege rings → policy engine → audit. First non-allow decision wins; the dispatcher feeds a synthetic Tool call denied by governance: <reason> tool_result back to the LLM so it can adjust its plan.

Blocks emit a governance_block AgentEvent:

agentCfg.OnEvent = func(e adk.AgentEvent) {
    if e.Type == adk.EventGovernanceBlock {
        slog.Warn("governance.block",
            "agent", e.AgentID, "tool", e.Tool, "reason", e.Err)
    }
}

Runtime control via the management API: POST /api/kill-switch (operator), GET /api/audit (viewer), POST /api/approvals/{id} (operator). See Governance Guide for the full schema, Go API, conflict resolver, MCP scanner, and prompt-defense classifier.

6. Agent Flows¶

Agent flows are composable processor patterns that implement engine.Processor. They can be nested, combined, and used anywhere a processor is expected -- including inside workflow steps.

6.1 SequentialFlow¶

Runs processors in order. Each processor receives the same exchange (mutations accumulate). Stops at the first error.

flow := flow.NewSequentialFlow(
    agentProcessor,
    validationProcessor,
    formattingProcessor,
)

Exchange behavior: All steps share the same exchange. Step 2 sees the body and headers set by step 1.

6.2 ParallelFlow¶

Runs processors concurrently. Each processor receives a copy of the exchange. Results are aggregated back into the original exchange.

pf := flow.NewParallelFlow(
    technicalReview,
    businessReview,
    riskReview,
)
// Optional: custom aggregation
pf.Aggregator = func(results []*exchange.Exchange, target *exchange.Exchange) {
    var combined map[string]any
    for _, r := range results {
        // merge results...
    }
    target.In().SetBody(combined)
}

Default aggregation: Concatenates all branch bodies with newlines.

6.3 ConditionalFlow¶

Routes to different processors based on a predicate.

cf := flow.NewConditionalFlow(
    func(ctx context.Context, ex *exchange.Exchange) (bool, error) {
        return ex.In().HeaderString("priority") == "high", nil
    },
    urgentProcessor,
).WithElse(normalProcessor)

6.4 LoopFlow¶

Repeats a processor until a condition returns false or max iterations is reached. The condition is evaluated after each iteration.

lf := flow.NewLoopFlow(
    refinementProcessor,
    func(ctx context.Context, ex *exchange.Exchange, iteration int) (bool, error) {
        score := ex.In().HeaderString("quality_score")
        return score != "excellent", nil  // continue if not excellent
    },
    10, // max iterations
)

6.5 PipelineFlow¶

Chains processors where each step gets a fresh exchange seeded with the previous step's output body and headers. Unlike SequentialFlow, steps are isolated.

pf := flow.NewPipelineFlow(
    extractProcessor,   // extracts data -> sets body
    transformProcessor, // transforms body -> sets new body
    loadProcessor,      // loads body into destination
)

Key difference from SequentialFlow: PipelineFlow creates new exchanges between steps (only body and headers carry forward). SequentialFlow shares the same exchange throughout.

6.6 Combining Flows¶

Flows implement engine.Processor, so they compose naturally:

pipeline := flow.NewSequentialFlow(
    flow.NewParallelFlow(researchAgent, dataAgent),
    flow.NewConditionalFlow(isComplex, deepAnalysis).WithElse(quickSummary),
    flow.NewLoopFlow(refinement, qualityCheck, 5),
)

7. Exposing Agents via Protocols¶

Helix supports four standard protocols for exposing agents to external systems. Each is implemented as a native component with its own URI scheme.

7.1 AG-UI (Frontend Streaming)¶

When to use: You want to expose agents to web or mobile frontends with real-time streaming.

Protocol: AG-UI (Agent-User Interaction). Used by CopilotKit and other frontend frameworks. Streams Server-Sent Events (SSE) back to the client.

URI scheme: agui://host:port/path?name=agent-name Default port: 4100

SSE event flow:

RUN_STARTED
TEXT_MESSAGE_START (messageID, role=assistant)
TEXT_MESSAGE_CONTENT (messageID, "Hello")
TEXT_MESSAGE_CONTENT (messageID, " world")
...
TEXT_MESSAGE_END (messageID)
RUN_FINISHED

Complete YAML example:

helix:
  agui:
    servers:
      - enabled: true
        addr: "0.0.0.0:4100"
        path: "/agent"
        name: "my-assistant"

agents:
  - id: chat-agent
    type: react
    trigger: direct://chat
    model:
      provider: openai
      model: gpt-4o
      api_key: "${OPENAI_API_KEY}"
    system_prompt: "You are a helpful assistant."
    memory:
      enabled: true
      type: sliding-window
      window_size: 20

workflows:
  - id: agui-server-chat
    trigger: "agui://0.0.0.0:4100/agent?name=my-assistant"
    steps:
      - target: "direct://chat"

Frontend integration (JavaScript):

const response = await fetch('http://localhost:4100/agent', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    thread_id: 'thread-1',
    run_id: 'run-1',
    messages: [
      { role: 'user', content: 'Explain quantum computing' }
    ]
  })
});

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const text = decoder.decode(value);
  const lines = text.split('\n');
  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const event = JSON.parse(line.slice(6));
      if (event.type === 'TEXT_MESSAGE_CONTENT') {
        process.stdout.write(event.content);
      }
    }
  }
}

CORS: The AG-UI server sets Access-Control-Allow-Origin: * by default.

7.2 A2A (Agent-to-Agent)¶

When to use: You want agents to communicate with other AI agents using the Google A2A protocol. Enables multi-system agent orchestration.

Protocol: A2A (Agent-to-Agent). Uses JSON-RPC over HTTP. Includes agent discovery via Agent Card.

URI schemes: - Server: a2a-server://host:port?name=...&description=...&version=... - Client: a2a://host:port (for calling external A2A agents)

Default port: 9000

Agent Card: Automatically published at /.well-known/agent-card.json. Contains agent name, description, skills, capabilities, and preferred transport.

Complete YAML example (Server):

helix:
  a2a:
    servers:
      - enabled: true
        addr: "0.0.0.0:9000"
        name: "helix-support-agent"
        description: "Customer support agent with billing and technical expertise"
        version: "1.0.0"

agents:
  - id: support-agent
    type: supervisor
    trigger: direct://support
    description: "Handle customer support requests"
    model:
      provider: anthropic
      model: claude-sonnet-4-20250514
      api_key: "${ANTHROPIC_API_KEY}"
    system_prompt: "Route support requests to the appropriate specialist."
    sub_agents:
      - id: billing
        type: react
        model:
          provider: anthropic
          model: claude-sonnet-4-20250514
          api_key: "${ANTHROPIC_API_KEY}"
        system_prompt: "Handle billing inquiries."
      - id: technical
        type: react
        model:
          provider: openai
          model: gpt-4o
          api_key: "${OPENAI_API_KEY}"
        system_prompt: "Handle technical support requests."

workflows:
  - id: a2a-server
    trigger: "a2a-server://0.0.0.0:9000?name=helix-support&description=Support+Agent"
    steps:
      - target: "direct://support"