Route templates¶

Route templates are reusable, parameterised route snippets. A template declares a route definition with {{.paramName}} placeholders and a typed parameter list; instantiating it produces a concrete workflow.Definition with the placeholders replaced.

Source: pkg/template/template.go (registry, parameter resolution, deep copy, YAML loader, fluent builder) and pkg/template/builtins.go (eight pre-built templates).

Status of the YAML surface. Templates are first-class in the Go API and ship with a YAML loader (LoadTemplateYAML / LoadTemplateYAMLFile), but they are not yet auto-wired through the top-level workflow YAML — there is no templates: block or from-template: step in pkg/dsl/yaml_loader.go. The intended use today is from Go: define your templates in init(), register them with a template.Registry, and instantiate them into the engine via dsl.NewReifier(...).Reify(def). See Wiring templates in Go below.

Template shape¶

id: http-to-kafka-bridge
description: "Accept HTTP POST, validate content type, and route to a Kafka topic"
parameters:
  - name: httpPort
    required: true
    type: string
    description: "Port for the HTTP server to listen on"
  - name: kafkaBrokers
    required: true
    type: string
    description: "Comma-separated Kafka broker addresses"
  - name: kafkaTopic
    required: true
    type: string
    description: "Target Kafka topic"
  - name: contentType
    required: false
    default: "application/json"
    type: string
    description: "Expected Content-Type header value"
route:
  trigger:
    uri: "http://0.0.0.0:{{.httpPort}}/ingest"
    params:
      httpMethodRestrict: "POST"
  steps:
    - log:
        message: "Received HTTP request for Kafka bridge"
        level: info
    - validate:
        starlang: |
          def validate(exchange):
              ct = exchange.in_header("Content-Type")
              if ct == None or "{{.contentType}}" not in ct:
                  fail("expected Content-Type {{.contentType}}, got " + str(ct))
    - set-header:
        name: "kafka.topic"
        constant: "{{.kafkaTopic}}"
    - target: "kafka://{{.kafkaBrokers}}/{{.kafkaTopic}}"

Top-level fields¶

Decoded into template.Template (pkg/template/template.go).

Parameters¶

Each entry maps to template.Parameter.

route: — the parameterised route¶

Same shape as one entry in the top-level workflows: list (workflow DSL) but without an id: (the instance gets its ID at instantiation time) and using only the step types the template loader supports:

target, log, process, transform, set-header, set-body, remove-header, filter, choice, split, enrich, poll-enrich, wire-tap, multicast, bean, validate, delay, throttle, retry, circuit-breaker, pipeline.

Step types that the loader doesn't recognise return unsupported step type "<key>". If you need them, drive instantiation from Go where you have access to the full workflow.Definition type and every step type.

Placeholder substitution¶

{{.paramName}} placeholders are replaced by simple string substitution (no Go text/template engine — see pkg/template/template.go: replaceParams). Substitution applies to every string field the template loader knows about:

• route.description, route.trigger.uri, every value in route.trigger.params.
• Every step's stringy fields (URIs, Starlang scripts, constants, header names, regex patterns, exception type lists, …).
• The route ID — set on the instance either explicitly via Instance.WorkflowID or auto-generated as <templateID>-<counter>.
• The error handler's dead-letter-channel, redelivery-delay, retry-while, and every exception list.

The full walk is in pkg/template/template.go: replaceStep (every step type) and replaceErrorHandler.

Wiring templates in Go¶

The intended workflow today:

package main

import (
    "github.com/arvenza/helix/pkg/dsl"
    "github.com/arvenza/helix/pkg/engine"
    "github.com/arvenza/helix/pkg/template"
)

func main() {
    // 1. Build a registry and add the built-in templates.
    reg := template.NewRegistry()
    template.RegisterBuiltins(reg)

    // 2. Optionally load custom templates from YAML files.
    if t, err := template.LoadTemplateYAMLFile("templates/my-bridge.yaml"); err == nil {
        _ = reg.Register(t)
    }

    // 3. Instantiate.
    def, err := reg.Instantiate(&template.Instance{
        TemplateID: "http-to-kafka-bridge",
        WorkflowID: "orders-ingest",      // optional; otherwise auto-generated
        Parameters: map[string]string{
            "httpPort":     "8080",
            "kafkaBrokers": "localhost:9092",
            "kafkaTopic":   "orders",
        },
    })
    if err != nil {
        panic(err)
    }

    // 4. Reify into the engine.
    ctx := engine.NewHelixContext()
    rt, err := dsl.NewReifier(ctx).Reify(def)
    if err != nil {
        panic(err)
    }
    _ = rt
}

Built-in templates¶

template.RegisterBuiltins(reg) adds these to a registry:

The full definitions are in pkg/template/builtins.go. Each is a working starting point you can copy + tweak rather than a black box — extract the body with reg.Get("rag-query-pipeline") and inspect the Template.Route field.

Fluent Go builder¶

The template.Builder mirrors the YAML shape:

tmpl := template.NewTemplate("kafka-to-database-sink").
    Description("Consume from Kafka, transform, insert into PostgreSQL").
    Param("kafkaBrokers", template.Required(), template.Desc("Kafka brokers"), template.TypeString()).
    Param("kafkaTopic",   template.Required(), template.Desc("Source topic")).
    Param("kafkaGroup",   template.Default("helix-db-sink"), template.Desc("Consumer group")).
    Param("pgDSN",        template.Required(), template.Desc("PostgreSQL DSN")).
    Param("pgTable",      template.Required(), template.Desc("Target table")).
    Route(&workflow.Definition{
        Trigger: workflow.TriggerDefinition{
            URI:    "kafka://{{.kafkaBrokers}}/{{.kafkaTopic}}",
            Params: map[string]string{"groupId": "{{.kafkaGroup}}"},
        },
        Steps: []workflow.StepDefinition{
            { Transform: &workflow.TransformDefinition{
                Starlang: `read(payload, "application/json")`,
            }},
            { Target: &workflow.TargetDefinition{
                URI: "pg://{{.pgDSN}}/{{.pgTable}}",
            }},
        },
    }).
    Build()

Parameter options: Required(), Default("x"), Desc("..."), TypeString() / TypeInt() / TypeBool() / TypeDuration().

Behaviour notes / gotchas¶

• Templates are not currently loaded from the top-level YAML document. pkg/dsl/yaml_loader.go has no templates: field. To use a template today, instantiate it from Go (see above) or hand-render it (run Instantiate, marshal the result to YAML, paste into workflows:).
• Required parameters with empty-string values still pass. The loader only checks "was the key supplied?" — pass "" for an optional default if you really mean empty.
• String substitution is naive. {{.x}} matches anywhere it appears, so don't pick parameter names that collide with literal strings in your route. Prefer descriptive names like kafkaBrokers, not host.
• Step coverage in the YAML loader is partial. Only the 21 step types listed under route: are recognised by LoadTemplateYAML. Use the Go builder if you need a step type not on that list (do-try, aggregate, agent, every rag-*, every inline http-request: / db-query: / …).
• Deep copy is exhaustive. Instantiating the same template twice produces two independent workflow.Definitions — placeholder replacement never mutates the template.

See also¶

• Workflow DSL — the shape that route: mirrors
• Custom component guide — for when a template isn't enough and you want a new URI scheme

Source¶

• pkg/template/template.go — registry, parameter resolution, deep copy, YAML loader, fluent builder
• pkg/template/builtins.go — the 8 pre-built templates
• pkg/workflow/definition.go — the Definition shape templates produce