Skip to content

DynamicLoop

DynamicLoop is a counted-loop processor whose iteration count is computed at runtime by a Starlang expression. Unlike the fixed-count Loop (where count: is a static integer), DynamicLoop evaluates an expression against the incoming exchange and loops that many times.

Processors in this file

NewDynamicLoop

YAML form (the loop: keyword switches between fixed and dynamic based on whether count or starlang is set):

steps:
  - loop:
      starlang: toNumber(header("X-Iterations"))
      max_iterations: 1000
      steps:
        - target: kafka://retry-batch
Field Type Default Description
starlang string required Starlang expression yielding a non-negative integer. Wrapped automatically into a minimal program if no --- is present. Accepts ints, int64, float64, or string-encoded numbers.
max_iterations int 10000 Safety cap. The processor errors if the expression returns more iterations than this.
steps list required Inner pipeline executed each iteration.

Constructor: NewDynamicLoop(processor engine.Processor, expression string, maxIter int) *DynamicLoop. Pass 0 for maxIter to use the default DefaultLoopSafetyCap = 10000.

How it works:

  1. On construction, the expression is wrapped (if needed) into:
%sl 2.0
output application/json
input payload
input header
input property
import * from sl::util::Coercions
---
<expr>

So toNumber(header("X-N")) works without explicit imports.

  1. On each Process call, the wrapped program is evaluated against the exchange via the default Starlang engine. Result is coerced to int (accepts int, int32, int64, float64, or numeric strings).

  2. Validates: negative counts and counts exceeding MaxIter both return an error.

  3. Runs the inner Processor that many times. Each iteration sets HelixLoopIndex (0-based) and HelixLoopSize on the exchange — same property contract as Loop.

Pitfalls:

  • Bare expression statements in Starlang are silently dropped — see Starlang directive rules. Wrap side effects in var _ = ... if needed.
  • The expression is evaluated once at the start of Process; the count is fixed for the loop duration. Use Loop with internal stop: for early termination.
  • Exceeding MaxIter returns an explicit error mentioning the cap so users can raise max_iterations: intentionally rather than silently producing huge loops.

Examples

workflows:
  - id: process-batch-by-header
    trigger: { uri: direct://batch }
    steps:
      - loop:
          starlang: toNumber(header("X-Batch-Size"))
          max_iterations: 500
          steps:
            - set-header:
                name: X-Index
                starlang: property("HelixLoopIndex")
            - target: kafka://batch-items

Source

pkg/processor/loop_dynamic.go

See also

  • eip — Loop for fixed-count loops.
  • eip_extra — LoopProcessor with optional per-iteration exchange copy.
  • Starlang guide — expression syntax, imports, coercions.