Starlang user guide¶

Starlang is the small embedded language Helix uses for expressions (filters, choice predicates, header setters) and transforms (data shaping between route steps). It's a separate project — the sibling starlang repository — that Helix embeds via a Go replace directive.

This guide covers everything you need to use Starlang inside Helix YAML routes: where it runs, what the syntax looks like, every common builtin, the AI module, and the gotchas that trip people up.

What Starlang is¶

A small, declarative, expression-oriented language designed for data shaping. Each script produces one value — usually a transformed payload. There are no statements, no loops, no return; iteration is expressed with map, filter, reduce over collections.

Think of it as a typed superset of JSON expressions.

Key properties: - Pure functional core — no side effects in expression context - One value out — every script evaluates to a single value - Format-aware — reads/writes JSON, XML, CSV, YAML, TOML, MessagePack, properties files - Arbitrary-precision decimals — no float surprises - First-class temporal types — Date, DateTime, Time, Period, TimeZone - Compiled & cached — Helix caches compiled scripts; expressions in a hot route are essentially free

The two engines¶

The same language, two different invocations, with subtly different rules. This is the single most important thing to understand.

Expression engine¶

Used inside YAML for predicate-style values: where you write a small expression to compute a header, decide a filter, or pick a choice branch.

Helix auto-wraps the expression for you. You write:

processors:
  - set-header:
      name: X-Tenant
      expression: payload.user.tenant

Helix turns that into:

%sl 2.0
output application/json
input payload
input headers
input properties
---
payload.user.tenant

before handing it to Starlang. You never have to think about directives in expression context.

Inputs available: payload (auto-parsed JSON/XML body), headers, properties. Plus helper functions: header("name"), property("name"), now(), today().

Where it's used: - set-header (expression: field) - set-body (expression: field) - filter (predicate) - choice → when (predicate) - validate (predicate) - Inline ${starlang: expr} substitutions

Transform engine¶

Used inside the transform processor step for full transformations — restructuring a JSON document, mapping fields, building a new payload.

No auto-wrap: the script must start with the %sl 2.0 directive header.

processors:
  - transform:
      starlang: |
        %sl 2.0
        output application/json
        input payload
        ---
        {
          orderId: payload.id,
          total: sum(payload.items.*.price),
          status: if (payload.paid) "settled" else "pending"
        }

Inputs available: same as expression engine, PLUS setHeader(name, value) and setProperty(name, value) if you declare input setHeader / input setProperty directives.

AI module auto-imports: every transform script gets sl::ai::Extract, sl::ai::Text, and sl::ai::Schema auto-imported. You can call extractTables(text), chunkByTokens(text, 500), parseResponse(llmText) without writing import * from sl::ai::Text.

Quick syntax tour¶

Literals¶

42                        // Number (arbitrary-precision decimal)
3.14
"hello"                   // String
'hello'                   // also a string
"hello ${name}"           // interpolated string
true   false   null
[1, 2, 3]                 // Array
{a: 1, b: 2}              // Object
|2026-06-04|              // Date
|2026-06-04T12:00:00Z|    // DateTime
/^foo\d+$/                // Regex
/^foo$/i                  // Regex with flags

Operators¶

Operator	Meaning
`+ - * /`	Arithmetic
`mod` or `%`	Modulo
`pow`	Exponentiation
`++`	Concatenate strings, append to arrays, merge objects
`--`	Remove from array / object
`== !=`	Equality
`~=`	Loose equality (string/number coercion)
`< > <= >=`	Comparison
`and or not`	Logical (also `&& \|\| !`)
`matches`	Regex match: `payload.email matches /@/`
`contains startsWith endsWith`	String/array predicates
`is`	Type check: `payload.value is Number`
`->`	Lambda: `(x) -> x.price`
`?.`	Nullable property access
`?:`	Default if null
`::`	Namespace access: `sl::ai::Text::chunkByTokens`

Selectors¶

The dot operator has variants for navigating nested data:

payload.user.name           // single field
payload.users.*.email       // array of emails (apply across all items)
payload..email              // recursive search for "email" anywhere
payload.users[0]            // index
payload.users[0..3]         // range slice
payload..users[?(@.age > 18)]  // filter expression

Control flow¶

No for / while / return. Use:

if (cond) "yes" else "no"

match value {
  case 0          -> "zero"
  case is Number  -> "a number"
  case /\d+/      -> "digits"
  default         -> "something else"
}

do {
  var subtotal = sum(payload.items.*.price)
  var tax = subtotal * 0.085
  ---
  subtotal + tax
}

try { riskyCall() } catch (err) { "fallback" }

Iteration via builtins¶

payload.items map (item) -> item.price          // → [10, 20, 30]
payload.items filter (item) -> item.qty > 0
payload.items reduce ((acc, item) -> acc + item.price, 0)
payload.items groupBy (item) -> item.category
payload.items orderBy (item) -> item.timestamp

The `%sl 2.0` directive header¶

Required for transform scripts. Anatomy:

%sl 2.0
output application/json                  // response MIME type
input payload                            // declare available inputs
input headers
input properties
input setHeader                          // optional: enable side-effect builtins
import * from sl::ai::Text               // optional: extra modules
var taxRate = 0.085                      // bindings used in the body
fun discount(p) = p * 0.10               // helper function
---
{                                        // ← body: ONE expression
  total: payload.total * (1 + taxRate)
}

Rules:

--- separates header from body. Everything after --- is one expression. To compose multiple expressions, wrap in a do { ... } block.
Variables go in directives. Use var x = ... in the header. You cannot reassign variables anywhere; there's no x = ... statement.
Side-effect calls need a var wrapper. Bare expression statements in the directive section are silently dropped. To call setHeader:
```
var _hdr = setHeader("X-Trace-Id", uuid())
```
The _hdr value is unused — the point is to force evaluation.
Use // for comments, not #. # is a token (namespace prefix delimiter) and is silently dropped in the directive section. Block comments: /* ... */.

Valid directive keywords¶

Directive	Use
`output <mime>`	Response format. Common: `application/json`, `text/plain`, `application/xml`, `text/csv`
`input <name>`	Declare an input variable. Inputs: `payload`, `headers`, `properties`, `setHeader`, `setProperty`
`import * from <ns>`	Pull in a module (auto-imported in transform: `sl::ai::Extract`, `sl::ai::Text`, `sl::ai::Schema`)
`var <name> = <expr>`	Bind a variable
`fun <name>(args) = <expr>`	Define a helper function
`type <name> = <struct>`	Define a type alias
`ns <prefix> <uri>`	Declare an XML namespace prefix

Anything else after %sl 2.0 and before --- is silently ignored. If your setHeader(...) call doesn't fire, double-check you wrapped it in var _x = ....

Common builtins¶

The standard library has ~85 globals and a few dozen namespaced functions. The ones you'll use most:

Strings¶

upper(s)             lower(s)            trim(s)
contains(s, "x")     startsWith(s, "x")  endsWith(s, "x")
replace(s, "a", "b")
splitBy(s, ",")      joinBy([a, b], "/")
match(s, /\d+/)      matches(s, /^\d+$/)
sl::core::Strings::camelCase("foo bar")  // "fooBar"
sl::core::Strings::truncate(s, 100)

Arrays¶

sizeOf(arr)          isEmpty(arr)
sum(arr)             avg(arr)            min(arr)    max(arr)
filter(arr, fn)      map(arr, fn)        flatMap(arr, fn)
reduce(arr, fn, init)
find(arr, fn)        indexOf(arr, val)
distinctBy(arr, fn)
groupBy(arr, fn)     orderBy(arr, fn)
flatten(arr)
pluck(arr, "field")  // shorthand for `arr map (x) -> x.field`

Objects¶

keysOf(obj)          valuesOf(obj)       entriesOf(obj)
sizeOf(obj)
obj1 ++ obj2         // merge (right wins)
obj -- "key"         // remove key
flattenObject(obj)   nestObject(obj)
coalesceFields(obj, ["a", "b", "c"])  // first non-null

Type / null handling¶

typeOf(value)                  // "Number" / "String" / "Object" / ...
isInteger(n)   isDecimal(n)
isEmpty(x)     isBlank(s)
onNull(value, default)         // null-coalescing
to(value, "Number")            // explicit coercion
value is String                // type predicate

Data formats¶

read(text, "application/json")    // parse JSON
read(text, "text/csv")            // parse CSV → array of objects
write(obj, "application/xml")     // serialize to XML
write(obj, "text/yaml")           // serialize to YAML

Supported formats: JSON, XML, CSV, YAML, TOML, properties files, MessagePack, plain text.

Temporal¶

now() / today() are always in scope. The richer date builtins live in sl::core::Dates — bring them in with import * from sl::core::Dates.

now()                        // current DateTime
today()                      // current Date
|2026-06-04| + period("P30D")   // date math (period)
date >> "America/New_York"   // shift to timezone
period("PT2H30M")            // 2.5 hours

import * from sl::core::Dates
addDays(d, 30)    addMonths(d, -1)    addYears(d, 1)   // preserve the input type
addHours(t, 5)    addMinutes(t, 45)   addSeconds(t, 90)
parseTime("2026-08-05", "date")               // → DateTime
parseTime("2026-08-05T10:00:00Z", "rfc3339")  // friendly layout names
formatTime(d, "2006-01-02")   // → plain string (Go layout, not pipe-wrapped)
formatTime(d, "datetime")     // friendly: date / datetime / rfc3339 / time / kitchen

Utility¶

uuid()                                  // generate UUID v4
header("X-Trace-Id")                    // read exchange header
property("region")                      // read exchange property
logInfo("loaded", {count: sizeOf(items)})  // structured log
logDebug   logWarn   logError

Using Starlang in Helix routes¶

The full set of integration points:

`set-header` / `set-property` / `set-body`¶

Predicate-style expression; no header needed.

- set-header:
    name: X-Customer
    expression: payload.customer.id

- set-property:
    name: requestTimestamp
    expression: now()

- set-body:
    expression: |
      {
        ok: true,
        ts: now(),
        id: payload.id
      }

`filter`¶

Drop the exchange when the predicate is false.

- filter:
    expression: payload.amount > 0 and headers.region == "US"

`choice`¶

Multi-way branch on predicates.

- choice:
    when:
      - expression: payload.priority == "high"
        do:
          - to: log://info?prefix=HIGH
      - expression: payload.amount > 10000
        do:
          - to: direct://review
    otherwise:
      - to: direct://normal

`transform` step (full transform engine)¶

The big one — restructuring data, calling AI helpers, building rich responses.

- transform:
    starlang: |
      %sl 2.0
      output application/json
      input payload
      ---
      {
        orderId: payload.id,
        lineCount: sizeOf(payload.items),
        total: sum(payload.items map (i) -> i.price * i.qty),
        currency: payload.currency,
        timestamp: now(),
        // Use auto-imported sl::ai::Text helpers:
        normalizedNote: normalize(payload.note),
        keywords: keywords(payload.description, 5)
      }

`validate` step¶

Same shape as filter but raises an error instead of dropping.

- validate:
    expression: payload.email matches /^[^@]+@[^@]+\.[^@]+$/
    message: invalid email format

Inline `${starlang: ...}` substitution¶

Anywhere a config string is expected, ${starlang: <expr>} interpolates a value. Useful for URI parameters:

- to: http://api.example.com/users/${starlang: payload.userId}

The AI module¶

sl::ai::* is the namespace for LLM-shaped helpers. Auto-imported in transform scripts. You don't need explicit imports — just call:

`sl::ai::Extract` — pull structured data from text¶

extractTables(markdownText)           // → array of {headers, rows}
extractKeyValuePairs(text)            // → object of key→value
extractDates(text)                    // → array of dates found
extractNumbers(text)                  // → array of numbers
extractUrls(text)
extractEmails(text)
cleanOCR(rawOcrText)                  // fix common OCR artifacts
extractSections(text)

`sl::ai::Text` — text processing, chunking, PII¶

chunkByTokens(text, 500)              // → array of chunks
chunkByMarkdownHeaders(md)
slidingWindow(text, size, overlap)
paragraphSplit(text)
sentenceSplit(text)

stripMarkdown(md)
normalize(text)                       // unicode normalize, whitespace fix

redactPII(text)                       // mask emails, phones, SSNs
detectPII(text)                       // → array of {type, span}

tokenCount(text)
truncateTokens(text, maxTokens)
keywords(text, n)                     // top-n keyword extraction
similarity(textA, textB)              // cosine on token-overlap

`sl::ai::Schema` — parse and validate LLM responses¶

parseResponse(llmRawText)             // robust JSON-or-fix parser
parseJsonOrFix(text)
extractStructured(text, jsonSchema)
extractJson(text)
extractXml(text)

extractToolCalls(response)            // → array of {name, args}
extractText(response)
extractUsage(response)                // → {prompt_tokens, completion_tokens, total_tokens}
extractModel(response)
extractStopReason(response)

validate(obj, jsonSchema)             // → bool
isValid(obj, jsonSchema)
jsonSchema(exampleObj)                // infer a schema

assistantMessage(text)                // build OpenAI-style messages
userMessage(text)
systemMessage(text)
toolResult(callId, content)

aggregateUsage(responses)
estimateCost(usage, modelName)

Example — extract OCR'd table data and post-process:

- transform:
    starlang: |
      %sl 2.0
      output application/json
      input payload
      ---
      do {
        var cleaned = cleanOCR(payload.text)
        var tables = extractTables(cleaned)
        var firstTable = tables[0]
        ---
        {
          rows: sizeOf(firstTable.rows),
          headers: firstTable.headers,
          totalAmount: sum(firstTable.rows map (r) -> to(r.amount, "Number"))
        }
      }

The geospatial module — `sl::gis`¶

Pure-Go geospatial functions (backed by pkg/geo, no cgo) — the same core as the geo:// route component and the geo.* agent tools. Import with import * from sl::gis. Points are [lon, lat] (WGS84); geometries are GeoJSON objects or WKT strings. Distances/areas are geodesic (meters / m²).

import * from sl::gis

// Measurement
distance(a, b)                 // meters between two [lon,lat] points
length(geometry)              // length / perimeter in meters
area(geometry)                // area in m²
bearing(a, b)                 // initial compass bearing, degrees 0–360
destination(point, deg, m)    // point m meters from `point` along bearing deg

// Predicates
contains(geometry, point)     // is point inside geometry?  → bool
within(point, geometry)       // is point within geometry?  → bool
dwithin(a, b, meters)         // are a and b within `meters`?  → bool
intersects(geomA, geomB)      // do the geometries intersect?  → bool

// Shape
bbox(geometry)                // [minLon, minLat, maxLon, maxLat]
centroid(geometry)            // → [lon,lat] centroid point
point(lon, lat)               // construct a GeoJSON Point

// Format
parseWKT(wktString)           // WKT → geometry object
toWKT(geometry)               // geometry → WKT string

// Geohash
geohashEncode(point, prec)    // → geohash string of length prec
geohashDecode(hash)           // → [lon,lat] cell center
geohashNeighbors(hash)        // → the 8 adjacent geohashes

Example — keep only the stores within 5 km of the user, tagged with a geohash:

- transform:
    starlang: |
      %sl 2.0
      output application/json
      input payload
      ---
      import * from sl::gis
      do {
        var here = [payload.user.lon, payload.user.lat]
        ---
        payload.stores
          filter (s) -> dwithin(here, [s.lon, s.lat], 5000)
          map (s) -> { name: s.name, cell: geohashEncode([s.lon, s.lat], 7) }
      }

Common gotchas¶

`#` is silently dropped — use `//`¶

%sl 2.0
output application/json
# this is silently dropped (bad)
// this is a comment (good)
/* multi-line
   also fine */
input payload
---
payload

Bare statements after `%sl 2.0` are dropped¶

// BAD — setHeader does nothing
%sl 2.0
output application/json
input payload
input setHeader
setHeader("X-Trace", uuid())    // ← silently dropped
---
payload

// GOOD — wrap in `var`
%sl 2.0
output application/json
input payload
input setHeader
var _t = setHeader("X-Trace", uuid())
---
payload

Regex literal: split on first unescaped `/`¶

/foo\/bar/                  // BAD — last char becomes trailing slash issue
/foo.bar/                   // GOOD — `.` matches `/`
match(s, "foo/bar")         // ALT — pass as string, post-2026-05-07 it's real regex

Variable assignment vs `var`¶

%sl 2.0
output application/json
input payload
x = 5                       // ← BAD: not a `var` directive; silently dropped
var x = 5                   // ← GOOD
---
x

Auto-wrap means body is ONE expression¶

%sl 2.0
output application/json
input payload
---
var x = 5                   // ← BAD: var only valid in directive section
x + 1

%sl 2.0
output application/json
input payload
var x = 5
---
x + 1                       // ← GOOD: var in directives, single expression in body

%sl 2.0
output application/json
input payload
---
do {
  var x = 5
  ---
  x + 1                     // ← also GOOD: do{} blocks allow local vars
}

Number vs Integer¶

Numbers are arbitrary-precision decimals. 5 and 5.0 are equal. Use isInteger(n) to check if a value is a whole number; use to(n, "Integer") to force integer arithmetic.

Null safety¶

payload?.user?.name returns null if any step is null. payload.user.name errors if user is null. Use ?. aggressively when shape isn't guaranteed; use onNull(value, "default") to coalesce.

Debugging¶

Print intermediate values¶

%sl 2.0
output application/json
input payload
var _log = logInfo("payload shape", {keys: keysOf(payload), size: sizeOf(payload)})
---
payload

logDebug, logInfo, logWarn, logError all accept (message, fieldsObj) and emit structured log lines visible in the helix log.

Type-check at runtime¶

%sl 2.0
output application/json
input payload
---
{
  is_object: typeOf(payload) == "Object",
  is_array:  typeOf(payload) == "Array",
  type:      typeOf(payload),
  size:      sizeOf(payload)
}

Errors¶

Starlang panics are caught by the engine's safeProcess wrapper. Failing scripts hit your route's error handler instead of crashing the runtime. The log line includes: - kind: starlang.expression or starlang.transform - exchangeId: the request that failed - stack: the Go stack trace at the recovery point - script_excerpt: a snippet of the failing Starlang body

If a script never seems to do what you expect and there's no error, 99% of the time it's the directive-section gotcha: a bare statement that was silently dropped. Wrap it in var _x = ....

Performance¶

Compiled scripts are cached per (script-text, engine) pair via sync.Map. The cache evicts past 10,000 entries. A hot route with a fixed expression pays no parse/compile cost after the first call.