Skip to content

elasticsearch://

Elasticsearch via elastic/go-elasticsearch/v8. 11 producer operations covering CRUD, search, multi-get, multi-search, bulk, and admin. Full TLS shape, API-key + Basic auth, and built-in sniffing.

URI shape

elasticsearch://<indexName>?operation=<op>&hostAddresses=<host:port>&...

A query-string URI shape (elasticsearch://?indexName=&operation=...) is also accepted.

Operations

Op Description
Index Create or replace a document.
Update Partial-update a document by id.
Delete Delete a document by id.
GetById Fetch one document by id.
MultiGet Fetch many documents by id (_mget).
Search Run a _search query — supports paging or scroll.
MultiSearch Multiple searches in one request (_msearch).
Bulk Newline-delimited bulk write (_bulk).
Exists HEAD-check whether a doc id exists.
DeleteIndex Drop the index.
Ping Cluster health-check (GET /).

Parameters

Direct lift from Component.Describe().

Core

Param Type Default Producer/Consumer/Both Description
operation enum required Producer One of the 11 operations above.
indexName string path 1st segment Producer Target index. URI path or this param.
hostAddresses string localhost:9200 Producer Comma-separated host:port list. Scheme inferred from enableSSL / secure.
clusterName string — Producer Cluster name. Informational for go-elasticsearch v8.

Auth

Param Default Description
username — HTTP Basic auth username.
password — HTTP Basic auth password. Required when username is set.
apiKey — Elastic API key (base64-encoded). Overrides Basic auth when set.

TLS

enableSSL / secure (force HTTPS), certificatePath (alias) / tlsCAFile, tlsCertFile, tlsKeyFile, tlsInsecureSkipVerify, tlsMinVersion / tlsMaxVersion (TLSv1.0..TLSv1.3), tlsCipherSuites.

Timing

Param Default Description
connectionTimeout 10s Connect / ping timeout. Bare integer = ms.
socketTimeout 30s Per-request timeout.
maxRetryTimeout 30s Total retry-budget timeout.

Sniffing

enableSniffer (default false), snifferInterval, sniffAfterFailureDelay — auto-discover cluster nodes on startup and periodically.

Param Default Description
from 0 Result offset.
size 10 Page size.
useScroll false Enable scroll API for large result sets.
scrollKeepAliveMs 60s Scroll-context keep-alive.
waitForActiveShards — Number of active shards to wait for on write (Index).

Body shape

Param Default Description
enableDocumentOnlyMode false When true, inbound body IS the document. When false, body is {document, id?} envelope.

Header filtering + reconnect

headerIncludePattern, headerExcludePattern — regex applied to exchange headers. reconnect (default true), maxReconnectAttempts (default -1, unlimited), reconnectTimeWait (default 2s).

Body contract

Operation Body
Index, Update {document, id?} envelope, OR if enableDocumentOnlyMode=true the body itself.
Delete, GetById, Exists {id} (or id header).
Search Elasticsearch _search query DSL JSON.
MultiGet {ids: [...]} or {docs: [...]}.
MultiSearch Array of {header, body} pairs.
Bulk Pre-formed NDJSON OR []map[string]any actions.

Headers

Header Direction Description
elasticsearch.id out Document id from the response.
elasticsearch.index out Index name from the response.
elasticsearch.version out Document version.
elasticsearch.totalHits out (Search) Total matches.
elasticsearch.took out (Search) Server-side time in ms.
elasticsearch.scrollId out (Search w/ scroll) Scroll cursor id.

Refer to pkg/component/elasticsearch/headers.go for the full list.

Examples

Index a document:

- to: elasticsearch://products?operation=Index&hostAddresses=es:9200&enableDocumentOnlyMode=true

Search with a query DSL body:

- setBody:
    starlang: |
      %sl 2.0
      output application/json
      ---
      {"query": {"match": {"title": "helix"}}}
- to: elasticsearch://articles?operation=Search&hostAddresses=es:9200&size=20

API-key auth + TLS:

- to: elasticsearch://logs?operation=Search&hostAddresses=es.example.com:443&enableSSL=true&apiKey=${ES_API_KEY}

Bulk-load from a previous transform:

- to: elasticsearch://docs?operation=Bulk&hostAddresses=es:9200

Behavior notes

  • Producer-only. elasticsearch:// is not a from: source — to ingest ES results periodically use a timer or cron route that fans into Search.
  • No-silent-noop on unknown params. Unknown URI keys emit a WARN log line on endpoint creation but don't fail. This is by design — it lets older URIs keep working after spec drift.
  • Sniffing can interact badly with k8s services where pod IPs rotate; prefer enableSniffer=false behind a stable Service.
  • outputType doesn't apply here (Mongo concept). Result shape is always JSON-decoded into the exchange body.

Source

  • pkg/component/elasticsearch/elasticsearch.go — entry + Describe.
  • pkg/component/elasticsearch/operations.go — operation enum.
  • pkg/component/elasticsearch/producer.go — producer dispatch.
  • pkg/component/elasticsearch/op_*.go — per-operation handlers.
  • pkg/component/elasticsearch/config.go — URI parsing.

See also