Pipeline

A pipeline is a reusable processing recipe. It owns:

• A linear chain of processors (filter, transform, mask, enrich, …) — each step gets the output of the previous one as its input
• An output filter that can drop records before they’re handed off
• A parameter schema that routes can override per-deployment

Pipelines are pure recipes. They don’t know where they run or where they send data. A route binds a pipeline to a source and one or more destinations — which means the same pipeline can be applied to many sources, on many collectors, sending to different destinations. Build “PII Redaction” once, attach it everywhere.

Mental model

The chain editor. Input and Output anchor the chain; each processor you add becomes a step in between, evaluated top to bottom.

Picture the pipeline as a left-to-right pipe with Input on one end and Output on the other. Processors stack vertically in between in the order they execute. The dry-run panel re-runs the whole chain against a sample on every change so you can see what each step did.

Where it lives

Surface	Path
UI	Process → Pipelines in the sidebar
Storage	One MongoDB document per pipeline, mirrored to YAML at gitops/pipelines/<pipelineID>.yaml
Reach	Tenant-wide. Any route on any collector or Collector Group in the same tenant can reference it.

A pipeline isn’t deployed anywhere on its own. Saving a pipeline updates the recipe; the collectors that already reference it via a route pick up the new definition on the next config push.

See also

• Route — how a pipeline gets attached to a real source + destinations
• Build your first pipeline — a step-by-step chain-editor walkthrough with dry-run
• Drop noisy logs — filter-processor patterns worth knowing before you build your first chain