Deploy a collector fleet on Kubernetes

This guide deploys a collector fleet on Kubernetes: one collector pod per node, enrolled with LinkMesh for central config and throughput. The collector is upstream — either Grafana Alloy (via its official Helm chart) or otelcol-contrib + opampsupervisor (as a DaemonSet from the upstream OTel release image). LinkMesh ships no collector distribution of its own; it’s the control plane that configures whichever runtime you pick.

If your hosts are plain Linux (VMs, bare metal, EC2 instances), the Add a collector flow with the per-host installer is simpler — this page is the Kubernetes substrate for the same two runtimes.

The collector runtime is always upstream

Both paths below run upstream Grafana Alloy or upstream otelcol-contrib, pulled from their official charts/images. LinkMesh pushes (or serves) the pipeline config and reads back throughput — it does not replace, wrap, or rebuild the collector.

Pick a runtime

A collector runs one of two runtimes. The runtime decides how its config arrives — pick one per fleet (you can run different runtimes in different clusters or namespaces).

Runtime	managementMode	How config arrives	Deploy as
Grafana Alloy + remotecfg	alloy-remotecfg	Alloy pulls config over Bearer-authenticated HTTPS	Upstream grafana/alloy Helm chart
otelcol-contrib + OpAMP	opamp	Server pushes config over OpAMP (WSS); opampsupervisor applies it	DaemonSet on the upstream OTel release image

Both give you central config push, fleet status, and per-component throughput on the topology canvas. Grafana Alloy via its Helm chart is the recommended Kubernetes default — it’s a single upstream image with a first-class chart.

Prerequisites

• A Kubernetes cluster (any flavour — kind, k3s, EKS, GKE, AKS, on-prem).
• kubectl configured for the cluster; helm for the Alloy path.
• A running LinkMesh server, reachable from your pods over HTTPS (Alloy remotecfg + own_metrics) or WSS (OpAMP). One instance is enough; to make the server itself resilient, run it highly available on an external MongoDB database.
• Your server’s public base URL handy, e.g. https://linkmesh.example.com.
• That same URL configured on the server as externalUrl — see Quickstart step 2. It must be an address your pods resolve (an Ingress hostname or a Service DNS name, never a pod IP). Skip it and the fleet enrols and ships data correctly while every collector shows throughput 0 / CPU 0 / memory 0.

1. Mint a reusable enrollment token

Open Collectors → + Add Collector in the LinkMesh UI, pick your runtime, and choose the Kubernetes snippet — it’s pre-filled with your server URL and a token.

Because a DaemonSet enrols many pods (and reschedules them), use a reusable enrollment token rather than a single-use one: one token enrols every pod, and a rescheduled pod re-attaches without minting anything new. Mint reusable, scoped tokens under Settings → Enrollment Tokens; see Enrollment tokens for TTL, scope, and revocation.

2. Deploy the fleet

Grafana Alloy (Helm)

Deploy the upstream grafana/alloy chart as a DaemonSet. Alloy pulls its pipeline config from LinkMesh via remotecfg and pushes its own metrics back so the topology canvas shows per-component throughput.

helm repo add grafana https://grafana.github.io/helm-charts && helm repo update

Write linkmesh-alloy-values.yaml, substituting your server URL, a fleet id, and the token:

controller:
  type: daemonset
# Cluster-read RBAC (chart default) so pod-log enrichment and the kubelet
# receiver can look up pods, namespaces, and nodes.
rbac:
  create: true
alloy:
  # Mount the node's log tree read-only so a Kubernetes Pod Logs source can
  # tail /var/log/pods; dockercontainers covers the symlink target on some
  # distros. The node name + IP feed the kubelet receiver and enrichment.
  mounts:
    varlog: true
    dockercontainers: true
  extraEnv:
    - name: K8S_NODE_NAME
      valueFrom: { fieldRef: { fieldPath: spec.nodeName } }
    - name: K8S_NODE_IP
      valueFrom: { fieldRef: { fieldPath: status.hostIP } }
  configMap:
    content: |
      logging { level = "info" }

      remotecfg {
        url            = "https://linkmesh.example.com"
        id             = "k8s-fleet"
        poll_frequency = "60s"

        // LinkMesh's remotecfg auth accepts ONLY the Bearer scheme.
        bearer_token = "<ENROLLMENT_TOKEN>"
      }

      // own_metrics -> LinkMesh (per-component throughput on the canvas)
      prometheus.exporter.self "default" { }
      prometheus.scrape "linkmesh_self" {
        targets         = prometheus.exporter.self.default.targets
        forward_to      = [otelcol.receiver.prometheus.linkmesh.receiver]
        scrape_interval = "30s"
      }
      otelcol.receiver.prometheus "linkmesh" {
        output { metrics = [otelcol.exporter.otlphttp.linkmesh.input] }
      }
      otelcol.exporter.otlphttp "linkmesh" {
        client {
          endpoint = "https://linkmesh.example.com"
          headers  = { "Authorization" = "Bearer <ENROLLMENT_TOKEN>" }
        }
      }

helm install linkmesh-alloy grafana/alloy \
  --namespace linkmesh --create-namespace \
  --values linkmesh-alloy-values.yaml

url is the base, not /v1/opamp

remotecfg.url and the own_metrics exporter endpoint are the server’s base URL (e.g. https://linkmesh.example.com). Alloy’s remotecfg client appends the CollectorService path itself; /v1/opamp is the OpAMP endpoint, a different protocol. Behind a reverse proxy, use the public hostname — see Run behind a reverse proxy.

All pods share the same id and token, so the fleet registers as one logical collector. Want each node as a distinct collector? Template a per-pod id (e.g. from the node name) instead of the shared k8s-fleet id.

The line-by-line meaning of this config — and the standalone-host version — is in Onboard Grafana Alloy via remotecfg.

otelcol-contrib (OpAMP)

Run the upstream opampsupervisor image, with an init container that stages the matching otelcol-contrib binary into a shared volume. Two upstream artifacts are involved because the supervisor image contains only the supervisor, and the otelcol-contrib image is scratch-based (no shell), so the collector binary comes from the upstream release tarball instead — pin both to the same version to avoid skew. The supervisor connects to LinkMesh over OpAMP, spawns otelcol-contrib as a subprocess, and applies the config the server pushes.

The manifest below is self-contained: it grants the ServiceAccount the read-only RBAC the Kubernetes receivers need (kubelet metrics, pod/namespace lookup for attribute enrichment), mounts /var/log/pods for pod logs, and exposes the node name + IP — so activating a Kubernetes Pod Logs or node metrics source in the UI works with no further edits to the cluster.

kubectl create namespace linkmesh --dry-run=client -o yaml | kubectl apply -f -

kubectl apply -n linkmesh -f - <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: linkmesh-otelcol
  namespace: linkmesh
---
# Read-only cluster access — never write, never cluster-admin. Covers the
# kubelet (node/pod/container metrics), pod-log attribute enrichment, and
# cluster object metrics.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: linkmesh-otelcol
rules:
- apiGroups: [""]
  resources: [nodes, nodes/stats, nodes/proxy, pods, namespaces]
  verbs: [get, list, watch]
- apiGroups: ["apps"]
  resources: [replicasets, deployments, daemonsets, statefulsets]
  verbs: [get, list, watch]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: linkmesh-otelcol
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: linkmesh-otelcol
subjects:
- kind: ServiceAccount
  name: linkmesh-otelcol
  namespace: linkmesh
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: linkmesh-supervisor
data:
  supervisor.yaml: |
    server:
      endpoint: "wss://linkmesh.example.com/v1/opamp"
      headers:
        Authorization: "Bearer <ENROLLMENT_TOKEN>"
    capabilities:
      accepts_remote_config: true
      reports_effective_config: true
      reports_health: true
      reports_remote_config: true
      reports_own_metrics: true
    agent:
      executable: /otelcol-bin/otelcol-contrib
    storage:
      directory: /var/lib/otelcol-supervisor
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: linkmesh-otelcol
spec:
  selector:
    matchLabels: { app: linkmesh-otelcol }
  template:
    metadata:
      labels: { app: linkmesh-otelcol }
    spec:
      serviceAccountName: linkmesh-otelcol
      # Run on every node — including control-plane — so no node's pod logs or
      # node metrics are missed. Narrow this if you don't want telemetry from
      # tainted/specialised nodes.
      tolerations:
      - operator: Exists
      initContainers:
      # Stages otelcol-contrib into the shared volume; keep the version in
      # this URL identical to the supervisor image tag below.
      - name: fetch-otelcol
        image: alpine:3.20
        command: ["sh", "-c"]
        args:
        - wget -qO- https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.153.0/otelcol-contrib_0.153.0_linux_amd64.tar.gz | tar -xz -C /otelcol-bin otelcol-contrib
        volumeMounts:
        - { name: otelcol-bin, mountPath: /otelcol-bin }
      containers:
      - name: supervisor
        # The image's entrypoint already runs the supervisor — pass args only.
        image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-opampsupervisor:0.153.0
        args: ["--config", "/etc/otelcol-supervisor/supervisor.yaml"]
        # The node name + IP the kubelet receiver targets and the attributes
        # processor filters on. The supervisor passes them to its otelcol child.
        env:
        - name: K8S_NODE_NAME
          valueFrom: { fieldRef: { fieldPath: spec.nodeName } }
        - name: K8S_NODE_IP
          valueFrom: { fieldRef: { fieldPath: status.hostIP } }
        volumeMounts:
        - { name: cfg, mountPath: /etc/otelcol-supervisor }
        - { name: data, mountPath: /var/lib/otelcol-supervisor }
        - { name: otelcol-bin, mountPath: /otelcol-bin }
        # Pod logs, read-only. The filelog receiver tails /var/log/pods; on
        # containerd those are symlinks into /var/lib/docker/containers on some
        # distros, so mount both.
        - { name: varlogpods, mountPath: /var/log/pods, readOnly: true }
        - { name: varlibdockercontainers, mountPath: /var/lib/docker/containers, readOnly: true }
        # Read-offset checkpoints so a pod restart resumes where it left off
        # instead of re-shipping whole log files.
        - { name: storage, mountPath: /var/lib/otelcol/storage }
      volumes:
      - { name: cfg, configMap: { name: linkmesh-supervisor } }
      - { name: data, emptyDir: {} }
      - { name: otelcol-bin, emptyDir: {} }
      - { name: varlogpods, hostPath: { path: /var/log/pods } }
      - { name: varlibdockercontainers, hostPath: { path: /var/lib/docker/containers } }
      - { name: storage, hostPath: { path: /var/lib/linkmesh-otelcol/storage, type: DirectoryOrCreate } }
EOF

Note

Use wss:// for any server reachable over the network; ws:// is only for loopback/lab. The endpoint host/port matches the web UI — only the path (/v1/opamp) differs. Behind a reverse proxy, point at the public hostname (see Run behind a reverse proxy). The init container URL pins linux_amd64; on arm64 node pools substitute linux_arm64.

The standalone-host version of this runtime, plus how the two binaries fit together, is in Onboard otelcol-contrib via OpAMP.

3. Verify

Grafana Alloy (Helm)

# Wait for an Alloy pod on every node
kubectl -n linkmesh rollout status ds/linkmesh-alloy

# Watch remotecfg fetch its config
kubectl -n linkmesh logs -l app.kubernetes.io/name=alloy --tail=20 | grep -i remotecfg

otelcol-contrib (OpAMP)

# Wait for a supervisor pod on every node
kubectl -n linkmesh rollout status ds/linkmesh-otelcol

# Watch the OpAMP handshake + config apply
kubectl -n linkmesh logs -l app=linkmesh-otelcol --tail=20

In your LinkMesh UI, open Collectors — the fleet appears with its runtime (alloy-remotecfg or opamp) and leaves awaiting_connection within ~60s (Alloy’s first poll) or ~30s (the OpAMP handshake). The topology canvas renders throughput once own_metrics start landing.

Production hardening

The manifests above are deliberately minimal. Before promoting to production:

• Pin the image tag to a specific version instead of a floating one — both the OTel release image and grafana/alloy move forward and will surprise you on the next pod cycle. Pin the Alloy chart version too (helm install --version).
• Add a NetworkPolicy on the linkmesh namespace allowing egress only to your LinkMesh server’s HTTPS / WSS port.
• Move the token into sealed-secrets, SOPS, or Vault rather than templating it into values/manifests. A reusable enrollment token is a fleet credential — treat it accordingly, and revoke + re-mint to rotate.
• Set resource requests/limits sized to your telemetry volume; the Alloy chart exposes alloy.resources, and you can add a resources: block to the OpAMP DaemonSet container.
• Review the cluster access. The RBAC above is read-only (get/list/watch, never write, never cluster-admin) — the least the Kubernetes receivers need. The DaemonSet also tolerates every node so no node’s logs are missed; narrow the tolerations if you don’t want telemetry from tainted or specialised nodes.

Trying it on kind

For a quick local evaluation:

# Create a kind cluster
kind create cluster --name linkmesh-eval

# Run a LinkMesh server reachable from the kind cluster
# (host.docker.internal works from kind pods on macOS/Windows)

# Then follow steps 1-3 with your server URL set to, e.g.:
#   https://host.docker.internal:8080   (Alloy remotecfg / own_metrics)
#   wss://host.docker.internal:8080/v1/opamp   (OpAMP)

kind nodes share a Docker network, so any service reachable from your host on host.docker.internal:<port> is reachable from the collector pods too.

Uninstall

Grafana Alloy (Helm)

helm uninstall linkmesh-alloy --namespace linkmesh
kubectl delete namespace linkmesh

otelcol-contrib (OpAMP)

kubectl delete namespace linkmesh

Then revoke the fleet’s enrollment token under Settings → Enrollment Tokens if you’re decommissioning the cluster.

Optional: + onboarding agent

Either runtime can be paired with linkmesh-agent, an optional edge connector that discovers the workloads, services, and log sources in your cluster and feeds them into the onboarding wizard. It is not a third way to run a collector — it never installs, supervises, or configures the collector, and carries no telemetry. Add it only when discovery saves you onboarding work. See The agent.

Onboard from the cluster inventory

The steps above enrol a collector fleet you then wire by hand. To instead browse your namespaces and workloads in the UI and onboard their logs in one click, install the linkmesh-agent alongside the fleet — it reports the cluster inventory the onboarding view reads. The agent’s packaging/k8s kustomize base installs both the agent and an OpAMP collector fleet in one apply, sharing a single reusable fleet token.

The Enroll Agent wizard (Agents → Enroll Agent → Kubernetes) hands you this manifest ready to apply, with the reusable fleet token already filled in:

The Enroll Agent wizard's Kubernetes tab: a zero-edit DaemonSet manifest carrying the reusable fleet token, pod-log mounts, and read-only RBAC. One kubectl apply enrols one agent per node.

Or apply the same kustomize base from the command line:

kubectl -n linkmesh-system create secret generic linkmesh-agent-bootstrap \
  --from-literal=enrollment-token="$LINKMESH_TOKEN"
kubectl apply -k <path-to>/linkmesh-agent/packaging/k8s

Within ~60s the fleet enrols and your namespaces, workloads, and services appear on the group’s Kubernetes tab. Each workload has a one-click Onboard logs action, and a Kubernetes Pod Logs source starts tailing it with no further cluster edits — the manifest already carries the pod-log mounts and read-only RBAC:

The group's Kubernetes tab shows the live cluster the agent discovered — reporting members, namespaces, workloads, and services — with a one-click Onboard logs action on any workload.

Cluster-wide telemetry is one click too. The Collect cluster metrics card enables node metrics (kubeletstats, on every DaemonSet member), plus cluster-object metrics (k8s_cluster) and Kubernetes events (k8s_events) as cluster-scoped singletons:

One click enables cluster telemetry for the whole fleet — per-node kubeletstats, plus cluster-object metrics and events collected once cluster-wide. Multi-member fleets disable the cluster-singleton sources so only one replica collects them.

See the agent’s packaging/k8s/README.md for the step-by-step, and The agent for what it does and does not do.

Next steps

• Build your first pipeline — wire your new K8s-enrolled collectors through a processing pipeline to a destination.
• Concepts → Collector — what a collector is + the two collector runtimes.
• Concepts → Collector group — organise a fleet into a group on the canvas.
• Troubleshooting → Enrollment — when a pod doesn’t show up.