Skip to content
LinkMesh Docs

Enrollment troubleshooting

A collector enrols by connecting to the LinkMesh server over its runtime channel — Alloy on its first remotecfg poll, otelcol-contrib on its first OpAMP handshake — and appears in the fleet within a few seconds. If a collector doesn’t show up, work through this list top-down.

Collector doesn’t appear in the fleet

Give it ~30 seconds, then check from the collector host that the server is reachable and the runtime is running:

Terminal window
# Alloy runtime — watch the remotecfg poll
sudo journalctl -u alloy -f | grep -i remotecfg


# otelcol + OpAMP runtime — watch the OpAMP handshake
sudo journalctl -u otelcol-contrib -f | grep -i opamp

Common causes:

  • Bad / expired token — the Bearer (Alloy remotecfg) or enrollment token (otelcol OpAMP) is wrong or already consumed. Mint a fresh one in the wizard.
  • Server unreachable — outbound HTTPS/WSS blocked, DNS doesn’t resolve, or the server is behind a VPN/network the host isn’t on. Test with curl -v https://your-server.example.com/api/v1/health.
  • Wrong endpoint — Alloy’s remotecfg.url must be the server base URL; otelcol’s OpAMP endpoint must be wss://<server>/v1/opamp. See the configuration reference.

The errors below are written from the agent installer’s [FAIL] markers, but the same token / reachability / TLS root causes apply to the collector runtime too.

”Enrollment token required”

You ran the installer without a token. The installer takes its token and server as flags, not environment variables — pass them after sh -s -- so the piped shell forwards them to the script:

Terminal window
curl -fsSL https://your-server.example.com/install.sh | \
  sudo sh -s -- --token eyJ... --server https://your-server.example.com

--server defaults to the public LinkMesh URL, so you only need it when you self-host. The OpAMP collector installer takes the same flags: install-opamp.sh --token <TOKEN> [--server <URL>].

”Enrollment failed” — token expired

Tokens default to 15 minutes. If the host took longer to install the package (slow network, big mirror), the token’s already gone.

Fix: mint a new token in the wizard and re-run. The script is idempotent — it won’t reinstall the package, just re-enrol.

”Enrollment failed” — server unreachable

The host can’t reach your LinkMesh server. Check:

Terminal window
curl -v https://your-server.example.com/api/v1/health

Common causes:

  • Outbound HTTPS blocked by host firewall
  • Internal DNS doesn’t resolve the server hostname from this host
  • Server is behind a VPN the host isn’t on

For air-gapped fleets, use the server-hosted install variant documented at linkmesh.io/install instead — agent host only needs to reach the LinkMesh server, not the public internet.

”Enrollment failed” — invalid or used token (401 unauthorized)

A single-use token was already used, or the token is invalid or expired. Each single-use token enrols exactly one collector — mint a fresh one for the next host, or use a reusable token for a fleet.

If you didn’t intentionally use it twice, check the audit log — someone or something has already redeemed this token.

Optional agent: “linkmesh-agent restarted” but it doesn’t appear in the UI

This applies only if you’re running the optional onboarding agent. Wait 30 seconds — the agent’s first heartbeat to the server takes up to a heartbeat interval to land. If it’s still missing after 60s:

Terminal window
journalctl -u linkmesh-agent -f

Common failure modes the journal will reveal:

  • Server unreachable — the agent can’t reach the server’s HTTPS endpoint. Run linkmesh-agent preflight (or nc -vz your-server.example.com 443) from the host. The control channel is a WebSocket over HTTPS on the same port as the web UI — there is no separate gRPC port.
  • 401 unauthorized — the enrollment token is invalid, already used, or expired. Mint a fresh token in the Add Collector wizard. A single-use token can’t enroll a second host — use a reusable token for fleets.
  • TLS verify failure — the journal shows x509: certificate is valid for …, not <addr>. The server’s TLS certificate doesn’t cover the address the agent dials. See just below.

”tls: failed to verify certificate” / “certificate is valid for … not …”

The agent’s journal repeats a line like:

tls: failed to verify certificate:
x509: certificate is valid for 10.0.1.3, 127.0.0.1, ::1, not 35.230.148.135

The agent reached the server, but the server’s TLS certificate — terminated by the ingress / reverse proxy in front of the server — has no SAN for the address the agent connects to (35.230.148.135 above). The agent authenticates with a Bearer token, not a client certificate, so the fix is always on the server’s TLS front door.

Why it happens: on a cloud VM the external IP is a 1:1 NAT that never appears on the VM’s own interface, so a certificate generated from local interface IPs won’t list it — you reach the box on an address its cert doesn’t know about.

Fix — reissue the server’s TLS certificate at your ingress / reverse proxy with the address agents use as a SAN. A DNS name is preferred (it survives IP changes); point the agent’s server.url at that same name.

Lab-only escape hatch: for a throwaway host, set certificates.insecureSkipVerify: true in the agent config to skip server-cert verification (the agent logs a WARN while it’s on). Never use it in production — it exposes the control channel to a man-in-the-middle. See Agent configuration.

Throwaway lab and can’t touch the server right now? The agent has an escape hatch — certificates.insecureSkipVerify: true in /etc/linkmesh/config.yaml — that connects without verifying the server cert. It disables MITM protection on the control plane and the agent will nag with a WARN on every connect, so treat it as dev-only and never ship it. The fix above is the right one for anything real; details under the agent config.yaml entry in the configuration reference.

”Two operators using the same token”

The first one wins. With a single-use token the second gets 401 unauthorized (enrollment token already used). The second operator should mint their own token (or use a reusable one for a shared fleet).

If you’re scripting bulk enrolment via Ansible/Terraform, mint one token per host inside the loop — don’t share.

All collectors went offline after a server restart

This resolves itself. Telemetry never flows through the server, so your data keeps moving during the restart; once the server is healthy again each collector reconnects its control channel automatically within a short backoff (seconds to a minute or two). The enrollment token each host holds is a durable reconnect credential, so no manual re-enrollment is needed — the fleet flips back to healthy on its own.

If a collector is still offline more than a couple of minutes after the server is back:

  • Confirm the server is actually healthy (/health returns 200) and reachable from the host.
  • A 401 on the handshake now means the token was revoked or the collector was deregistered — not a transient restart. Re-enrol it with a fresh token.
  • Otherwise check the collector’s own journal for the reconnect attempts and any TLS/endpoint error — see the sections above.

Still stuck?

  • Check the server’s audit log under Settings → Audit Log for the enrollment attempt — server-side error messages are richer than what the script can surface to a piped curl.
  • Open the relevant journal: journalctl -u alloy -f (or -u otelcol-contrib) for the collector runtime, journalctl -u linkmesh-agent -f for the onboarding agent.