Upgrade LinkMesh

A LinkMesh deployment has parts that version independently: the server (with its embedded UI), the collector runtime on each managed host (otelcol-contrib supervised by opampsupervisor, or Grafana Alloy), the optional agent on hosts where you run one for onboarding, and the server’s stored state (its database and its internal certificate authority). This page covers how to move all of them from one version to the next without interrupting the fleet.

The good news up front: telemetry never travels through the server, so even while the server is being upgraded your collectors keep running and your data keeps flowing. What an upgrade briefly affects is the control plane — the UI, the API, config push, and fleet status.

The one rule: preserve the server secret

Never lose or change the server secret across an upgrade

The server uses a configured server secret to encrypt its internal certificate authority — the root of trust for the mTLS edge-connector channel. If the server starts up and finds a certificate authority it cannot decrypt (because the secret changed, was dropped from the environment, or the state volume was replaced), it stops with a fatal error rather than guessing. Keep the secret identical across upgrades, and back it up somewhere separate from the server itself.

Concretely, before any upgrade make sure the value the server reads for its secret (its environment variable or mounted secret) is unchanged and that the state volume carries over. On Kubernetes that means the Secret and the PersistentVolumeClaim; on a package install it means the secret in the server’s environment file and the data directory. If you ever need to move the server to fresh storage, restore both the state and the secret together — one without the other is not enough.

Upgrade order: server first

Always upgrade in this order:

1. Server — first, on its own. A newer server understands older collectors and agents, but an older server does not understand newer ones. Upgrading the server first is the only safe direction.

2. Collector runtimes — next, and whenever convenient. Their configuration is delivered by the server through each runtime’s own channel (Alloy over remotecfg, otelcol-contrib over OpAMP), so the runtime binary and its config move independently — there is no coordinated step.

3. Agents (where you run one) — last, at your own pace. The agent is the optional onboarding edge connector, not every host has one; where present it may lag the server by one minor version and keep working normally. The fleet UI flags any agent that has drifted further behind so you know which hosts to attend to.

Upgrade the server

1. Take a state backup. Snapshot the database (or the BoltDB data directory) and confirm you have the server secret recorded separately. This backup is your only way back across a release that changes the data format — see Rollback below.

2. Roll the new version. On Kubernetes, point the deployment at the new image tag and let it roll. On a package install, run your package manager’s upgrade (apt upgrade linkmesh-server / yum update linkmesh-server) and restart the service.

3. Watch the first boot. On startup the server applies any forward data migrations the new version needs and records the new version. Confirm the server reports healthy and the UI loads. If the server refuses to start citing the certificate authority, stop — that is the secret-mismatch guard above, not a transient error; restore the correct secret rather than working around it.

Note

Migrations only run forward. A server will refuse to start against a database written by a newer version than itself, which is what makes a careless downgrade safe-by-default rather than silently destructive.

Upgrade the agents

On hosts where you run the optional onboarding agent, it’s upgraded by the operator on that host — there is no remote “upgrade” button, by design. Use whichever you installed with:

• Package install: apt upgrade linkmesh-agent or yum update linkmesh-agent, then restart the service.
• Install script: re-run the same install one-liner you used to enrol the host; it installs the current version in place. Existing enrollment and certificates are preserved.

You don’t need to do every host at once. Within the one-minor skew window the fleet runs mixed versions happily; work through the hosts the collector list flags as out-of-date first.

Upgrade the collector runtimes

Because the server delivers collector configuration, the runtime upgrades on its own track:

• otelcol-contrib (OpAMP): re-run the install-opamp.sh one-liner on the host. It installs the runtime version LinkMesh currently pins and re-applies the supervised setup; the server keeps driving config across the change.
• Grafana Alloy (remotecfg): upgrade Alloy from its own repository (apt upgrade alloy) and restart it. Its remotecfg block keeps pulling configuration from the server unchanged.

No server-side step is required for either.

Rollback

Server rollback is a version pin: point back at the previous image tag, or downgrade the package, and restart.

Rollback only within the same data version

A rollback is safe only if the previous version reads the same data format. If the upgrade you are undoing changed the data format, the older server will refuse to start against the newer database (the downgrade guard above). In that case the way back is to restore the pre-upgrade backup you took in step 1 — which is why that step is not optional.

Agents and collector runtimes roll back the same way you rolled them forward: install the previous package version, or re-run an installer pinned to the prior version.

Pre-upgrade checklist

1. Server secret is recorded somewhere safe and will be unchanged after the upgrade.

2. The server’s state volume (database / data directory) carries over, or you have a fresh backup of it.

3. You’re upgrading the server first, agents and collectors after.

4. You’ve noted the current version so you know what to pin back to if you need to roll back.