Unattended updates¶

Start with what this costs. Turning it on hands your vendor unattended root on this machine: hooks run as root, and an update runs the incoming release's migration hook. A compromise of the vendor's registry or signing key becomes root here with no human in the path. Today, the operator pasting a command is a control — a weak one, but a real one — and this removes it.

That is the trade. What you get back is that the fetch, the verification and the downtime happen without anyone being awake for them, on a machine where the release has declared that a failure cannot leave the database somewhere the previous release cannot read.

What the gate promises¶

Not "no human will be required". An update can still stop and page someone: a migration hook that exits non-zero, a health check that never passes, a converge the engine cannot compensate. None of that is inspected here, and all of it already ends in requires-manual-intervention.

What it promises is narrower and is the one that matters at 03:00: the failure will not be the unrecoverable kind. A failed unattended update that compensates back to the previous release is an incident report. One that needs a restore decision is the thing morzer refuses to make automatically, awake or not.

The gate¶

An update installs itself only if all of these hold:

The middle three are one idea: could this update end needing a database restore? They are the rollback assessment run before the update instead of after one, over what the manifest declares rather than over anything measured.

Anything that fails the gate is still fetched, verified, staged and notified. Nothing is silently skipped, and the operator's remaining decision is only the one that costs downtime.

Turning it on¶

morzer config set update.check=true
morzer config set update.channel=oci://registry.example/demo/bundle:stable
morzer config set update.auto_apply=true

update.auto_apply is refused on an installation that does not require signatures — at the moment you set it, not at the tick that would have acted on it. A machine that accepts the setting and then refuses to act every night is worse than one that refuses the setting: you would believe it was armed.

With update.check on, setting a channel installs a <product>-update.timer, and clearing either one removes the timer again: a channel with checking off is followed by nothing. morzer status shows the channel on its following line, with when the timer looks and whether it installs what it finds, or says it is not polled:

  following     oci://registry.example/demo/bundle:stable (on Sun *-*-* 04:00:00, installs what it may)

The maintenance window is an OnCalendar expression¶

The timer ticks daily at 03:30 unless you say otherwise. A window is one setting, and its value is a systemd OnCalendar expression, written into the timer exactly as you typed it:

morzer config set update.schedule='Sun *-*-* 04:00:00'
morzer config unset update.schedule   # back to daily at 03:30

There is no second maintenance-window mechanism, deliberately: systemd already expresses this better than one would, including "the first Sunday of the month" and "weekdays only". The setting is refused if systemd cannot parse the expression, so a typo never becomes a timer that does not load, and systemd-analyze calendar 'Sun *-*-* 04:00:00' shows when an expression next elapses before you set it.

Do not narrow the window with a drop-in. OnCalendar= in a drop-in adds a trigger to the one the manager wrote, so systemctl edit demo-update.timer with OnCalendar=Sun *-*-* 04:00:00 runs updates on Sunday and every night. Earlier versions of this page recommended exactly that; if you followed it, delete the drop-in and use the setting.

RandomizedDelaySec spreads installations across half an hour so a vendor's registry does not see every customer at the same second, and Persistent=true catches up a tick missed while the machine was off. Set the spread with update.randomized_delay, 0 for none; morzer status shows it beside the schedule:

morzer config set update.randomized_delay=10m
morzer config unset update.randomized_delay   # back to 30m, or none in dev mode

A tick that cannot take the lock exits 0¶

Your interactive backup or apply is in the way; the next tick is soon. Queueing would make the start time unpredictable, and a failing unit every time somebody deploys by hand is a unit whose alerts get muted.

A release left staged is also a successful tick. It is the configured behaviour when the gate refuses, and a unit that failed every night because a release is waiting would train you to ignore it.

A release that fails to install is tried once¶

When an unattended install fails, the release stays staged and the timer stops trying it. Each attempt costs a full pre-update backup and a migration run against your database, so retrying on every tick would repeat both and hide the failure behind the next attempt. You get one operation.finished notification for the failure, and morzer status keeps saying the release did not install until something changes:

• The channel moves. Whatever the tag points at next is a new release to the timer, and is installed as normal if the gate admits it.
• You install it yourself with morzer update --to <version>, once you know why it failed.

While an operation is flagged for manual intervention, a tick does nothing at all: it fetches nothing, installs nothing, and fails with the same refusal update gives, naming the operation. Clear the flag with morzer status --clear-intervention once the machine is repaired.

Dev mode: a machine that is a sandbox from birth¶

morzer init --mode dev --product demo --release ./bundle

A sandbox is a machine whose data is disposable and whose purpose is rehearsing what will happen to a real one. It relaxes:

• The recovery gate. Auto-apply installs whatever the channel offers: there is nothing here to be unable to recover.
• Retention. It prunes old releases after every update, which a fast rebuild loop needs — one release directory per build, each with its own images. The images a pruned release pulled go with it, so the image store no longer grows by one build's images per update. An image that a release left on the machine still names stays; see release prune.
• --skip-backup, without --force, and backup.before_update=false, which turns the pre-update backup off for every update, the timer's included.
• Prereleases. They are admissible update candidates. On a production machine they are not: every development build carries a prerelease version, and a check answering "1.4.1-dev.7 is available" is a check nobody reads.
• The update timer's spread. A sandbox's timer has no RandomizedDelaySec unless you set update.randomized_delay, so update.schedule='*:0/5' follows the channel every five minutes rather than every five to thirty-five. The spread exists for a vendor's customers, and a sandbox polls its own vendor's registry.

It relaxes nothing about verification. Signature checking, digest pinning and SHA256SUMS completeness are unchanged, because the sandbox's entire value is fidelity: every relaxation there reduces what a successful rehearsal proves, and what you are rehearsing is a customer's install. Sign dev builds with a dev key the sandbox pins — one minisign -Sm in CI, and it rehearses key handling too.

Mode is fixed when the installation is created¶

Not one-way. No way, through any of the manager's surfaces: not config set, not import, not any command that writes installation state.

The second is the quieter one and it lands when it costs most, which is why neither direction is allowed.

Promotion is backup → fresh init → restore. That already works, and it is the right amount of ceremony for a machine about to hold real data.

installation import is the other moment a mode is chosen, because import reproduces an export wholesale:

morzer installation import ./demo.export.yaml --identity ~/recovery.key --mode dev

That is how you test a customer's backup on a sandbox. It drops everything that would let the sandbox act on production's infrastructure, and this is not optional: an import keeps the original installation id so a lost machine's backups stay restorable, and an export carries credentials so a rebuilt machine can reach what it needs to.

• Backup targets. A sandbox that kept them would push throwaway backups — and fleet rows, which go to the same targets — into the customer's bucket under a matching id.
• Notify targets. A webhook URL is itself the credential, so a sandbox that kept them would page the customer's on-call about a machine that exists in order to be broken.

The drop is a single list rather than a special case per hazard, and it is reported rather than silent.

Importing a dev export as production is refused. With no --mode at all, an import reproduces whatever the export was — a lost sandbox comes back a sandbox.

What is not claimed¶

mode is a field in a JSON file, and root can edit it. Nothing here is tamper-evident, and the immutability above is a statement about the manager's own surfaces. Defending one boolean against an operator who can equally edit the recipient list, the backup targets or the installation id would be defending the wrong thing; root on the machine is outside every threat model in this documentation.

status and doctor mark a sandbox permanently and prominently — not as a first-run notice, because the failure mode is a machine nobody remembers the provenance of.

For vendors¶

Declaring database_schema_produces is what opts your releases into this. See the manifest reference and publishing.

Omitting it is a valid choice, and it costs your customers nothing: their machines still fetch, verify and stage your releases, and still tell them one is waiting.