Sleep and wake
When an app sleeps, and what wakes it.
An app with no traffic is snapshotted and suspended. The next request wakes it. This is how a
host holds more apps than it can run at once. Which apps sleep, and
after how long, is set per instance in desired.json.
Desired state
desiredState says whether an instance should be up:
desiredState | Behaviour |
|---|---|
running | Keep the microVM up. |
on-request | Start it on first deploy and on every request that arrives while it sleeps. Let it sleep in between. |
stopped | Stop the microVM. Requests to its hostnames get a 503. |
When it sleeps
activation.sleepWhen says what puts an on-request instance to sleep:
{
"desiredState": "on-request",
"activation": {
"sleepWhen": { "kind": "traffic-idle", "timeoutMs": 900000 }
}
}sleepWhen | The microVM is suspended when |
|---|---|
{ "kind": "traffic-idle", "timeoutMs": N } | Nothing has been sent to it for N ms. A request or stream the proxy is still carrying counts as traffic, however quiet it is. |
{ "kind": "max-lifetime", "ttlMs": N } | It has been up for N ms, busy or not. A request still open at that point is given up to 30 s more to finish. |
{ "kind": "never" } | Never. Only desired.json takes it down. |
Without sleepWhen, an on-request instance sleeps after five minutes idle. Any other
desiredState never sleeps.
Note
- Only an
on-requestinstance may set asleepWhenother thannever. Nothing would wake any other kind, and the next reconcile pass would start it again. A file that does this is rejected as a whole. idleTimeoutMsis the old spelling ofsleepWhen: traffic-idle. A file that sets both is rejected.- A host that has not managed to read an app's traffic lately leaves it up rather than taking
the silence for quiet, and says so in its log.
max-lifetimeruns on the clock either way. - A sleep that does not happen leaves the app up. The host tries again after 30 s, then waits twice as long after each further failure, up to 10 minutes. Traffic in between resets the wait.
What wakes it
Any inbound traffic: an HTTP request on one of its hostnames, or anything on a raw port. The connection is accepted and held, then forwarded once the app's health check passes. The client sees a slow first response, not a refused connection.
Snapshot integrity
Before resuming a restored app, the host sets the guest's wall clock to the current host time. The app stays frozen until that succeeds. If the guest cannot synchronize its clock, the host starts it from cold instead.
Guests started before a host and guest-image upgrade keep running. Their snapshots retain the image version they booted with. On the next wake, a snapshot from a different or unknown guest image is discarded and the app starts from cold with the installed image. Files on its volume survive, but in-memory state does not.
If an older guest's init does not support the freeze protocol, the host refuses sleep and retries after its normal backoff. Redeploy the app to start it with the new guest image.
The host commits a snapshot only after its state, memory and integrity record have been synced to disk. It checks both files' sizes and the state file's SHA-256 hash before restoring. On filesystems supporting fs-verity, memory is sealed during publication: wake compares its fs-verity digest, and the kernel verifies pages as they are read. Other filesystems retain full SHA-256 verification of memory on each wake. Existing SHA-256 snapshots remain readable. An fs-verity page error can surface after resume. The host replaces the memory-faulted VM from cold without spending the app's restart window. A missing, damaged or older snapshot without integrity records is discarded; the request starts the app from cold instead. Files on its volume survive, but in-memory state does not.
Snapshot cost
Both paths read memory during sleep. Only full-file SHA-256 verification reads all memory again before wake; fs-verity measures its digest in constant time. The guest stays paused until the snapshot is committed, so a failed commit can resume it without losing memory.
Restore timings
nibrunner_vm_restore_duration_seconds measures successful native
restores. nibrunner_vm_restore_phase_seconds{phase="..."} breaks the same restores into
histograms with sub-millisecond buckets. App readiness checks and request forwarding follow
the restore and have their own wake and proxy metrics.
phase | Work measured |
|---|---|
snapshot_verify | Integrity verification, including the wait for its blocking worker. |
compatibility | Checking the recorded VMM and guest image against the snapshot. |
jail | Consuming the snapshot stamp and rebuilding the jail. |
resources | Attaching log and cron channels, setting TAP ownership and linking snapshot files. |
spawn | Launching the jailed VMM process. |
load | Loading the snapshot, including waiting for the VMM's API socket. |
resume | The VMM resume API call. |
clock_sync | Synchronizing the guest clock and releasing the tenant freeze. |
neighbour | Refreshing the guest's network neighbour entry. |
cleanup | Removing consumed snapshot files and temporary links. |