Guides

Volumes

What a volume starts with, where volumes are stored, and how to back one up.

Every app has one volume. It is the writable top of the app's root filesystem: all writes go there, and it is the only thing that gets backed up. Declare it in volumes and point the instance at it with volumeId.

"volumes": [
  { "volumeId": "vol-1", "appId": "app-1", "sizeBytes": 8589934592, "desiredState": "present" }
]

Seeding a volume from an archive

A volume is formatted empty the first time it appears in desired.json. To start it with content, name an archive in the artifact store and where to unpack it:

"volumes": [
  {
    "volumeId": "vol-1", "appId": "app-1", "sizeBytes": 8589934592, "desiredState": "present",
    "initialContents": {
      "digest": "<sha256 of the archive, lowercase hex>",
      "objectKey": "seeds/app-1",
      "destinationPath": "/app/data"
    }
  }
]
  • The archive is a tar, gzipped or not, or a zip.
  • Entries are unpacked under destinationPath: nested/hello.txt becomes /app/data/nested/hello.txt. The directory and everything in it are owned by uid 65534.
  • Files the archiver added on its own are skipped: .DS_Store, __MACOSX, Thumbs.db, editor swap files.
  • The volume fails, with the reason in reported.json, if an entry escapes destinationPath, a zip contains a symlink, or the archive does not fit.

Note

The archive is unpacked once, when the volume is formatted. Changing or removing initialContents on an existing volume has no effect.

Storage backends

volumes.backend in config.toml selects one of two backends. The reference lists every key of both.

local-file

[volumes]
backend = "local-file"
storage_prefix = "host-1"

Volumes are sparse files under paths.state_dir. There is nothing else to configure. A [volumes.zerofs] section on this backend is an error, not ignored. This is the installer's default.

The limitation: volumes do not survive the machine. Checkpoints and exports are not supported on this backend, because they are taken in the object store.

zerofs

[volumes]
backend = "zerofs"
storage_prefix = "host-1"

[volumes.zerofs]
binary = "/opt/nibrunner/bin/zerofs"
config_file = "/etc/zerofs/config.toml"
mount_path = "/mnt/zerofs"
nbd_socket_path = "/run/zerofs/nbd.sock"
ninep_socket_path = "/run/zerofs/9p.sock"
rpc_socket_path = "/run/zerofs/rpc.sock"
storage_url = "s3://my-bucket/host-1"
cache_dir = "/data/zerofs"
cache_disk_gib = 200
cache_memory_gib = 2
checkpoint_runtime_dir = "/run/zerofs-checkpoint"
checkpoint_config_file = "/etc/zerofs/checkpoint.toml"
checkpoint_cache_dir = "/data/zerofs-checkpoint"

Volume blocks live in an object store and reach the guest over NBD, through ZeroFS. install generates ZeroFS's configuration files and systemd units from these keys, so this section is the only place to configure it. The object-store credentials and the encryption password go in host.env.

  • Size cache_disk_gib for the disk it is on. It shares that disk with the snapshots in paths.snapshot_dir. Losing a snapshot costs one cold boot. Filling the disk breaks the filesystem every app on the host runs from.
  • Use whole gibibytes. The daemon subtracts cache_memory_gib from the memory it can promise to guests, and reads the value back from the file it generated, which truncates fractions.
  • Slot N uses /dev/nbdN. install loads the nbd module with max_apps + 1 minors. If the module is already loaded with fewer, it is not reloaded, because that would detach every device under every guest. Raising max_apps on this backend therefore needs a reboot.

Warning

volumes.storage_prefix is per host, not per app. Every volume on the host shares it. Deleting the prefix destroys all of them.

Checkpoints and exports

On the zerofs backend, desired.json can take a point-in-time copy of a volume and bundle it as an archive:

"checkpoints": [
  { "checkpointId": "ckpt-1", "volumeId": "vol-1", "desiredState": "present" }
],
"exports": [
  {
    "exportId": "exp-1", "appId": "app-1", "volumeId": "vol-1",
    "format": "tar-gzip",
    "objectKey": "exports/app-1/bundle.tar.gz", "desiredState": "present",
    "environment": { "DATABASE_URL": "sqlite:///app/data.db" }
  }
]
  • A checkpoint is a copy of the volume at the moment it was taken. It stays in the object store until desiredState is absent.
  • An export freezes the guest's filesystem, takes its own checkpoint, and writes a gzipped tar to objectKey in exports.store_url. The tar holds the volume's contents under data/ and environment as .env, plus the original crontab when the app has registered cron jobs. The crontab preserves comments and environment assignments; .env and crontab have mode 0600. Empty tables and tables containing only comments or environment assignments are omitted. To start a new volume from it, point initialContents at an archive of the data/ contents, with destinationPath /.

After restoring the volume, install the exported crontab with crontab /path/to/crontab inside the new deployment.

Each export selects its required format: tar-gzip or directory. For a local consumer that builds its own bundle, set "format": "directory" in that export request. The host publishes the prepared data/, .env and crontab tree without creating an archive. objectKey must be a single filename, such as exp-1.export; the directory is atomically renamed into the local store, so staging and the store must share a filesystem. A repeated publication of the same key retains the first completed directory. sizeBytes is null because no archive was produced. The consumer owns the published directory's cleanup. A single local store supports both formats; an S3 store accepts tar-gzip only. Unsupported destinations fail before the guest is frozen.

reported.json reports the state of each checkpoint and export, with the reason on failure.

On this page