# Logs and metrics (/docs/guides/logs-and-metrics)



## reported.json [#reportedjson]

After every pass, the daemon writes `reported.json` in `paths.state_dir` —
`/var/lib/nibrunner/reported.json` on a host the installer configured. It is the state of every
volume, instance, checkpoint and export the document names, and why, when the why is not
obvious: an instance that cannot start, a document the host refused, an export that failed.

```bash
jq '.instances[] | {appId, state, message}' /var/lib/nibrunner/reported.json
```

[The reference](/docs/reference/reported-state) is every field.

## Logs [#logs]

Each app's output goes to `<paths.state_dir>/logs/<appId>.log`, one line per line the program
wrote:

```
2026-09-16T10:41:07.318Z stdout 6f1c…/42 listening on :3000
```

That is when the host saw it, which stream, an id for the microVM's log session and the line's
sequence in it, then the line. A gap where the host could not keep up says how many bytes it
dropped, and a restart of the program inside the guest is a line on `stderr` saying why.

The host keeps the newest 256 MiB of each app, in two files: when `<appId>.log` passes the cap
it becomes `<appId>.log.1`, over the one before it, and a fresh `<appId>.log` starts. `tail -F`
follows across the cut. An app that floods fills its own share of the disk and no more of it,
and `[logs]` in `config.toml` is the share:

```toml
[logs]
keep_mib_per_app = 256
```

The daemon's own log is in the journal:

```bash
journalctl -u nibrunnerd -f
```

`NIBRUNNER_LOG` in the daemon's environment is a `tracing` filter — `debug`, or
`nibrunnerd::domain=trace` — for one restart's worth of detail.

## Metrics [#metrics]

Add `[metrics]` to `config.toml` and the host serves a Prometheus page at `/metrics`:

```toml
[metrics]
listen_address = "127.0.0.1"
port = 9100
```

The page is rendered from the same state `reported.json` is written from, so a scraper and the
file cannot disagree. Every series is prefixed `nibrunner_`: the state of each instance, volume,
checkpoint and export as a one-hot; what a converge, a sleep and a wake took, phase by phase;
the proxy's requests by what it answered with — an app's own answer is `served`, whatever its
status, and the 502 the proxy writes when an app does not answer at all is `unreachable`;
each app's requests by status class and the proxy's TLS handshakes; each app's CPU, memory,
disk and network; the host's slots, memory and capacity.

The one to alert on is `nibrunner_conntrack_entries{of="used"}` against `{of="max"}`: the
kernel's connection-tracking table, which every flow between the proxy and a guest and every
flow a guest opens is an entry of, for its lifetime and 120 s after. Full, the kernel drops
packets for every app on the host and says so only in `dmesg`. The daemon says it in its log at
four fifths, once, and once more when it is back under; [`max_apps`](/docs/guides/host-capacity)
sizes the table.

<Callout title="Good to know">
  `metrics.port` cannot be `proxy.http.port`, a port in the slot range from 21000, or 9091 on a
  `zerofs` host, which ZeroFS holds.
</Callout>
