Results, reports, and comparisons

Documentation index

Where results go

The default output is .litetraffic/runs/. Every invocation gets a unique run_<timestamp>_<suffix> directory. verify --json prints its run_id; combine that with your output directory to locate the files.

File	Contents
run.json	Target, seed, bundle digest (scenario_sha256), engine, resolved schedule, run and engine-window timestamps (engine_started_at, engine_finished_at when k6 was launched), lifecycle
scenario.lock.json	{"manifest", "engine", "files"}: the parsed manifest, the k6 version string, and the sha256 of the script, of every file in its relative import closure, and of the fixtures.pool file when set, keyed by path inside the bundle
result.json	Mode (verify), seed, verdict, completeness, assertion summaries, metrics, planned vs observed rate, limitations, notes
report.html	Standalone readable report (seed, target, engine, latency, assertions, limitations, notes); open in a browser
events/000001.jsonl	Validated, sequenced assertion events
metrics.jsonl	Raw k6 JSON metrics, if emitted
console.log	Raw k6 console output, if emitted
engine.stdout.log, engine.stderr.log	Captured engine diagnostics
observation.json	Optional final observer expectations, compact recorded actual fields, per-pointer checks (matcher, actual, pass), and expressions (the original ${...} forms of resolved expected values, only when used); a list with one such entry per observation when the manifest uses observations
fixture.json	Optional fixture outcomes: owned_http create/cleanup, or command setup/teardown argv, exit code, duration and stderr tail
server.json, server/	Only with verify --capture: container summary, raw server/NAME.log logs and server/stats.jsonl samples (server capture)
artifacts.json	Finalization manifest, written last: {"schema_version", "run_id", "files", "total_bytes"}, where files lists every other run file as {path, bytes, sha256} sorted by path and total_bytes is their sum. It excludes itself; the max_artifact_bytes budget is summed over the same file set

The controller restricts permissions on directories and files it creates. Raw engine output may still contain anything the script logs. Treat the entire run directory as potentially sensitive and review it before sharing. Scripts and imported files are recorded by hash in scenario.lock.json but not copied into the run directory.

Server capture

verify --capture docker:api,worker writes server.json:

{"schema_version": 1, "source": "docker", "problems": [],
 "containers": {"api": {"baseline_cpu_percent": 0.4, "peak_cpu_percent": 87.1, "peak_at": "2026-09-28T10:00:04+00:00",
   "baseline_mem_mb": 120.5, "peak_mem_mb": 188.0, "log_lines": 912, "error_lines": 37, "truncated": false,
   "signatures": [{"signature": "ERROR db timeout after <n>ms for <uuid>", "count": 37,
     "first_seen": "2026-09-28T10:00:03.1Z", "example": "ERROR db timeout after 5000ms for 1b4e..."}]}}}

Baseline is the first docker stats sample; peaks are the maxima over the run (null without samples). Memory is in MiB. An error line matches error, critical, fatal, panic, timed out, refused (as words, any case), Traceback, Exception, or a 5xx status. A signature is the line with quoted strings, timestamps, UUIDs, hex and numbers of four or more digits replaced, so repeats collapse; the top 8 per container are kept with the first matching line as example (at most 300 characters). Signatures, examples and the raw server files are scrubbed of the run’s secrets. problems lists what could not be captured; when it is not empty, result.json gets one Server capture incomplete: ... limitation, appended after the verdict is decided, so capture never changes the verdict. Without --capture none of these files exist.

Verdict and lifecycle

Verdict	Meaning
pass	Declared assertions passed with complete expected evidence and no reported run limitation
fail	At least one declared business assertion definitely failed
inconclusive	Evidence/progress was incomplete without a definitive failure, such as a timeout
error	Engine/lifecycle, budget/fixture failure, or an unreachable target prevented a clean evaluation

Lifecycle is finished, timed_out, cancelled, or crashed, independently of the verdict. While a run is in progress, run.json records its current stage as it happens: preparing (fixture setup), running (k6 launched), observing (final observer and fixture cleanup), then finalizing, before the final lifecycle is written. Ctrl-C at any stage finalizes the run as cancelled (CLI exit 130): a cancel during fixture create skips the engine; a cancel during the engine or observer still cleans up a created fixture, and the observer assertion becomes unknown; a cancel during fixture cleanup reports fixture cleanup cancelled; fixture ID may remain and makes the verdict error. A cancel anywhere else once the run directory exists (evidence parsing, redaction, finalizing, report writing) still runs owed fixture cleanup and teardown, then writes a best-effort result.json with lifecycle cancelled, verdict inconclusive, the limitation run cancelled by user, and empty assertions and metrics; evidence files already written are kept, after every one is scrubbed of the run’s credentials (a file that cannot be scrubbed is deleted and named in the limitations). For example, a timed-out run can retain a definite failing assertion; it can never pass. A timed-out run reports engine stopped after its E-second share of the S-second budget, where E is max_seconds minus the fixture (10 s) and observer (5 s) reserves the scenario declares. k6 exit status 99 (a script threshold was crossed) is not a crash: the run stays finished, reports the limitation k6 thresholds breached, and its verdict is fail unless an error condition applies. Any other non-zero k6 exit is crashed. An observer that cannot respond yields unknown evidence. A failed configured fixture cleanup makes the verdict error.

When k6 made requests but every http_req_failed point is a transport-level failure (k6 tag status of "0", meaning no HTTP response arrived; HTTP 4xx/5xx responses also carry an error_code tag but are not counted as transport failures), the verdict is error, failing assertions become unknown, and limitations contains target unreachable: all N requests failed before an HTTP response. metrics.http_req_failed_rate.transport counts those failures when any occur. A run that receives any HTTP response, including HTTP 500, is judged normally.

For ordinary journey assertions, each declared assertion must have exactly one sample per planned journey, and every sample must carry a distinct non-empty logical_key: an unkeyed sample adds evidence without journey identity for <assertion> and a repeated key adds duplicate evidence for <key>, and either makes the assertion unknown (see scenarios). Each final-observer assertion instead receives one aggregate observation. Missing samples and delivered/planned journey mismatches prevent a pass. Inspect limitations, completeness, and each assertion’s sample count when diagnosing a result.

Each entry in result.json assertions has id, status, and samples. A failing journey assertion also has failures: up to the first three failing samples, each {sequence, logical_key, expected, actual, detail} taken from the event (null when the event omitted a field; sequence matches events/000001.jsonl). A final-observer assertion that was evaluated also carries the observer’s expected and actual values from observation.json. A final-observer assertion that is unknown carries the observer’s reason instead (for example observer HTTP 503). When the target is unreachable, failing rows become unknown and drop these details. report.html shows a “Failing samples” table (sample, logical key, expected, actual, detail) for every failing assertion; all values are HTML-escaped, and a value longer than 300 characters is collapsed into an expandable <details> whose summary shows its first 80 characters and its length.

Engine event or metric lines that cannot be parsed (invalid JSON, a non-object value, or a metric record whose data field is present but is not an object) are skipped and reported as an ignored N malformed event record(s) or ignored N malformed metric record(s) limitation; the run is still finalized and result.json is still written.

metrics.write_attempts counts write requests (k6 http_reqs tagged POST, PUT, PATCH or DELETE, plus fixture create and cleanup), and metrics.vus_max is the highest k6 vus_max point when k6 reports one. Exceeding max_requests, max_write_attempts or max_in_flight adds a ... budget exceeded: N > M limitation and makes the verdict error; see safety.

result.json metrics.http_req_duration_ms holds samples, average, p50, p95, and max when k6 supplies durations. metrics.by_operation breaks HTTP points down by the k6 operation request tag: each entry holds samples (duration count), p95 (ms), and failed_rate (share of http_req_failed points), with null when the operation had no points of that kind. Points without an operation tag are grouped under _untagged. The key is absent when k6 recorded no HTTP duration or failure points. metrics.overlap is a client-side overlap check: it shows requests were in flight together, not that the server executed them concurrently. It maps each operation (same operation tag grouping) to the peak number of its requests that were open at the same instant, computed from each timestamped http_req_duration point as the interval [time - duration, time]; a request ending exactly when another starts does not count as overlapping. The key is absent when no duration point carried a timestamp. When a journey declares min_overlap and the observed peak for an operation (0 if it never ran) is below the minimum, the run gets the limitation concurrency not achieved for OP: observed M < N, so it cannot pass (it is inconclusive unless something else makes it fail or error). report.html shows the seed, target, engine version, HTTP avg/p50/p95/max (or Unavailable), the latency sample count, HTTP error rate, journeys, throughput, and a Concurrency table of peak in-flight requests per operation. The dashboard run page lists overlap with the other metrics.

When any journey declares expected_statuses, metrics.unexpected_http_failure_rate holds samples (all http_req_failed points), failed, and rate. failed counts http_req_failed points with value 1 whose k6 status tag is not listed for the point’s operation tag; operations without a declaration keep every k6 failure, and transport failures (status "0") always count. metrics.http_req_failed_rate is unchanged. The key is absent when no journey declares expected_statuses or k6 recorded no failure points. When it is present, report.html adds an Unexpected HTTP failure rate row, the verify text summary adds unexpected HTTP failure rate: X% (FAILED/SAMPLES), and the dashboard run page lists it with the other metrics. The rate is descriptive and does not affect the verdict.

planned_journeys_per_second is the resolved schedule’s admitted journeys divided by its total seconds; compare it with the observed metrics.iterations_per_second (iterations divided by the engine window only, from engine_started_at to engine_finished_at in run.json; fixture setup, observation and cleanup are excluded, as they are from metrics.http_reqs_per_second, while metrics.elapsed_seconds stays the whole run’s wall-clock time). When k6 reports a nonzero dropped_iterations metric, limitations contains k6 dropped N iterations (under-delivered load) and the run cannot pass.

Every result also has notes, which list what this preview never measures: per-arrival lateness not measured and workload is synthetic (no traces supplied). Notes are separate from limitations and do not affect completeness or the verdict. Those are observations, not automatic performance promises. Expected application rejections may contribute to k6’s HTTP failure rate even when the business assertion passes; declare them in expected_statuses to get unexpected_http_failure_rate alongside it.

Journey funnel

When any journey declares stages (scenarios), result.json has a journeys object keyed by journey name (absent otherwise):

"journeys": {"purchase": {
  "started": 12,
  "stages": [{"name": "paid", "reached": 12}, {"name": "retried", "reached": 12}, {"name": "confirmed", "reached": 9}],
  "stalled": [{"after": "start", "count": 0}, {"after": "paid", "count": 0}, {"after": "retried", "count": 3}],
  "undeclared": []
}}

started is the k6 iterations count when the scenario has one journey; with several journeys k6 does not split iterations, so it is the number of distinct journey keys that reported any stage. reached counts distinct journey keys that reported that stage; a later stage says nothing about earlier ones, so each stage counts only what was reported. stalled has one entry per position: journeys that started ("after": "start") or reached a stage but did not report the next one. undeclared lists stage names the journey reported but does not declare; each adds the limitation journey NAME reported undeclared stage(s): ..., and names no single journey can own add stage events match no single journey: .... Stage events are written to events/000001.jsonl in order with the assertion events.

The funnel is evidence, not a pass/fail rule, with one exception: when a run would otherwise pass but zero journeys reached a journey’s final stage, the verdict is inconclusive with the limitation No journey reached its final stage 'NAME': the checks passed on journeys that never finished. A green run where nothing finished is a silent failure, not a pass. The dashboard shows the funnel on the run page’s Journeys tab, the plain-English explanation adds a “Where journeys stopped” section, and diff adds a journeys object with per-stage baseline, candidate, and change reached counts for journeys and stages both runs recorded; it does not affect the comparison verdict.

Repeat summaries

--repeat 3 --seed 42 executes seeds 42–44 and writes series_*.json alongside the run directories. The summary includes every result, requested/completed counts, aggregate verdict, and whether verdict/lifecycle outcomes were consistent. Cancellation stops the series early.

--repeat 5 --seed 42 --same-seed runs seed 42 five times instead, to measure run-to-run noise with the timing held fixed; the summary records same_seed: true. --same-seed needs --repeat of at least 2.

Every summary has dispersion with p95_ms (from metrics.http_req_duration_ms.p95), http_req_failed_rate (from metrics.http_req_failed_rate.rate), and http_reqs_per_second. Each holds min, max, mean, and sample stdev over the runs that reported that metric, computed with Python’s statistics module. stdev is null when only one run reported the metric, and the entry is null when none did.

Comparison

diff requires equal bundle digest, seed, engine version string, and resolved schedule. Incompatible inputs produce an inconclusive comparison. The bundle digest is the sha256 of the canonical manifest JSON (sorted keys, compact separators, so reformatting whitespace does not change it) plus the sha256 of the script and each file it reaches through relative imports (./ or ../ specifiers in import/export ... from, import() and require()), so editing journey code or a local helper makes runs incompatible. An import of the bundled ./litetraffic/runtime.js helper hashes the helper shipped with LiteTraffic, so upgrading to a different helper also makes runs incompatible. A fixtures.pool file, fixtures.command.inputs files, and bundle files named by a fixtures.command argv element are hashed too. Module imports such as k6/http and remote URLs are not hashed. Target environment and fixture equivalence remain the operator’s responsibility.

The comparison fails whenever the candidate verdict is fail (including when the baseline also failed), or the optional p95 gate reports a regression. The gate compares the unrounded p95 change against the threshold; performance.p95.change_percent is rounded to three decimals for display only, so a 20.0001% change fails a 20% gate even though it is shown as 20.0. It is inconclusive when the candidate verdict is error/inconclusive, when the candidate completed fewer iterations or HTTP requests than the baseline (candidate delivered less work), when the p95 gate is inconclusive or unavailable, or, without a gate, when the candidate has no latency samples. Otherwise it passes. reasons lists why the verdict is not pass; for incompatible runs it contains only the compatibility mismatch, and correctness.assertion_regressions is empty. Individual assertion regressions (baseline pass, candidate not pass) are listed for compatible runs but are not a separate aggregate gate.

performance.by_operation reports baseline_p95_ms, candidate_p95_ms, and change_percent for each operation present with a p95 in both runs (change_percent is null when the baseline p95 is not positive). Per-operation changes are descriptive and do not affect the verdict or the p95 gate, which uses the overall p95. When either run has metrics.unexpected_http_failure_rate, performance.unexpected_http_error_rate holds baseline, candidate (null when that run lacks it), and change_percentage_points, shaped like performance.http_error_rate; it is descriptive and does not affect the verdict.

A p95 gate requires a positive baseline p95 and at least 200 duration samples in each run. A smaller sample count produces an inconclusive gate. Run in comparable environments, preserve reviewed scripts, and avoid interpreting proxy or machine noise as a code regression. Without a p95 gate, the comparison reports performance differences without grading them.