Reporting & Artifacts

Each Stoney run generates structured output.

JSON Report

Default location:

stoney-report.json

Structure (high-level):

{
  "ok": false,
  "total": 3,
  "passed": 2,
  "failed": 1,
  "results": []
}

Each result typically includes:

• feature
• contract
• id
• ok
• notes
• steps
• optional work_item

PR Comment

On pull requests, Stoney can post:

• Pass/Fail summary
• First N failing checks
• work_item metadata (if provided)
• Step failure details
• Artifact reference

Artifact Upload

If enabled:

upload_artifact: "true"

The full JSON report is uploaded for inspection.

Fail Fast Behavior

If fail_fast: "true" is enabled:

• Stoney stops after the first failing check
• The JSON report and PR comment reflect only what ran up to that failure
• This speeds up CI, but may require reruns to discover later failures

If fail_fast: "false" (default):

• Stoney runs all checks and reports all failures in a single run

Failure Surface

Failures include:

• HTTP mismatches
• SQL invariant violations
• Exec command errors
• Missing work item metadata (if required by policy)

Why Reporting Matters

Stoney is not just a pass/fail tool.

It maps failures to invariants and work items.

This creates:

Work Item → Contract → CI Result → Audit Trail