CI integration

GitHub Actions

Two lines. The action downloads a pinned release, verifies SHA256SUMS with cosign against the release workflow’s identity, and runs the gate:

name: provider-contract
on: [pull_request]

jobs:
  spec-diff:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: Pikopod/spec-diff-action@v1
        id: gate
        with: { old: "git:origin/main:openapi.yaml", new: openapi.yaml }
      - name: Post the report on the PR
        if: always()
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: pikopod pr comment --handoff "${{ steps.gate.outputs.report }}"

What each piece does:

fetch-depth: 0 so git:origin/main:openapi.yaml can be read from object storage. No checkout of the base branch is needed.
The action sets safe.directory, runs spec-diff with --format githubactions so every finding lands inline on the diff, and always writes the JSON report. Inputs fail-on (default ERR), format, handoff and version; outputs findings, report and exit-code. Source and the verification script: Pikopod/spec-diff-action.
pr comment posts one marker-tagged comment and updates it in place on re-runs. Platform, repository, PR number and head SHA are detected from the Actions environment. The token comes from GITHUB_TOKEN or GH_TOKEN, never from a flag. The binary is on the PATH after the action ran.

Without the action, install the release yourself and run the same command; the verification command is under Installation. If the token cannot comment (a read-only token on a fork PR), pr comment degrades: it writes the report to the job summary, or to stdout, and exits 0 with a note on stderr.

GitLab CI

spec-diff:
  image: alpine:3
  script:
    - apk add --no-cache curl git
    - VERSION=0.2.0
    - curl -fsSL "https://github.com/Pikopod/pikopod/releases/download/v${VERSION}/pikopod_${VERSION}_linux_amd64.tar.gz" | tar xz
    - git config --global --add safe.directory "$CI_PROJECT_DIR"
    - ./pikopod spec-diff git:origin/main:openapi.yaml openapi.yaml --fail-on ERR --handoff report.json
  after_script:
    - ./pikopod pr comment --handoff report.json

pr comment detects GitLab from GITLAB_CI, reads the project and merge request IID from the CI variables, and needs GITLAB_TOKEN with api scope. Job tokens usually cannot post merge request notes.

Container image

The image is FROM scratch and runs as an unprivileged user:

- name: Gate on declared drift
  run: |
    docker run --rm -v "$PWD:/work" -w /work \
      ghcr.io/pikopod/pikopod:0.2.0 spec-diff old.yaml new.yaml --fail-on ERR

There is no git inside the image, so use file sources or check the base version out to a path first.

Exit codes in CI

Treat 1 and 2 differently. 1 means the provider changed something that breaks you. 2 means the tool could not run: a document did not parse, a $ref was refused, the checkout was unreadable. A gate that collapses them teaches people to ignore both. See Exit codes.

Gating on recorded traffic too

Once an agent has been observing a provider, add the offline replay gate to the same job. It needs the data_dir (or a copy of it) and no network:

./pikopod agent replay --ci --handoff replay.json

See Replay gate.

Running the sandbox in CI

Scenario runs need no server: pikopod scenario check builds its own engine in-process. The second action in the same repository imports the spec and runs the scenarios:

- uses: Pikopod/spec-diff-action/scenario-check@v1
  with: { sandbox: examplepay, spec: specs/examplepay.json, scenarios: "declines timeouts retry_storm partial_failure" }

Or by hand, as a complete job that fails the build on any failed step, annotates the pull request inline, and keeps the JUnit report as an artifact your test reporter can read:

name: provider-failures
on: [pull_request]

jobs:
  scenarios:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      checks: write
    steps:
      - uses: actions/checkout@v4
      - name: Install pikopod
        run: |
          VERSION=0.2.0
          curl -fsSL "https://github.com/Pikopod/pikopod/releases/download/v${VERSION}/pikopod_${VERSION}_linux_amd64.tar.gz" | tar xz
      - name: Import the provider
        run: ./pikopod import examplepay --spec ./specs/examplepay.yaml --seed ci-fixed
      - name: Run the suite, annotate the diff
        run: ./pikopod scenario check examplepay declines timeouts retry_storm partial_failure incident-14835fa32dfb --format githubactions
      - name: Write the JUnit report
        if: always()
        run: ./pikopod scenario check examplepay declines timeouts retry_storm partial_failure incident-14835fa32dfb --format junit > junit.xml || true
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: pikopod-scenarios, path: junit.xml }

The first scenario check is the gate: it exits 1 on a failed step and 2 when a step could not run, and --format githubactions puts each failure on the pack’s line in the pull request. The second run writes the same result as JUnit for whatever reads test reports in your pipeline; runs are deterministic under a pinned --seed, so the two agree. Pin --seed so transcripts are byte-identical between runs. Packs are looked up in ./scenarios first, then in <data_dir>/scenarios, so a committed scenarios/ directory is the regression suite. See Reports a build can be gated on.

Getting started

Gate: catch declared changes in CI

Sandbox: rehearse failures

Scenarios

Observe: the agent

Reproduce and fix

Operations

Project

GitHub Actions

GitLab CI

Container image

Exit codes in CI

Gating on recorded traffic too

Running the sandbox in CI

​GitHub Actions

​GitLab CI

​Container image

​Exit codes in CI

​Gating on recorded traffic too

​Running the sandbox in CI

GitHub Actions

GitLab CI

Container image

Exit codes in CI

Gating on recorded traffic too

Running the sandbox in CI