Skip to main content
Severity in spec-diff is derived by law. One function maps the shape of a change onto a level, so “breaking” means the same thing on every endpoint and on every provider, and changing the rule changes every verdict at once. A test over the whole catalogue checks that every emitted level is the law’s answer, and a check that is not in the inventory panics at its emit site.

​
The inputs

​
The law

One clause applies after the table: if the claim is Uncertain and the level came out ERR, it becomes WARN. An extracted or low-confidence fact can never page as an error. In words: narrowing what the caller may send, or what the caller is guaranteed to get, is breaking. Widening is informational, except widening a response the consumer may switch on exhaustively, which is a warning. Anything incomparable is breaking. An announced sunset softens a break to a warning. A convention consumers already tolerate (ignoring unknown fields, an error status vanishing) softens it to info. An uncertain fact is never an error.

​
Why a removed response field is WARN

Withdrawing a presence guarantee on a response field, whether by removing the field or by making it optional, is WARN in the declared diff. The response set grew while a guarantee was withdrawn, and consumers ignore unknown fields by dominant convention. The two spellings of the same withdrawal (response-required-property-removed and response-property-became-optional) carry the same level by a test. Inside pikopod up, traffic evidence raises it: if the agent has seen consumers receive that field in at least 98% of samples, the declared finding becomes ERR with the note that consumers receive this field today. See Spec watch.

​
The check inventory

Fifty-five check IDs, sixty-one variants. The level column is the law’s answer with the static Tolerated guard only. Emit sites may add DeprecatedHonored or Uncertain at runtime.

​
Endpoints and auth

​
Parameters and request bodies

​
Responses

​
Schemas

Applies to request and response schemas, walked to a depth of 32. allOf is flattened. oneOf and anyOf members are not diffed pairwise, because their identity is positional and a reorder would flood false findings. Only presence, kind and count of variants are compared.

​
Fingerprints

Every finding carries a fingerprint: fp_ plus the first 12 hex characters of a SHA-256 over the check ID, method, template and the check’s identity arguments, joined with NUL bytes. The level, the detail text and the source position are deliberately excluded, so the same divergence has the same fingerprint across runs and re-fetches, and a moved line never changes the identity. The declared-by-observed join can raise the level and extend the detail; the fingerprint never moves.

​
What the tests enforce

  • Every emitted level equals the law’s answer for that check’s effect, direction, scope and guards. The list of allowed deviations ships empty.
  • Every check ID emitted anywhere in the code is in the inventory, and the inventory has no entry without an emit site.
  • More changes can never lower the verdict. An added INFO never lowers an existing ERR.
  • Every check names an inverse, or appears in a documented list of asymmetries with a written reason. Reversing a fixture emits the inverse at the law’s level.