Fixture replay and regression comparison
Distinguish regressions, improvements, unchanged outcomes and Unknown dimensions in deterministic fixture replay.
Inputs and behavior
The Python 3.10+ standard-library example returns predetermined fixture records. Contract-invalid input is rejected. A valid comparison is a record-level result, not a physical-system test.
Baseline and candidate: same suite identity and scope. Dimension A: candidate regresses against the baseline. Dimension B: candidate improves. Overall comparison retains the regression; missing evidence stays Unknown.
Interpret the result
The package runs no simulator, physical system, trained policy, sensor or safety controller. It establishes no physics fidelity, robot safety or real-policy performance. Fixture metadata and digests bind records but do not authenticate their origin. CLI success means a valid comparison, not acceptance or safety.
Unknown is unresolved evidence. Do not treat it as a pass or permission to take a side effect.
This website provides documentation. It exposes no execution endpoint, account, upload form or hosted API.