Skip to content
LoomCache

Jepsen Testing

LoomCache ships a Java Jepsen-style framework under loom-server/src/test/java/com/loomcache/server/jepsen. It is not an integration with the Clojure Jepsen toolchain — histories, checkers, and nemeses are pure Java.

Methodology

Section titled “Methodology”

Each scenario:

  1. Generates operations — concurrent clients drive reads/writes/CAS/locks against the cluster.
  2. Injects faults — JepsenNemesis simulates partitions, leader crashes, clock skew, disk failures, flapping processes.
  3. Records a history — JepsenHistory captures invocation and completion timestamps.
  4. Verifies consistency — WglLinearizabilityChecker (Wing-Gong algorithm) or model-specific checkers validate the history.

Framework

Section titled “Framework”
  • JepsenTestHarness — wires workload + nemesis + cluster.
  • JepsenCluster / JepsenRealCluster — in-process vs. multi-node cluster.
  • JepsenClient / JepsenRealClient — op generators.
  • JepsenNemesis — failure injector.
  • JepsenHistory / JepsenReport — history and reporting.
  • WglLinearizabilityChecker — linearizability checker.
  • model/ — RegisterModel, CounterModel, QueueModel, LockModel, SetModel.

Scenario categories

Section titled “Scenario categories”

Each class under tests/ is one scenario.

tests/linearizability/

Section titled “tests/linearizability/”

CasLinearizabilityTest, CounterLinearizabilityTest, LockLinearizabilityTest, MultiRegisterLinearizabilityTest, QueueLinearizabilityTest, RegisterLinearizabilityTest.

tests/partition/

Section titled “tests/partition/”

SymmetricPartitionTest, AsymmetricPartitionTest, BridgePartitionTest, PartialPartitionTest, PartitionHealRecoveryTest, PartitionDuringElectionTest, RepeatedPartitionCycleTest.

tests/disk/

Section titled “tests/disk/”

CorruptWalTest, FsyncFailureTest, FullDiskTest, PartialWriteTest, SlowDiskTest.

tests/clock/

Section titled “tests/clock/”

ClockSkewTest, ClockJumpTest, ClockSkewLeaderLeaseTest.

tests/process/

Section titled “tests/process/”

CrashDuringWalWriteTest, CrashDuringCompactionTest, FlappingNodeTest, FollowerCrashRecoveryTest, GracefulVsKillTest, LeaderCrashRecoveryTest, MajorityCrashTest.

tests/combined/

Section titled “tests/combined/”

ClockSkewPlusNetworkDelayTest, MembershipChangeDuringPartitionTest, PartitionPlusLeaderCrashTest, RollingRestartUnderLoadTest.

Scope of guarantees

Section titled “Scope of guarantees”
  • The real-cluster suite drives actual Raft, TCP, WAL, and snapshots.
  • The in-process stubs verify the checker logic but don’t touch the network stack.
  • loom-integration-tests also includes RealClusterLinearizabilityTest as a high-level smoke.

Running

Section titled “Running”
Terminal window
mvn -pl loom-server test -Dtest='com.loomcache.server.jepsen.tests.**.*'
mvn -pl loom-server test -Dtest=RegisterLinearizabilityTest
mvn -pl loom-server test -Dtest=LeaderCrashRecoveryTest

Most scenarios complete in 30–120 s. Combined disk/clock tests can take several minutes — they’re annotated with long timeouts in-source.

Reporting

Section titled “Reporting”

JepsenReport emits a human-readable summary plus a full history trace for failing runs. Replay the trace through WglLinearizabilityChecker to reproduce and debug locally.

For the broader correctness story (Raft invariants, durability, near-cache coherence), see the architecture overview.