Class PersistentRaftLog

java.lang.Object

com.loomcache.server.persistence.PersistentRaftLog

All Implemented Interfaces:

AutoCloseable

public class PersistentRaftLog
extends Object
implements AutoCloseable

Persistent Raft Log that wraps RaftLog with WAL persistence.

On construction, recovers state from WAL and snapshot files. On append: writes to WAL before updating in-memory log. On compactUpTo: creates a snapshot and truncates WAL.

Thread-safe wrapper around RaftLog.

Constructor Summary

Constructors

Constructor	Description
PersistentRaftLog(String nodeId, Path walDirectory)	Constructor: initializes persistent log with recovery.
PersistentRaftLog(String nodeId, Path walDirectory, @Nullable LoomMetrics metrics)	Constructor: initializes persistent log with recovery and optional metrics.
PersistentRaftLog(String nodeId, Path walDirectory, @Nullable LoomMetrics metrics, ClusterDataRecoveryPolicy recoveryPolicy)
PersistentRaftLog(String nodeId, Path walDirectory, @Nullable LoomMetrics metrics, ClusterDataRecoveryPolicy recoveryPolicy, long validationTimeoutSeconds, long dataLoadTimeoutSeconds)
PersistentRaftLog(String nodeId, Path walDirectory, @Nullable LoomMetrics metrics, ClusterDataRecoveryPolicy recoveryPolicy, long validationTimeoutSeconds, long dataLoadTimeoutSeconds, int recoveryParallelism)

Method Summary

Modifier and Type	Method	Description
long	append(long term, byte[] command)	Append a new entry with WAL persistence.
void	appendAll(List<LogEntry> newEntries)	Append multiple entries with WAL persistence (batch operation).
void	close()	Close and release resources held by the persistent log.
void	compact(long throughIndex)	Compact the WAL directly by truncating entries up to and including throughIndex.
void	compactUpTo(long snapshotIndex, long snapshotTerm)	Compact the log by removing entries up to snapshotIndex.
void	compactUpTo(long snapshotIndex, long snapshotTerm, long confirmedDurableSnapshotIndex)	Compact the log by removing entries up to snapshotIndex.
List<LogEntry>	getEntriesFrom(long startIndex)	Get entries from a starting index to the end of the log (inclusive).
@Nullable LogEntry	getEntry(long index)	Get a log entry at a specific index.
long	getLastDurableSnapshotIndex()	Get the index of the last durably persisted snapshot.
long	getLastIndex()	Get the index of the last entry in the log.
long	getLastTerm()	Get the term of the last entry in the log.
protected RaftLog	getRaftLog()	Get the underlying RaftLog (for testing/debugging).
long	getSnapshotIndex()	Get the index of the last snapshot.
protected SnapshotStore	getSnapshotStore()	Get the snapshot store (for testing).
long	getSnapshotTerm()	Get the term of the last snapshot.
long	getTermAt(long index)	Get the term at a specific index.
protected WalWriter	getWalWriter()	Get the WAL writer (for testing).
void	saveSnapshot(long index, long term, byte[] data)	Save a state machine snapshot to persistent storage.
int	size()	Get the log size (number of in-memory entries).
void	sync()	Force all pending WAL writes to disk via fsync.
void	truncateFrom(long fromIndex)	Truncate log from a given index onwards (inclusive).

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PersistentRaftLog

public PersistentRaftLog(String nodeId,
 Path walDirectory)
                  throws IOException

Constructor: initializes persistent log with recovery.

On startup: 1. Loads the latest snapshot (if exists) 2. Reads the WAL and recovers log entries 3. Reconstructs in-memory RaftLog state

Parameters:

nodeId - The node identifier (must not be null)

walDirectory - Directory for WAL files (must not be null)

Throws:

IOException - If I/O fails

IllegalArgumentException - if arguments are invalid

PersistentRaftLog

public PersistentRaftLog(String nodeId,
 Path walDirectory,
 @Nullable LoomMetrics metrics)
                  throws IOException

Constructor: initializes persistent log with recovery and optional metrics.

On startup: 1. Loads the latest snapshot (if exists) 2. Reads the WAL and recovers log entries 3. Reconstructs in-memory RaftLog state

Parameters:

nodeId - The node identifier (must not be null)

walDirectory - Directory for WAL files (must not be null)

metrics - Optional LoomMetrics instance to record recovery metrics

Throws:

IOException - If I/O fails

IllegalArgumentException - if arguments are invalid

PersistentRaftLog

public PersistentRaftLog(String nodeId,
 Path walDirectory,
 @Nullable LoomMetrics metrics,
 ClusterDataRecoveryPolicy recoveryPolicy)
                  throws IOException

Throws:

IOException

PersistentRaftLog

public PersistentRaftLog(String nodeId,
 Path walDirectory,
 @Nullable LoomMetrics metrics,
 ClusterDataRecoveryPolicy recoveryPolicy,
 long validationTimeoutSeconds,
 long dataLoadTimeoutSeconds)
                  throws IOException

Throws:

IOException

PersistentRaftLog

public PersistentRaftLog(String nodeId,
 Path walDirectory,
 @Nullable LoomMetrics metrics,
 ClusterDataRecoveryPolicy recoveryPolicy,
 long validationTimeoutSeconds,
 long dataLoadTimeoutSeconds,
 int recoveryParallelism)
                  throws IOException

Throws:

IOException

Method Details

append

public long append(long term,
 byte[] command)
            throws IOException

Append a new entry with WAL persistence.

Durability guarantee: Entry is written to WAL before in-memory log is updated, ensuring that acknowledged writes persist even after crashes. If WAL write fails, neither the in-memory nor persistent state is modified.

Thread-safe via ReentrantLock.

Parameters:

term - The Raft term of the entry

command - The serialized state machine command (may be null or empty)

Returns:

The index assigned to this entry (1-based)

Throws:

IOException - If WAL write fails (entry not persisted)

IllegalStateException - If log is closed

appendAll

public void appendAll(List<LogEntry> newEntries)
               throws IOException

Append multiple entries with WAL persistence (batch operation).

The accepted suffix is durably rewritten to the WAL before the in-memory log is updated. If validation or the rewrite fails, neither state changes.

Thread-safe via ReentrantLock.

Parameters:

newEntries - The entries to append (may be empty)

Throws:

IOException - If any WAL write fails

IllegalStateException - If log is closed

NullPointerException - If newEntries is null

sync

public void sync()
          throws IOException

Force all pending WAL writes to disk via fsync.

Ensures that all previously appended entries are durably persisted. This is called after the leader commits entries to ensure durability before acknowledging writes to clients.

Thread-safe via ReentrantLock.

Throws:

IOException - If fsync fails

IllegalStateException - If log is closed

getEntry

public @Nullable LogEntry getEntry(long index)

Get a log entry at a specific index.

Delegates to the underlying RaftLog (thread-safe).

Parameters:

index - The log index (1-based)

Returns:

The LogEntry, or null if index is out of bounds or before snapshot

getEntriesFrom

public List<LogEntry> getEntriesFrom(long startIndex)

Get entries from a starting index to the end of the log (inclusive).

Delegates to the underlying RaftLog (thread-safe). Returns an immutable collection to prevent external modification.

Parameters:

startIndex - The starting index (1-based)

Returns:

An immutable list of entries, or empty list if startIndex is out of bounds

getLastIndex

public long getLastIndex()

Get the index of the last entry in the log.

Returns:

The last log index (1-based), or snapshot index if log is empty, or 0 if empty

getLastTerm

public long getLastTerm()

Get the term of the last entry in the log.

Returns:

The last log term, or snapshot term if log is empty, or 0 if empty

getTermAt

public long getTermAt(long index)

Get the term at a specific index.

Parameters:

index - The log index (1-based)

Returns:

The term at the index, the snapshot term if index is snapshot index, or 0 if index is before the snapshot or out of bounds

truncateFrom

public void truncateFrom(long fromIndex)

Truncate log from a given index onwards (inclusive).

Delegates to the underlying RaftLog (thread-safe).

Parameters:

fromIndex - The index to start truncating from (1-based, inclusive)

compactUpTo

public void compactUpTo(long snapshotIndex,
 long snapshotTerm,
 long confirmedDurableSnapshotIndex)
                 throws IOException

Compact the log by removing entries up to snapshotIndex.

This is called after creating a snapshot of the state machine to bound memory usage. Performs two operations: 1. Compact the in-memory RaftLog (discard entries before snapshotIndex) 2. Truncate the WAL file to only keep entries after the snapshot index

Safety invariant: A snapshot at or above snapshotIndex must have been durably persisted (via saveSnapshot(long, long, byte[])) before calling this method. If no durable snapshot covers the compaction range, an IllegalStateException is thrown to prevent WAL truncation that could cause an unrecoverable gap on crash.

Thread-safe via ReentrantLock.

Parameters:

snapshotIndex - The index to compact up to (1-based)

snapshotTerm - The Raft term of the entry at snapshotIndex

confirmedDurableSnapshotIndex - The index of the durably persisted snapshot that the caller guarantees covers all state up to snapshotIndex

Throws:

IOException - If WAL truncation fails

IllegalStateException - If log is closed or no durable snapshot covers the compaction range

compactUpTo

public void compactUpTo(long snapshotIndex,
 long snapshotTerm)
                 throws IOException

Compact the log by removing entries up to snapshotIndex.

Uses the internally tracked lastDurableSnapshotIndex to verify that a snapshot covering the compaction range has been durably persisted via saveSnapshot(long, long, byte[]) before allowing WAL truncation.

Parameters:

snapshotIndex - The index to compact up to (1-based)

snapshotTerm - The Raft term of the entry at snapshotIndex

Throws:

IOException - If WAL truncation fails

IllegalStateException - If log is closed or no durable snapshot covers the compaction range

compact

public void compact(long throughIndex)
             throws IOException

Compact the WAL directly by truncating entries up to and including throughIndex.

This method truncates the WAL file to prevent unbounded disk growth, independent of state machine snapshots. It is useful for bounding disk usage when snapshotting is not available or for aggressive cleanup.

Performs atomic WAL rotation: writes remaining entries to a new file and atomically renames it to replace the old file.

Thread-safe via ReentrantLock.

Parameters:

throughIndex - The index up to and including which entries should be removed (1-based)

Throws:

IOException - If WAL truncation fails

IllegalStateException - If log is closed

size

public int size()

Get the log size (number of in-memory entries).

Returns:

The number of log entries currently in memory (includes sentinel)

getSnapshotIndex

public long getSnapshotIndex()

Get the index of the last snapshot.

Returns:

The last snapshot index (1-based), or 0 if no snapshot exists

getSnapshotTerm

public long getSnapshotTerm()

Get the term of the last snapshot.

Returns:

The last snapshot term, or 0 if no snapshot exists

getLastDurableSnapshotIndex

public long getLastDurableSnapshotIndex()

Get the index of the last durably persisted snapshot.

This tracks the highest snapshot index that has been successfully written to disk via saveSnapshot(long, long, byte[]). Used as a safety precondition for log compaction.

Returns:

The last durable snapshot index, or 0 if no snapshot has been saved

saveSnapshot

public void saveSnapshot(long index,
 long term,
 byte[] data)
                  throws IOException

Save a state machine snapshot to persistent storage.

The snapshot captures the complete state machine state at a given point. It can be used for fast recovery and log compaction.

Thread-safe via SnapshotStore's internal locking.

Parameters:

index - The snapshot index — the last entry included in the snapshot (1-based)

term - The Raft term of the entry at snapshotIndex

data - The serialized snapshot data (state machine state)

Throws:

IOException - If write to disk fails

close

public void close()
           throws IOException

Close and release resources held by the persistent log.

Flushes pending WAL writes and closes the file channel. The log becomes unavailable after closing and cannot be reused.

Thread-safe via ReentrantLock.

Specified by:

close in interface AutoCloseable

Throws:

IOException - If close or flush fails

getRaftLog

protected RaftLog getRaftLog()

Get the underlying RaftLog (for testing/debugging).

Returns:

The RaftLog instance

getWalWriter

protected WalWriter getWalWriter()

Get the WAL writer (for testing).

Returns:

The WalWriter instance

getSnapshotStore

protected SnapshotStore getSnapshotStore()

Get the snapshot store (for testing).

Returns:

The SnapshotStore instance