Networking Layer

LoomCache handles inter-node communication and client-server traffic through a TCP layer that pairs blocking java.net sockets with one virtual thread per accepted connection. This page covers the binary wire protocol, thread-pool isolation, backpressure, and transport security.

The Binary Protocol

Rather than communicating over HTTP or REST JSON, LoomCache uses a custom versioned binary protocol.

Wire Protocol Pipelining

Compact binary (Kryo) serialization with batched pipelining over persistent TCP connections.

CLIENT

OP

BIN

OP

BIN

OP

BIN

OP

BIN

SERVER

1. Framing: Every packet carries a versioned binary frame that declares the payload size and format, so the receiver never scans ahead to find message boundaries.
2. Kryo 5.6.2 Serialization: Map entries and keys are serialized with Kryo’s binary format, using object pooling to minimize GC pressure.
3. Pipelining: Multiple commands can be in flight on the same connection with Pipelining.withDepth(int), which reduces round-trip latency. Multi-operation LoomBatch requests are a separate protocol feature capped by the configured batchMaxOperations limit.

Message Categories

The protocol groups messages into the following categories:

Category	Examples
Data operations	Map, queue, set, and topic commands
Raft consensus	AppendEntries, voting, pre-vote, snapshot install, and leadership-transfer messages
Cluster management	Join/leave, health probes, routing-table refresh, and controlled partition-migration commands
Consistency subsystem	Atomic long reads/CAS, lock acquire, and semaphore acquire (Java client lock/semaphore calls carry a managed CP session; sessionless production requests fail closed)
Client control	Authentication, near-cache subscription, batch execution, and SQL query requests
WAN / executor	WAN replication control/data messages and executor dispatch (production route is enabled but deny-all by default until task classes are explicitly allowlisted)
Responses	Success, typed errors, redirects, server-busy backpressure, and sequence-loss notifications

Thread-Pool Isolation

LoomCache isolates networking work in three ways:

• TCP listener — One virtual thread per connection. It blocks freely on I/O; the JVM unmounts the carrier thread automatically.
• Command execution pool — A dedicated pool for state-machine operations. It prevents slow clients from backing up consensus.
• Backpressure control — Watermark-based flow control over each peer’s outbound send queue. When a peer’s queue depth crosses the high watermark, further send-slot reservations for that peer are refused (no slot is granted) rather than messages being silently dropped.

Connection Lifecycle

Each connection is tracked with the negotiated TLS identity (derived from the certificate CN), the resolved permission set, and per-connection request counters. The TCP listener rejects connections that exceed the bounded per-client-IP limit with a too-many-connections error.

Backpressure

LoomCache implements watermark-based backpressure over each peer’s outbound send queue:

Peer queue depth	Behavior
At or below low watermark	Backpressure clears; send slots granted normally
Between watermarks	Sends continue; queue depth tracked per peer
At or above high watermark	Peer flagged inBackpressure; new send-slot reservations refused until the queue drains back to the low watermark

This prevents a slow or overwhelmed peer from monopolizing outbound capacity and starving Raft replication. Admission-time backpressure is raised separately by server-side idempotency handling and by the client request path.

Security (mTLS)

Production deployments should authenticate and encrypt internal traffic. LoomCache supports Mutual TLS (mTLS) for inter-node communication. Each node validates the identity of its peers before participating in Raft elections or accepting writes.

• TLSv1.3 and TLSv1.2 in the default protocol list; production deployments may pin TLSv1.3 only after every client and node runtime has been validated for it
• PKCS12 keystores for identity and trust
• Per-connection TLS handshake through the configured socket provider
• Client connections support optional mTLS outside production; production startup requires client mTLS (requireClientAuth=true)

For keystore setup and production TLS requirements, see Security & mTLS.