Clients
When a client connects to the cluster, it connects to the leader Consensus Module. The Clustered Service takes all its input from the Log, so the Consensus Module tells all the Clustered Services about the new connection by publishing a SessionOpenEvent on the Log.
SessionOpenEvent¶
The ClusteredServiceAgent processes the SessionOpenEvent in onSessionOpen()
. It creates a new
ContainerClientSession, which implements the ClientSession interface that the ClusteredService has access
to.
If the ClusteredServiceAgent is on the leader, it asks the ClientSession to connect to the client's Egress channel. This creates a ConcurrentPublication to the client's Egress. The Consensus Module already has its own ConcurrentPublication to the client's Egress, which it created before sending the SessionOpenEvent, and both Publications end up sharing the same underlying Publication log buffer.
Then, whether on the leader or a follower, the ClientSession is added to the list of sessions
in the
ClusteredServiceAgent. Whenever the Clustered Service polls the Log and finds a new application message, it looks up
the ClientSession and passes it to the ClusteredService, along with the message.
void onSessionMessage(
ClientSession session,
long timestamp,
DirectBuffer buffer,
int offset,
int length,
Header header);
Finally, the ClusteredServiceAgent calls onSessionOpen()
on the ClusteredService, to tell it about the new
ClientSession.
void onSessionOpen(ClientSession session, long timestamp);
ClientSession¶
ClientSession is the interface that represents client connections. It has methods for publishing Egress messages:
long offer(DirectBuffer buffer, int offset, int length);
long offer(DirectBufferVector[] vectors);
long tryClaim(int length, BufferClaim bufferClaim);
The Clustered Service sends Egress messages to a client via its ClientSession, which calls into the ClusteredServiceAgent. The ClusteredServiceAgent checks its cluster role and if it's not the leader, it returns ClientSession.MOCKED_OFFER, which is treated as an indication of a successful send. Followers don't connect to the client's Egress, so message sending is faked. If it is the leader, it wraps the application message with a SessionMessageHeader and publishes it using the client's Egress Publication.
Egress Messages¶
Both the Consensus Module and Clustered Service can send Egress messages to the client, but the Consensus Module is only concerned with session handling. At this point, when a session has been established and the ClusteredService notified of it, the Consensus Module would only send a NewLeaderEvent if an Election was held, to tell it about the new leader, or a SessionEvent when it closes the connection. The Clustered Service sends all the application-level Egress messages to the client.
Cluster interface¶
When the Clustered Service starts, the ClusteredServiceAgent calls onStart()
on the ClusteredService, passing
it an instance of the Cluster interface.
void onStart(Cluster cluster, Image snapshotImage);
Some of the operations on the Cluster interface are:
- find the cluster role of the member
- look up a ClientSession
- close a ClientSession
- schedule or cancel a timer
- publish a message onto the Log
The Cluster interface is implemented by the ClusteredServiceAgent. The first two operations above are just look-ups. The other operations require updating something in the Cluster. They work by the ClusteredServiceAgent publishing a message to the Consensus Module on the ConsensusModule channel. This happens on both leader and follower Clustered Services.
For example, if a ClusteredService wanted to close a ClientSession, the ClusteredServiceAgent would send a
CloseSession message to the Consensus Module. This is arrives in the ConsensusModuleAgent in
onServiceCloseSession()
, which ignores it if it's a follower. The leader marks the session as closing,
publishes a SessionCloseEvent onto the Log, and publishes a SessionEvent on the Egress with EventCode
CLOSED, to tell the Client. All the Clustered Services eventually receive the Log message, which is handled in the
ClusteredServiceAgent in onSessionClose()
. This removes the ClientSession from the sessions
list, closes
the Egress Publication, then tells the ClusteredService by calling onSessionClose()
.
void onSessionClose(ClientSession session, long timestamp, CloseReason closeReason);
Snapshots¶
The onStart()
method, mentioned above, is where the ClusteredService loads its snapshot. If there is no snapshot,
the snapshotImage
is null.
When requested to take a snapshot, the ClusteredServiceAgent calls onTakeSnapshot()
on the ClusteredService.
void onTakeSnapshot(ExclusivePublication snapshotPublication);
Snapshots are covered in more detail later, in State and Snapshots.