Skip to content

Egress Messages

Clustered Service Consensus Module Archive log Clustered Service Consensus Module Archive log Client B Client A Network cnc.dat cnc.dat cnc.dat cnc.dat cons-module service control-response control-request control-response control-request cons-module service consensus0-1 consensus2-1 consensus1-0 consensus1-2 consensus1-0 consensus2-0 consensus0-1 consensus0-2 log log egressB ingressB egressA ingressA ingressB ingressA egressA egressB Conductor Receiver Sender Conductor Receiver Sender Receiver Conductor Sender Receiver Conductor Sender StreamId: 102 StreamId: 104 StreamId: 105 StreamId: 100 StreamId: 108 StreamId: 101 LEADER FOLLOWER

Egress messages are response messages from the cluster back to a client. When a client connects to the cluster, it specifies an Egress endpoint so the cluster can connect back to it. Egress messages will always come from the leader, and can come from its Consensus Module or Clustered Service. Application level responses will be from the Clustered Service. The Consensus Module can send other messages when the connection is being established, after an Election, or if the connection is being closed.

The Egress messages are described below.

SessionMessageHeader

This is the same SessionMessageHeader as on the Ingress, but this is being used in the opposite direction, for an application level response from the leader's Clustered Service. It is an envelope for the application message. When a follower Clustered Service attempts to publish the same Egress message, the publishing is faked, so the Clustered Service thinks it succeeded, and the client doesn't receive duplicate responses.

Sent when the leader's Clustered Service calls one of the offer() or tryClaim() methods on the ClientSession interface. The call is routed through the ClusteredServiceAgent, which checks whether it is the leader, and fakes the sending otherwise.

The Clustered Service should only publish messages when it is responding to a Log message. Log messages originate from the Ingress, timers that fire, and messages that the Clustered Service publishes back into the cluster. In all cases, the Log message will contain a timestamp, which is the time it was appended to the Log. When the Clustered Service sends SessionMessageHeader messages on the Egress, the ClusteredServiceAgent will automatically set the timestamp to this same timestamp. This means for a given Log message, any outbound SessionMessageHeader messages will contain the same timestamp, as though time stood still while it was being processed.

SessionMessageHeader {
    leadershipTermId  long  // current leadership term
    clusterSessionId  long  // identifies the session
    timestamp         long  // timestamp of the current Log message (epoch time in cluster time units)
}

It is received in the AeronCluster client when the client application calls pollEgress(), and it gives the message to the EgressListener (the interface that the client implements to receive Egress messages).

SessionEvent

Sent by the leader Consensus Module to the client, related to the session itself. SessionEvent contains an EventCode, which is one of OK, ERROR, REDIRECT, AUTHENTICATION_REJECTED or CLOSED. They are all used while a connection is being established, apart from CLOSED, which can occur later. The session can be closed on request by the client, by the Clustered Service, or after a timeout by the Consensus Module.

Sent from EgressPublisher.sendEvent(), which is called from several places in ConsensusModuleAgent.

EventCodes are:

  • OK - successful response to SessionConnectRequest
  • ERROR - failure to connect, such as if the version in the SessionConnectRequest is not compatible with the cluster's version, or there are too many clients connected (the default is 10)
  • REDIRECT - if the SessionConnectRequest was sent to a follower, which is how clients discover which member is the leader
  • AUTHENTICATION_REJECTED - if authentication is enabled and the client failed authentication
  • CLOSED - on client request, from the Clustered Service, or from the Consensus Module on timeout / shutdown
SessionEvent {
    clusterSessionId  long       // created by the Consensus Module for this session
    correlationId     long       // from the client's last Ingress message
    leadershipTermId  long       // current leadership term
    leaderMemberId    int        // current leader
    code              EventCode  // enum, described above
    version           int        // protocol version
    detail            String     // further detail, e.g. error message or list of cluster ingress endpoints
}

It is received in the AeronCluster client when the client application calls pollEgress(), and it gives the message to the EgressListener (the interface that the client implements to receive Egress messages).

ClusterBackupAgent is another cluster client that can receive Egress messages. It doesn't listen to SessionMessageHeader, but does listen to SessionEvents.

Challenge

An authentication challenge, which can be sent from the leader's Consensus Module to a client, when it tries to connect. The Authenticator can challenge the client, which needs to return a ChallengeResponse.

Sent by EgressPublisher.sendChallenge(), which can be used by the Authenticator via ClusterSessionProxy.challenge(). 

Challenge {
    correlationId     long    // from the client's last Ingress message
    clusterSessionId  long    // created by the Consensus Module for this session
    encodedChallenge  byte[]  // the challenge response
}

It is received in the AsyncConnect object (used to build the AeronCluster client object) when it polls the Egress while connecting.

ClusterBackupAgent is another cluster client that can handle receiving a Challenge message when it connects to the cluster.

NewLeaderEvent

Sent by the leader Consensus Module at the end of an Election, to tell the client that there is a new leader. The client uses this to switch its Ingress connection.

Each Consensus Module has a list of ClusterSessions, even though the followers don't connect to their Egress. At the end of an Election, the new leader marks each ClusterSession, saying that it has a NewLeaderEvent pending. Later, any marked sessions go into sendNewLeaderEvent(), which sends the event.

NewLeaderEvent {
    leadershipTermId  long    // from the Election
    clusterSessionId  long    // the same id from the last leader connection
    leaderMemberId    int     // the new leader id
    ingressEndpoints  String  // as before - this will not have changed
}

It is received in the AeronCluster client when the client application calls pollEgress(), and it eventually gets into onNewLeader(), which connects to the new leader's Ingress and notifies the ClusteredService via the EgressListener interface.

AdminResponse

Sent by the leader Consensus Module in response to an AdminRequest.

Sent from ConsensusModuleAgent.onAdminRequest().

AdminResponse {
    clusterSessionId  long               // session id of the cluster client that made the request
    correlationId     long               // from the AdminRequest
    requestType       AdminRequestType   // enum: SNAPSHOT is the only type
    responseCode      AdminResponseCode  // enum: OK, ERROR, UNAUTHORISED_ACCESS
    message           String             // e.g. an error message
    payload           byte[]             // optional response payload, empty for SNAPSHOT
}

AeronCluster doesn't do anything with this - it's just passed to EgressListener.