Service Messages

Service messages are sent from the Consensus Module to the Clustered Service via an IPC Publication. Within the Clustered Service, the ClusteredServiceAgent polls its ServiceAdapter to check for new Service messages. The ServiceAdapter polls the Service Subscription, decodes any messages and calls methods on the ClusteredServiceAgent. Note that these messages are directed at the ClusteredServiceAgent, not the ClusteredService - the ClusteredService only takes input from the Log.

In addition, when ClusterTool is used, it connects to the Consensus Module using the same IPC streams, so it can also receive Service messages (see ClusterTool for more details). Both the Service and the ClusterTool receive Service messages, but they each care about a different, mutually exclusive set of messages.

The Service messages are described below.

• JoinLog (to ClusteredService)
• ServiceTerminationPosition (to ClusteredService)
• RequestServiceAck (to ClusteredService)
• ClusterMembersResponse (to ClusterTool)
• ClusterMembersExtendedResponse (to ClusterTool)

JoinLog

Message sent from the Consensus Module during an Election, telling the Clustered Service to subscribe to the Log, from a particular position, to start processing it. The Consensus Module waits for the Clustered Service to acknowledge that it has done so.

JoinLog {
    logPosition     long     // start position, to subscribe from
    maxLogPosition  long     // stop position, or Long.MAX_VALUE
    memberId        int
    logSessionId    int      // sessionId for the Log Subscription
    logStreamId     int      // streamId for the Log Subscription
    isStartup       Boolean  // true if the cluster member is starting
    role            int      // 0 = FOLLOWER, 1 = CANDIDATE, 2 = LEADER
    logChannel      String   // channel for the Log Subscription
}

Sent in two scenarios:

• when replaying Log messages from the Log Recording in either LEADER_REPLAY or FOLLOWER_REPLAY, replaying up to the end of the Recording to restore cluster state (this sets the stop position to the end of the Recording)
• near the end of the Election to get ready for processing new Log messages in either LEADER_INIT, FOLLOWER_CATCHUP_AWAIT or FOLLOWER_LOG_AWAIT (stop position is Long.MAX_VALUE)

Received by the ClusteredServiceAgent when it polls the ServiceAdapter. If it finds a JoinLog message, it calls ClusteredServiceAgent.onJoinLog(), which stores the request in an activeLogEvent field. This is picked up straight after the poll, where it then calls joinActiveLog(). It subscribes to the Log and uses the ConsensusModuleProxy to send a ServiceAck in response, acknowledging that it is at the requested position.

ServiceTerminationPosition

Message sent from the Consensus Module when it is terminating, telling the Clustered Service to terminate when it reaches the specified Log position. The Consensus Module waits for the Clustered Service to acknowledge that it has done so, before continuing with its own termination.

ServiceTerminationPosition {
    logPosition long
}

Can come from a number of places:

• on the leader, when ClusterTool is used to set the Cluster control toggle Counter to SHUTDOWN (snapshot and stop) or ABORT (stop)
• on a follower, when the leader has been told to SHUTDOWN or ABORT, it sends a Consensus TerminationPosition message to get the followers to do the same, at the same position
• an abnormal termination (unexpected cluster state)

Received by the ClusteredServiceAgent when it polls the ServiceAdapter. If it finds a ServiceTerminationPosition message, it calls ClusteredServiceAgent.onServiceTerminationPosition(), which stores the position in a terminationPosition field. After each poll, it checks to see if it has reached the terminationPosition. When it has, it notifies the ClusteredService using ClusteredService.onTerminate(), then uses the ConsensusModuleProxy to send a ServiceAck in response, acknowledging that it is at the specified position. The ClusteredServiceAgent then throws a ClusterTerminationException, to terminate its own thread, stopping the Clustered Service.

RequestServiceAck

Message sent from the Consensus Module when it is starting from bootstrap state (which is for internal use only), telling the Clustered Service to acknowledge when it has reached the specified Log position.

RequestServiceAck {
    logPosition long
}

Originates from ConsensusModuleAgent.recoverFromBootstrapState().

Received by the ClusteredServiceAgent when it polls the ServiceAdapter. If it finds a RequestServiceAck message, it calls ClusteredServiceAgent.onRequestServiceAck(), which stores the position in a requestedAckPosition field. After each poll, it checks to see if it has reached the requestedAckPosition. When it has, it uses the ConsensusModuleProxy to send a ServiceAck in response, acknowledging that it is at the specified position.

ClusterMembersResponse

Message sent from the Consensus Module in response to a ClusterMembersQuery, which would have been sent by ClusterTool. For this message, ClusterTool is the recipient, and it is ignored by the Clustered Service. ClusterMembersQuery is used by a few ClusterTool commands, such as snapshot and list-members.

ClusterMembersResponse {
    correlationId     long
    leaderMemberId    int
    activeMembers     String
    passiveFollowers  String
}

Sent ConsensusModuleAgent.onClusterMembersQuery() when handling a ClusterMembersQuery.

Received by ClusterControlAdapter, which is part of ClusterTool.

ClusterMembersExtendedResponse

This is the same as ClusterMembersResponse above, but with a larger response payload. This is returned by the Consensus Module if the ClusterMembersQuery has the extended field set to true.

ClusterMembersExtendedResponse {
    correlationId            long
    currentTimeNs            long (epoch time in cluster time units)
    leaderMemberId           int
    memberId                 int
    activeMembers [          // Members of the cluster which have voting rights
        leadershipTermId     long
        logPosition          long
        timeOfLastAppendNs   long (epoch time in cluster time units)
        memberId             int
        ingressEndpoint      String
        consensusEndpoint    String
        logEndpoint          String
        catchupEndpoint      String
        archiveEndpoint      String
    ]  
    passiveMembers [         // Members of the cluster which do not have voting
                             // rights but could become active members
        leadershipTermId     long
        logPosition          long
        timeOfLastAppendNs   long (epoch time in cluster time units)
        memberId             int
        ingressEndpoint      String
        consensusEndpoint    String
        logEndpoint          String
        catchupEndpoint      String
        archiveEndpoint      String
    ]
}

Sent ConsensusModuleAgent.onClusterMembersQuery() when handling a ClusterMembersQuery.

Received by ClusterControlAdapter, which is part of ClusterTool.