Overview¶
Aeron has three related components that build on each other.
- Aeron Transport (originally known as just Aeron) is a high performance message transport
- Aeron Archive adds the ability to record and replay Aeron Transport messages
- Aeron Cluster is a high performance clustering framework built on Aeron Transport and Aeron Archive
Aeron Transport¶
Aeron Transport is a messaging system, used for sending messages between services. What sets it apart from other messaging systems is its primary goal:
A design goal for Aeron is to be the highest throughput with the lowest and most predictable latency of any messaging system.
It has a set of Design Principles that guided its development. These include:
- being garbage free in steady state running - this avoids latency jitter from garbage collection
- using Smart Batching to handle bursts of traffic
- using lock-free algorithms
- using non-blocking IO
- avoiding unnecessary data copies
These allow it to have a lot of mechanical sympathy for the underlying hardware, which provides the best latency and throughput for a given system. Its design also allows it to provide high throughput and low latency at the same time, rather than having to choose between them, as with some other messaging systems.
Aeron Transport provides a Media Driver application to run on each machine. Services on a machine talk to the Media Driver through shared memory, both to give it instructions, and to send messages to other services. The Media Driver deals with sending messages reliably over the network to other Media Drivers.
Aeron Transport was created first, and was originally known simply as Aeron.
Aeron Archive¶
Aeron Transport does not store messages on disk and requires the recipient of a message to be present when the sender sends. If an application needs messages to be stored, it can use Aeron Archive.
Aeron Archive is another application that runs alongside the Media Driver. It provides a control interface, through which applications can request it to record or replay streams of messages, or even replicate a recording from one Archive to another.
Aeron Archive acts like a network tap, listening in on messages that flow through Aeron Transport, recording them without any impact on Transport performance. Messages can be recorded at either end of a connection. When messages are replayed, they are received using the same mechanism (APIs) as receiving messages from Aeron Transport.
Aeron Cluster¶
Aeron Cluster is a framework for writing high performance, fault-tolerant services. Multiple instances of a service (typically 3 or 5) run together as a cluster, which provides a level of fault-tolerance. It uses an implementation of the Raft Consensus Algorithm to keep the instances in sync, so they behave as a single entity (they all process the same inputs and are not shards).
They elect a leader and the followers stay in sync with the leader. Clients only talk to the leader. If the leader dies, one of the followers will be elected as the new leader, which provides automatic failover that can be completed in seconds (great at 3AM!).
Aeron Cluster is a framework for high-performance in-memory fault-tolerant services.
"In-memory" refers to the fact that the state held in memory is considered the source of truth. The Consensus Algorithm provides enough safety that the state does not need storing in a database. Instead, Aeron Cluster streams all the input messages to disk (which is a lot more efficient) and relies on each service processing the inputs in a deterministic way, to create the same state.
Not having to talk to a database yields a massive performance improvement. In addition, the state can be held in memory in efficient data structures that are suited to how it is processed, rather than being influenced by database structures. When run with Aeron Cluster, a good, well written service can process 100,000's of messages per second with consistent, 99.999% latency measured in microseconds.
Aeron Cluster has become "the global technology standard for high-throughput, low-latency, fault-tolerant trading systems".