Kaveh, Master in theoretical physics in Cologne, Zuerich and Bonn. Ph.D. in physics. Joined ArangoDB after medical imaging research and lecturing at Juelich Research Center, RWTH Aachen University, Maastricht University and New York University. He joined ArangoDB in 2016 as a core software engineer. He has been responsible for the development of core facilities in the ArangoDB cluster; among them the consensus infrastructure.
A RAFT based application store on ArangoDB's Agency
The need for replicated logs and state machines, resilient to local failures and network partitions, has led to the development of the Paxos and more recently of the RAFT consensus protocols. While many very good implementations of RAFT have been developed by the community over the last years, we demonstrate that significant added value can be derived from immersing an application framework into the consensus realm.
The real world implementation of such a consensus appliance, regardless of the seeming simplicity of RAFT, proves to be very tricky in detail. One should preferably rely on tested and battle-hardened implementations. At ArangoDB however, we realised that there is more to be gained from RAFT, if we allowed for the consensus cluster to be extended in functionality through custom user apps.
Arango speak for consensus is “Agency”. The agency is formed through a limited number of specialised database nodes called agents. In addition to serving the replicated log and state machine, the agency performs the supervision and failover of servers, shards and of the foxx application framework.