ArangoDB Starter Architecture
What does the Starter do
The ArangoDB Starter is a program used to create ArangoDB database deployments on bare-metal (or virtual machines) with ease. It enables you to create everything from a simple Single server instance to a full blown Cluster with datacenter to datacenter replication in under 5 minutes.
The Starter is intended to be used in environments where there is no higher level orchestration system (e.g. Kubernetes) available.
The Starter is a separate process in a binary called
arangodb.exe on Windows).
This binary has its own version number that is independent of a ArangoDB (database)
This means that Starter version
a.b.c can be used to run deployments
of ArangoDB databases with different version.
For example, the Starter with version
0.11.2 can be used to create
ArangoDB deployments with ArangoDB version
3.2.<something> as well
as deployments with ArangoDB version
It also means that you can update the Starter independently from the ArangoDB database.
Note that the Starter is also included in all binary ArangoDB packages.
To find the versions of you Starters & ArangoDB database, run the following commands:
# To get the Starter version arangodb --version # To get the ArangoDB database version arangod --version
Starter deployment modes
The Starter supports 3 different modes of ArangoDB deployments:
- Single server
- Active failover
Note: Datacenter replication is an option for the
cluster deployment mode.
You select one of these modes using the
--starter.mode command line option.
Depending on the mode you’ve selected, the Starter launches one or more
arangosync) server processes.
No matter which mode you select, the Starter always provides you a common directory structure for storing the servers data, configuration & log files.
Starter operating modes
The Starter can run as normal processes directly on the host operating system, or as containers in a docker runtime.
When running as normal process directly on the host operating system, the Starter launches the servers as child processes and monitors those. If one of the server processes terminates, a new one is started automatically.
When running in a docker container, the Starter launches the servers as separate docker containers, that share the volume namespace with the container that runs the Starter. It monitors those containers and if one terminates, a new container is launched automatically.
The Starter uses a single directory with a well known structure to store all data for its own configuration & logs, as well as the configuration, data & logs of all servers it starts.
This data directory is set using the
--starter.data-dir command line option.
It contains the following files & sub-directories.
setup.jsonThe configuration of the “cluster of Starters”. For details see below. DO NOT edit this file.
arangodb.logThe log file of the Starter
dbserver<port>: directories for launched servers. These directories contain among others the following files:
apps: A directory with Foxx applications
data: A directory with database data
arangod.conf: The configuration file for the server. Editing this file is possible, but not recommended.
arangod.log: The log file of the server
arangod_command.txt: File containing the exact command line of the started server (for debugging purposes only)
Running on multiple machines
cluster mode, it is required to run multiple
Starters, as every Starter will only launch a subset of all servers needed
to form the entire deployment.
For example in
cluster mode, a Starter will launch a single agent, a single dbserver
and a single coordinator.
It is the responsibility of the user to run the Starter on multiple machines such that enough servers are started to form the entire deployment. The minimum number of Starters needed is 3.
The Starters running on those machines need to know about each other’s existence.
In order to do so, the Starters form a “cluster” of their own (not to be confused
with the ArangoDB database cluster).
This cluster of Starters is formed from the values given to the
command line option. You should pass the addresses (
<host>:<port>) of all Starters.
For example a typical commandline for a cluster deployment looks like this:
arangodb --starter.mode=cluster --starter.join=hostA:8528,hostB:8528,hostC:8528 # this command is run on hostA, hostB and hostC.
The state of the cluster (of Starters) is stored in a configuration file called
setup.json in the data directory of every Starter and the ArangoDB
agency is used to elect a master among all Starters.
The master Starter is responsible for maintaining the list of all Starters
involved in the cluster and their addresses. The slave Starters (all Starters
except the elected master) fetch this list from the master Starter on regular
basis and store it to its own
setup.json config file.
setup.json config file MUST NOT be edited manually.
Running on multiple machines (under the hood)
As mentioned above, when the Starter is used to create an
cluster deployment, it first creates a “cluster” of Starters.
These are the steps taken by the Starters to bootstrap such a deployment from scratch.
- All Starters are started (either manually or by some supervisor)
- All Starters try to read their config from
setup.json. If that file exists and is valid, this bootstrap-from-scratch process is aborted and all Starters go directly to the
runningphase described below.
- All Starters create a unique ID
- The list of
--starter.joinarguments is sorted
- All Starters request the unique ID from the first server in the sorted
--starter.joinlist, and compares the result with its own unique ID.
- The Starter that finds its own unique ID, is continuing as
bootstrap masterthe other Starters are continuing as
bootstrap masterwaits for at least 2
bootstrap slavesto join it.
bootstrap slavescontact the
bootstrap masterto join its cluster of Starters.
- Once the
bootstrap masterhas received enough (at least 2) requests to join its cluster of Starters, it continues with the
bootstrap slaveskeep asking the
bootstrap masterabout its state. As soon as they receive confirmation to do so, they also continue with the
running phase all Starters launch the desired servers and keeps monitoring those
servers. Once a functional agency is detected, all Starters will try to be
running master by trying to write their ID in a well known location in the agency.
The first Starter to succeed in doing so wins this master election.
running master will keep writing its ID in the agency in order to remaining
running master. Since this ID is written with a short time-to-live,
other Starters are able to detect when the current
running master has been stopped
or is no longer responsible. In that case the remaining Starters will perform
another master election to decide who will be the next
API requests that involve the state of the cluster of Starters are always answered
by the current
running master. All other Starters will refer the request to