ArangoDB Spartan Mode: Optimize Performance and Resource Usage

Most of us saw the fantastic movie 300 (I did it last night…again) or at least read the comics. 300 spartans barely wearing anything but achieving a lot. This little how-to will show you how to put ArangoDB into Spartan-Mode and thereby reduce memory-footprint and CPU usage.

Big thanks to Conrad from L.A. for his time and for giving us the impulse for this little how-to!

Background

Recently, we had a lot of cool talks with users who are new to ArangoDB or doing their PoC at the moment. Two topics came up several times. First ArangoDB’s memory footprint which is higher compared to some other DBs. We stated this already in our latest performance benchmark. The second issue is the CPU usage in standby mode. Users claimed 4-6% CPU usage when ArangoDB isn’t really doing anything.

First of all we use Google’s V8 engine for our JS framework Foxx. At first glance it might seem that V8 is memory thirsty but most of the memory is virtual memory and therefore does not actually fill up your physical RAM at all. If you don’t need V8 (not using Foxx) then you can run ArangoDB with less V8 contexts which reduces the memory usage significantly.

Let’s start with reducing the memory footprint

On our little test system (Laptop, with x86_64 Ubuntu Linux) we installed a new ArangoDB instance. On this system we measured a memory-footprint of ~300 MB and 5.6 GB of reserved disk space (all the numbers might vary a bit on other systems) for an empty database. With ArangoDB in Spartan-Mode we reduced memory-footprint to ~110 MB and reserved disk space to under 1GB … and here is how:

We need to change the configuration of ArangoDB, which you can either do by editing the configuration file or by using command line arguments when you start ArangoDB from the console. The location of the configuration file on Unix (Linux, BSD) with the normal binary packages is /etc/arangodb/arangod.conf, on other systems or installation options the place varies.

You can reduce the number of V8 contexts by changing the configuration option javascript.v8-contexts to 1. For this, you would add the command line option

--javascript.v8-contexts 1

to the command with which you start ArangoDB on the console. In the configuration file, you would add

[javascript] v8-contexts=1

IMPORTANT NOTE: You can NOT set the number of contexts to 0 because ArangoDB uses them for large parts of the API as well as for example for the emergency shell and user generated functions.

The next step to Spartan-Mode is to reduce the memory needed for the Write Ahead Logs (WAL). This can be done using three different WAL options:

Reduce the number of historic WAL files which will reduce the memory usage when ArangoDB is in use. You can do this as follows (on the command line)

--wal.historic-logfiles 1

This is suitable if you don’t plan to use asynchronous replication.

Reduce the prepared WAL log files which are kept ready for future write operations

--wal.reserve-logfiles 1

This is suitable if you can live with a small delay when the write load suddenly increases.

In addition you can reduce the size of all WAL files to e.g. 8 MB by setting

--wal.logfile-size 8388608

With all these changes you should see a real decrease in memory used by ArangoDB.

So now that we reduced the memory footprint…

Let’s reduce CPU usage

With ArangoDB you’re able to reduce the CPU usage baseline as well. In our test we could put it to nearly 0 when no requests hit ArangoDB.

You can reduce the CPU usage in two steps.

You can get rid of the Foxx queues which allow Foxx apps to run regular tasks or background jobs. You can add the following command line option and you are done with step 1:

--server.foxx-queues false

NOTE: If you use this configuration option your Foxx apps cannot run background jobs and regularly repeating tasks.

In some cases you might want to turn off all statistics of ArangoDB (request statistics, resource usage, etc). If you turn them off you’ll further reduce CPU usage.

--server.disable-statistics true

By these small configurations you can reduce the resources ArangoDB needs and put ArangoDB into full Spartan-Mode.

Note: Putting ArangoDB into Spartan-Mode might be useful for testing, little projects with low data volume or if you run ArangoDB on small machines. We DON’T recommend Spartan-Mode for high performance needs!

If you have further questions on this topic just leave a comment or ask on Stack Overflow

Your Arangos