TL;DR

ArangoDB has supported two storage engines for a while: RocksDB and MMFiles. While ArangoDB started out with just the MMFiles storage engine in its early days, RocksDB became the default storage engine in the 3.4 release. Due to its drawbacks ArangoDB 3.6 deprecated the old MMFiles storage engine and with the upcoming 3.7 release we plan to fully remove support. This blog post will provide the background of why storage engines matter, why we chose to deprecate the MMFiles storage engine, and what you should be aware of when migrating from MMFiles to the RocksDB storage engine.

Storage Engines

What role does a storage engine play in databases like ArangoDB? A storage engine, such as RocksDB, provides low level storage primitives focused on storing and retrieving basic key/value pairs on a single node.

ArangoDB uses this storage engine and then provides different data models, distribution across nodes, fault tolerance, transactions and all of your favorite ArangoDB features, on top.

MMFiles vs RocksDB

In its early days ArangoDB started out with a custom storage engine called MMFiles, short for Memory-Mapped Files. The MMFiles engine is optimized for use cases where the data fits into the main memory. It allows for very fast and concurrent reads. However, writes block reads and locking occurs on the collection level. Indexes are only in-memory and need to be rebuilt on startup. This gives better performance but imposes a longer startup and recovery time.

Therefore with the 3.2 release ArangoDB added support for RocksDB, an open source storage engine originally developed under the name LevelDB by Google Inc., and then improved and rebranded to its final name by Facebook, Inc.. As of today used by many other companies. Internally it is based on a Log-Structured-Merge-Database (LSM) and can scale for data sets that are considerably larger than main memory. The engine will keep a hot set of the data in main memory, but will happily load additional data from the disk as needed. The engine will write all index values to disk too, so there is no need for it to rebuild indexes when first accessing a collection after a restart. This makes the engine’s startup very fast. After startup, the engine will gradually fill its caches during operations, therefore its performance may increase over time.

MMFiles RocksDB
dataset needs to fit into memory work with as much data as fits on disk
indexes in memory hot set in memory, data and indexes on disk
slow restart due to index rebuilding fast startup (no rebuilding of indexes)
volatile collections (only in memory, optional) collection data always persisted
collection level locks (writes block reads and other writes) document level locks (document level locking
(reads never block, writes only block writes to the same key)

Given these advantages, RocksDB quickly became the default storage engine with the ArangoDB 3.4 release and newer features such as Hot Backup are only available for this storage  engine.

MMFiles Deprecation

With ArangoDB 3.6 the MMFiles storage engine was deprecated and with the upcoming 3.7 release it will be removed from ArangoDB. This implies that if you still use the MMFiles storage engine in 3.6 you cannot directly upgrade to 3.7. In this case you will have to export your data and re-import into RocksDB, see Switching the Storage Engine in the documentation.

One caveat of migrating to RocksDB (actually an advantage!),  is that in contrast to the MMFiles engine, the RocksDB engine does not require collection-level locks. Different write operations on the same collection do not block each other, as long as there are no write-write conflicts on the same documents. If your application relies on collection level locks (e.g., to enforce serial writes across an entire collection) you can use the exclusive option to achieve this effect on a per query basis with RocksDB:

Similarly the  --rocksdb.exclusive-writes startup option will enforce this behavior for the entire system. Please note that exclusive locks on collections will introduce a considerable throughput penalty and should only be used for migration purposes!

If you have any questions please feel free to reach out via our GoogleGroup or community slack!