Upcoming ArangoDB 3.7 and Storage Engines

Estimated reading time: 4 minutes

TL;DR

ArangoDB has supported two storage engines for a while: RocksDB and MMFiles. While ArangoDB started out with just the MMFiles storage engine in its early days, RocksDB became the default storage engine in the 3.4 release. Due to its drawbacks ArangoDB 3.6 deprecated the old MMFiles storage engine and with the upcoming 3.7 release we plan to fully remove support. This blog post will provide the background of why storage engines matter, why we chose to deprecate the MMFiles storage engine, and what you should be aware of when migrating from MMFiles to the RocksDB storage engine.

Storage Engines

What role does a storage engine play in databases like ArangoDB? A storage engine, such as RocksDB, provides low level storage primitives focused on storing and retrieving basic key/value pairs on a single node.

ArangoDB uses this storage engine and then provides different data models, distribution across nodes, fault tolerance, transactions and all of your favorite ArangoDB features, on top.

MMFiles vs RocksDB

In its early days ArangoDB started out with a custom storage engine called MMFiles, short for Memory-Mapped Files. The MMFiles engine is optimized for use cases where the data fits into the main memory. It allows for very fast and concurrent reads. However, writes block reads and locking occurs on the collection level. Indexes are only in-memory and need to be rebuilt on startup. This gives better performance but imposes a longer startup and recovery time.

Therefore with the 3.2 release ArangoDB added support for RocksDB, an open source storage engine originally developed under the name LevelDB by Google Inc., and then improved and rebranded to its final name by Facebook, Inc.. As of today used by many other companies. Internally it is based on a Log-Structured-Merge-Database (LSM) and can scale for data sets that are considerably larger than main memory. The engine will keep a hot set of the data in main memory, but will happily load additional data from the disk as needed. The engine will write all index values to disk too, so there is no need for it to rebuild indexes when first accessing a collection after a restart. This makes the engine’s startup very fast. After startup, the engine will gradually fill its caches during operations, therefore its performance may increase over time.

Given these advantages, RocksDB quickly became the default storage engine with the ArangoDB 3.4 release and newer features such as Hot Backup are only available for this storage  engine.

MMFiles Deprecation

With ArangoDB 3.6 the MMFiles storage engine was deprecated and with the upcoming 3.7 release it will be removed from ArangoDB. This implies that if you still use the MMFiles storage engine in 3.6 you cannot directly upgrade to 3.7. In this case you will have to export your data and re-import into RocksDB, see Switching the Storage Engine in the documentation.

One caveat of migrating to RocksDB (actually an advantage!),  is that in contrast to the MMFiles engine, the RocksDB engine does not require collection-level locks. Different write operations on the same collection do not block each other, as long as there are no write-write conflicts on the same documents. If your application relies on collection level locks (e.g., to enforce serial writes across an entire collection) you can use the exclusive option to achieve this effect on a per query basis with RocksDB:

FOR doc IN collection
  INSERT { myval: doc.val + 1 } INTO users 
  OPTIONS { exclusive: true }

Similarly the  --rocksdb.exclusive-writes startup option will enforce this behavior for the entire system. Please note that exclusive locks on collections will introduce a considerable throughput penalty and should only be used for migration purposes!

If you have any questions please feel free to reach out via our GoogleGroup or community slack!

Hear More from the Author

Introducing ArangoDB

Jörg Schad “ArangoDB 3.5 Feature Overview”

Continue Reading

ArangoDB Cluster Administration Course Released

Configuring ArangoDB-PHP to use active failover

Index types and how indexes are used in ArangoDB: Part II