Index types and how indexes are used in ArangoDB: Part II

00GeneralTags: , ,

In the first part of this article we dived deep into what indexes are currently available in ArangoDB (3.2 and 3.3), also briefly looking at what improvements are coming with ArangoDB 3.4. Read Part I here.

In this Part II, we are going to focus on how to actually add indexes to a data model and speed up specific queries.

Adding indexes to the data model

The goal of adding an extra index to the data model is to speed up a certain query or even multiple queries.

One of the first things that should be done during development of AQL queries should be to review the output of the explain command. A query can be explained using ArangoDB’s WEB UI or from the ArangoShell. In the ArangoShell it is as simple as db._explain(query), where query is the AQL query string. To explain a query which also has bind parameters, they need to be passed separately into the command, e.g. db._explain(query, bindParameters).
Read more

How We Wronged Neo4j & PostgreSQL: Update of ArangoDB Benchmark 2018

00GeneralTags: , ,

Recently, we published the latest findings of our Performance Benchmark 2018 including Neo4j, PostgGreSQL, MongoDB, OrientDB and, of course, ArangoDB. We tested bread & butter tasks in a client/server setup for all databases like single read/write and aggregation, but also things like shortest path queries which are a speciality for graph databases. Our goal was and is to demonstrate that a native multi-model database like ArangoDB can at least compete with the leading single model databases on their home turf.

Traditionally, we are transparent with our benchmarks, learned plenty from community feedback and want to keep it that way. Unfortunately, we did something wrong in our latest version and this update will explain what happened and how we fixed it. Read more

Index types and how indexes are used in ArangoDB: Part I

00GeneralTags: , ,

As in other database systems, indexes can be used in ArangoDB to speed up data retrieval queries, sometimes by many orders of magnitude. Getting the indexes set up the right way is essential for good query performance, so this is an important topic that affects most ArangoDB installations.

This is Part I of how indexes are used by ArangoDB where we discuss what types of indexes are available in the database. In Part II, we will dig deeper into how to actually add indexes to a data model and speed up specific queries. Read Part II here. Read more

NoSQL Performance Benchmark 2018 – MongoDB, PostgreSQL, OrientDB, Neo4j and ArangoDB

06GeneralTags: , ,

This article is part of ArangoDB’s open-source performance benchmark series. Since the previous post, there are new versions of competing software on which to benchmark. Plus, there are some major changes to ArangoDB software.

For instance, in latest versions of ArangoDB, an additional storage engine based on Facebook’s RocksDB has been included. So we waited until its integration was finished before conducting a new benchmark test. Besides all of these factors, machines are now faster, so a new benchmark made sense.

Before I get into the benchmark specifics and results, I want to send a special thanks to Hans-Peter Grahsl for his fantastic help with MongoDB queries. Wrapping my head around the JSON notation is for sure not impossible but boy can querying data be complicated. Thanks Hans-Peter for your help! Big thanks as well to Max De Marzi and “JakeWins” both team Neo4j for their contributions and improvements to the 2018 Edition of our benchmark. Also big thanks to Spain and ToroDB CEO/Founder Alvaro Hernandez for contributing your knowledge for PostgreSQL. Deep thanks to my teammates Mark, Michael and Jan for their excellent and tireless work on this benchmark. Great teamwork, crew! Read more

Performance Impact of Meltdown and Spectre V1 Patches on ArangoDB

01GeneralTags: , ,

To investigate the impact of the Meltdown and Spectre patches on the performance of ArangoDB, we ran benchmark tests with the two storage engines available in ArangoDB (MMFiles & RocksDB). We used the arangobench benchmark and test tool for these tests.

The tests include 10 different test cases with changing test parameters like concurrency, batch requests and asynchronous execution. Read more

RocksDB smoothing for ArangoDB customers

01GeneralTags:

I have varying levels of familiarity with Google’s original leveldb and three of its derivatives. RocksDB is one of the three. In each of the four leveldb offerings, the code is optimized for a given environment. Google’s leveldb is optimized for a cell phone, which has much more limited resources than a server. RocksDB is optimized for flash arrays on a large servers (per various Rocksdb wiki pages). Note that a flash array is a device of much higher throughput than a SATA or SSD drive or array. It is a device that sits on the processor’s bus. RocksDB’s performance benchmark page details a server with 24 logical CPU cores, 144GB ram, and two FusionIO flash PCI devices. Each FusionIO device cost about $10,000 at the time of the post. So RocksDB is naturally tuned for extremely fast and expensive systems. Here is an example Arangodb import on a machine similar to the RocksDB performance tester: Read more

ArangoDB 3.2 GA
RocksDB, Pregel, Fault-Tolerant Foxx & Satellite Collections

00General, ReleasesTags: , , , ,

We are pleased to announce the release of ArangoDB 3.2. Get it here. After an unusually long hackathon, we eliminated two large roadblocks, added a long overdue feature and integrated an interesting new one into this release. Furthermore, we’re proud to report that we increased performance of ArangoDB on average by 35%, while at the same time reduced the memory footprint compared to version 3.1. In combination with a greatly improved cluster management, we think ArangoDB 3.2 is by far our best work. (see release notes for more details)

One key goal of ArangoDB has always been to provide a rock solid platform for building ideas. Our users should always feel safe to try new things with minimal effort by relying on ArangoDB. Todays 3.2 release is an important milestone towards this goal. We’re excited to release such an outstanding product today. Read more

ArangoDB 3.2 beta release:
Pluggable Storage Engine with RocksDB, Distributed Graph Processing and a ClusterFoxx

01General, ReleasesTags: , , , , , ,

We’re excited to release today the beta of ArangoDB 3.2. It’s feature rich, well tested and hopefully plenty of fun for all of you. Keen to take it for a spin? Get ArangoDB 3.2 beta here.

With ArangoDB 3.2, we’re introducing the long-awaited pluggable storage engine and its first new citizen, RocksDB from Facebook

  • RocksDB: You can now use as much data in ArangoDB as you can fit on your disk. Plus, you can enjoy performance boosts on writes by having only document-level locks (more info below).
  • Pregel: Furthermore, we implemented distributed graph processing with Pregel for discovering hidden patterns, identify communities and perform in-depth analytics of large graph data sets.
  • ClusterFoxx: Another important upgrade is what we internally and playfully call the ClusterFoxx. The Foxx management internals have been rewritten from the ground up to make sure multi-coordinator cluster setups always keep their services in sync and new coordinators are fully initialised even when all existing coordinators are unavailable.
  • Enterprise: Working with some of our largest customers, we’ve added further security and scalability features to ArangoDB Enterprise like LDAP integration, Encryption at Rest, and the brand new Satellite Collections.

The goal of the whole ArangoDB 3 release cycle has been to scale the multi-model idea to new heights. Getting ‘ready’ for large scale applications is not done overnight and it’s definitely not possible without the help of a strong community. We’d like to invite all of you to lend us a helping hand to make ArangoDB 3.2 the best release ever. Please push this beta to its limits: test it for your use cases and compare the performance of the new features like RocksDB. Let us know on Github any bug that you find. Don’t worry about hurting our feelings: we want to fix any problems.

Join the Beta Bug Hunt Challenge and win a $200 Amazon Gift Card as first prize. You can find more details about this reward program at the end of this post. Read more

Read the latest NoSQL Performance Benchmark 2018: MongoDB, PostgreSQL, OrientDB, Neo4j and ArangoDB