Performance Impact of Meltdown and Spectre V1 Patches on ArangoDB

00GeneralTags: , ,

To investigate the impact of the Meltdown and Spectre patches on the performance of ArangoDB, we ran benchmark tests with the two storage engines available in ArangoDB (MMFiles & RocksDB). We used the arangobench benchmark and test tool for these tests.

The tests include 10 different test cases with changing test parameters like concurrency, batch requests and asynchronous execution.

Test setup

We used the arangobench tool to execute a few different performance tests. arangobench is an easy-to-use tool to measure performance, and it comes bundled with every ArangoDB release.

The test we ran measure the performance (in terms of elapsed time) for mass document insertion, because that is one of the most common use cases. For these document insertion tests, we varied some of the test invocation parameters to see if the impact varies depending on the setup.

Here are a few of the options we varied in the tests:

  • serial insertion (one client thread) vs. parallel insertion (multiple client threads)
  • using one HTTP request per insert vs. batching multiple operations in a single HTTP request (batching)
  • synchronous (blocking) requests vs. asynchronous (non-blocking) fire-and-forget requests

We combined these options sensibly to end up with 10 different test cases. The test cases were inspired/referenced from here
Read more

Performance analysis with pyArango: Part III Measuring possible capacity with usage Scenarios

00General, how to, PerformanceTags: , , , , ,

So you measured and tuned your system like described in the Part I and Part II of these blog post series. Now you want to get some figures how many end users your system will be able to serve. Therefore you define “scenarios” which will be typical for what your users do.
One such a user scenario could i.e. be:

  • log in
  • do something
  • log out

Since your users won’t nicely queue up and wait for other users to finish their business, the pace you need to test your defined system is “starting n scenarios every second”. Many scenarios simulating different users may be running in parallel. If your scenario would require 10 seconds to finish, and you’d start 1 per second, that means that your system needs to be capable to process 10 users in parallel. If it can’t handle that, you will see that more than 10 sessions are running in parallel, and the time required to handle such a scenario will lengthen. You will see the server resource usage go up and up, and finally have it burst in flames.
Read more

Read the latest NoSQL Performance Benchmark 2018: MongoDB, PostgreSQL, OrientDB, Neo4j and ArangoDB