libgcc: When exceptions collide

This is a story of an excursion to the bottom of a deep rabbit hole, where I discovered a foot gun in `gcc`’s `libgcc`. The investigation has cost me several days and I hope that by writing this up I can entertain others and save them the journey.

TL;DR

If a C++ application is compiled with GCC on Linux and statically linked against a non-GLibC C-library (like `libmusl`), then there is a danger of a data race which leads to a busy loop happening after `main()` and all static destructors have finished. The race happens, if the application does not use `pthread_cancel` explicitly and if the very first exception which is thrown in the processes’ life is thrown in two different threads at the same time.
Read more

More info...

Performance Impact of Meltdown and Spectre V1 Patches on ArangoDB

To investigate the impact of the Meltdown and Spectre patches on the performance of ArangoDB, we ran benchmark tests with the two storage engines available in ArangoDB (MMFiles & RocksDB). We used the arangobench benchmark and test tool for these tests.

The tests include 10 different test cases with changing test parameters like concurrency, batch requests and asynchronous execution. Read more

More info...

Performance analysis with pyArango: Part III Measuring possible capacity with usage Scenarios

So you measured and tuned your system like described in the Part I and Part II of these blog post series. Now you want to get some figures how many end users your system will be able to serve. Therefore you define “scenarios” which will be typical for what your users do. Read more

More info...

The great AQL shootout: ArangoDB 2.5 vs 2.6

For the ArangoDB 2.6 release from last week we’ve put some performance tests together. The tests will compare the AQL query execution times in 2.5 and 2.6.

The results look quite promising: 2.6 outperformed 2.5 for all tested queries, mostly by factors of 2 to 5. A few dedicated AQL features in the tests got boosted even more, resulting in query execution time reductions of 90 % and more. Finally, the tests also revealed a dedicated case for which 2.6 provides a several hundredfold speedup.

Also good news is that not a single of the test queries ran slower in 2.6 than in 2.5.

(more…)

More info...

Testing: Foxx + Mocha = <3

Foxx + Mocha

Some of the most frequent questions by new programmers learning Foxx have been about testing. While the HTTP API can easily be tested with any HTTP client and any JavaScript code that can be isolated from the ArangoDB APIs could be tested in similar environments like Node.js or io.js, it was all but impossible to test Foxx apps inside ArangoDB.

The upcoming release of ArangoDB sets out to change this. As of version 2.6, ArangoDB Foxx will come with full support for Mocha tests allowing you to use the same test framework for your frontend code in the browser, your server-side code running on Node.js or io.js, and your backend logic in Foxx. (more…)

More info...

Comparison: Lockless programming with atomics in C++ 11 vs. mutex and RW-locks

ArangoDB is multithreaded and able to use several CPU-cores at once. Because of that access to common data structures to these threads have to be protected from concurrent access. ArangoDB currently uses mutexes, spinlocks and RW-locks for that. With the ongoing development of the MVCC the number of situations where protected access is needed grows significantly. If locking is done too often the scalability is effectively limited to one core. So this test was done to estimate the costs, and evaluate other solutions – so called lockless programming with atomics.

(more…)

More info...

ArangoDB 2.3 Beta 2 is available for testing

This version is deprecated. Download the new version of ArangoDB

A new beta release is available for testing.

Please note that there are issues with the keys generated by the OBS. We will sort this out as fast as possible. Currently you simple have to trust the keys during install.

This beta release contains various fixes which were found during testing beta1.

More info...

Setting Up Test Data

Today I was asked to look at code that was supposed to read data from a MySQL data source, process it and then import it into ArangoDB.

To run and debug the code I had to have some MySQL data source. So I thought I’d quickly set up a simple example table with a few rows. It turned out that this took more time than what I had expected.

Maybe I’m spoilt by JavaScript-enabled, schema-free databases where creating such test setups is so much easier.

Read more on Jan’s Blog

More info...

Why we started to hate OpenSSL – but are still thankful

ArangoDB has an HTTP interface to talk to its clients. Sometimes people want to secure this connection and use SSL or TLS instead. That is where we are using OpenSSL. It provides all the methods to implemented HTTPS on top of an HTTP server. It worked well and the corresponding code is only some 300 lines of C++ code. The biggest obstacle was the documentation. You can basically only learn from examples. That’s what we did. However, we finally encountered a bizarre bug. ArangoDB uses a number of threads to handle I/O in an asynchronous manner. The underlying library for I/O is libev. We span three threads by default each with its own event loop. With HTTP everything is working fine and even HTTPS was no problem.

Until we made a mistake during testing. We started ArangoDB as HTTPS server and ran our unittests. Everything showed green until we connected with an HTTP client to the HTTPS port by accident. The client could not connect and returned an error message – as expected. Meanwhile the unittests started to fail for a few seconds and then recovered. What! This is a completely different socket connection. How is it possible that one connection influences the other?

First idea: We managed to somehow mangle the sockets. So we started to print out file descriptors for accept, read and write, close. As there are a lot of unittests, the output is quite messy. But after hours of debugging, we convinced yourself that everything looked fine.
(more…)

More info...

ArangoDB 1.4.0 & Raspberry PI

This is a follow to the post about ArangoDB 1.3 running on Raspberry PI. With the release of ArangoDB 1.4.0 I now tried to compile it again. As all patches required for 1.3 are part of 1.4.0 it is now much easier.

  pi> sudo apt-get install libreadline6 libreadline6-dev libssl-dev
  pi> git clone -b v1.4.0 https://github.com/triAGENS/ArangoDB
  pi> cd ArangoDB
  pi> ./configure --enable-all-in-one-icu --enable-all-in-one-v8 --enable-all-in-one-libev --disable-mruby
  pi> make

After a few hours, everything (including V8 & ICU) is compiled and you can start ArangoDB on a raspberry

  pi> mkdir /tmp/testbase
  pi> ./bin/arangod -c etc/relative/arangod.conf /tmp/testbase

and the shell in another window

  pi> ./bin/arangosh -c etc/relative/arangosh.conf

or point your browser to http://pi:8529/

However, if you do not want to wait for hours, I’ve tried to build a debian package, which can be downloaded from here:

    pi> wget http://www.arangodb.com/repositories/raspbian/arangodb-1.4.0-raspbian.deb
    pi> sudo dpkg -i arangodb-1.4.0-raspbian.deb

This has not been tested in detail. So any feedback or improvements are welcome.

More info...

Get the latest tutorials,
blog posts and news: