ArangoDb Weekly #33 | V8 upgraded, ArangoDB 2.6.7

In the last two weeks we’ve fixed some issues, upgraded the bundled V8 version in ArangoDB and released the maintenance release 2.6.5 and 2.6.7 . Besides that we continued our efforts to enhance our indexing capabilities and startet further cluster tests.

ArangoDB Release

The next maintenance release for ArangoDB 2.6 is available for download. You can find a full list of changes in our change-log (2.6.7) and change-log (2.6.5).

Download ArangoDB 2.6.7

ArangoDB related (Drivers & more)

  • (Elasticsearch) elasticsearch-river-arangodb: River Plugin for ElasticSearch, now builds against ES 1.7 (Patrick Huber)
  • (PHP) zend-arangodb-auth (early stage): provides auth and acl for zend framework (Cyberrebell)
  • (Python) python-arango (2.0.0): API call methods for user management and monitoring (Joohwan Oh)
  • (vert.x) vertx-arango-client (early stage): ArangoDB client library built with vert.x. (Craig G)

More »

Posted in Newsletter | Leave a comment

ArangoDB 2.6.7 – Maintenance Release

A new maintenance release 2.6.7 is available for download.

Changelog:

  • Improved edge index performance when using collections with more than 100 Mio. edges
  • added startup option --server.additional-threads to create separate queues for slow requests.
Posted in Releases | Leave a comment

ArangoDB 2.6.5 – Maintenance Release

ArangoDB 2.6.5 is available for download. This is a bugfix release that solves the following issues:

  • fixed busy wait loop in scheduler threads that sometimes consumed 100% CPU while waiting for events on connections closed unexpectedly by the client side

  • handle attribute indexBuckets when restoring collections via arangorestore. Previously the indexBuckets attribute value from the dump was ignored, and the server default value for indexBuckets was used when restoring a collection.

  • fixed “EscapeValue already set error” crash in V8 actions that might have occurred when cancelling V8-based operations.

Posted in Releases | Leave a comment

ArangoDB Weekly #32 | Throughput Enhancements, ArangoDB 2.6.4

In the past two weeks we’ve celebrated some lock-hunting days, trying to eliminate read/write locks to enhance throughput . Furthermore, we did some large cluster tests on GCE, which will be continued over the next days. A maintenance release of ArangoDB 2.6, released earlier this week, comes with a new V8 engine.

ArangoDB Releases

Two maintenance releases are available. ArangoDB 2.5.7 and ArangoDB 2.6.4 use an upgraded V8 engine (4.1.0.27).

Download ArangoDB 2.6.4

Articles and Presentations

More »

Posted in Newsletter | Leave a comment

Lockfree protection of data structures that are frequently read

Motivation

In multi-threaded applications running on multi-core systems, it occurs often that there are certain data structures, which are frequently read but relatively seldom changed. An example of this would be a database server that has a list of databases that changes rarely, but needs to be consulted for every single query hitting the database. In such situations one needs to guarantee fast read access as well as protection against inconsistencies, use after free and memory leaks.

Therefore we seek a lock-free protection mechanism that scales to lots of threads on modern machines and uses only C++11 standard library methods. The mechanism should be easy to use and easy to understand and prove correct. This article presents a solution to this, which is probably not new, but which we still did not find anywhere else.

The concrete challenge at hand

Assume a global data structure on the heap and a single atomic pointer P to it. If (fast) readers access this completely unprotected, then a (slow) writer can create a completely new data structure and then change the pointer to the new structure with an atomic operation. Since writing is not time critical, one can easily use a mutex to ensure that there is only a single writer at any given time. The only problem is to decide, when it is safe to destruct the old value, because the writer cannot easily know that no reader is still accessing the old values. The challenge is aggravated by the fact that without thread synchronization it is unclear, when a reader actually sees the new pointer value, in particular on a multi-core machine with a complex system of caches.

If you want to see our solution directly, scroll down to “Source code links“. We first present a classical good approach and then try to improve on it. More »

Posted in Architecture, Security | 4 Comments

Throughput Enhancements – a short Comparison with ArangoDB 2.6

We’ve recently been working on improving ArangoDB’s throughput, especially when using the ArangoDB’s interface.

In this post, I will show some of the improvements already achieved, though the work is not yet finished. Therefore, the results shown here are still somewhat preliminary.

We wanted to measure improvements for ArangoDB’s HTTP interface, and so we used wrk as an external HTTP load generator.

During the tests, wrk called some specific URLs on a local ArangoDB instance on an otherwise idle machine. The test was run with ArangoDB 2.6 and devel. The ArangoDB instances were started with their default configuration.

wrk was invoked with varying amounts of client connections and threads, so the tests cover serial and concurrent/parallel requests:

bash invoking wrk

The number of connections ($CONNECTIONS) and threads ($THREADS) were both varied from 1 to 8. wrk requires at least as many connections as threads.

More »

Posted in Performance | Leave a comment

ArangoDB 2.6.4 Maintenance Release

ArangoDB Version 2.6.4 comes with an upgraded V8 engine (4.1.0.27) and is ready to download now. In the 2.5 branch we’ve published a 2.5.7 maintenance release as well.

arangodb.com/download

Posted in Releases | Leave a comment

Running V8 isolates in a multi-threaded ArangoDB database

ArangoDB allows running user-defined JavaScript code in the database. This can be used for more complex, stored procedures-like database operations. Additionally, ArangoDB’s Foxx framework can be used to make any database functionality available via an HTTP REST API. It’s easy to build data-centric microservices with it, using the scripting functionality for tasks like access control, data validation, sanitation etc.

We often get asked how the scripting functionality is implemented under the hood. Additionally, several people have asked how ArangoDB’s JavaScript functionality relates to node.js.

This post tries to explain that in detail.

More »

Posted in API, Architecture, C++, Documentation, nodejs | Leave a comment

ArangoDB Weekly #31 | Official Docker Repo & new Release 2.6.3

ArangoDB is now an Official Repo in the Docker Hub, one of just four additions in the last 2 months. Please try and tell your friends! ArangoDB 2.6 is known as a performance release and we’ve continued to improve the core by killing locks and optimizing code. Looks like we can show some impressive performance boosts soon. Furthermore, Mike Williamson wrote a blog post on modeling data with ArangoDB last week, that is worth to read.

Follow ArangoDB on LinkedIn and add ArangoDB as a skill. We would appreciate your help. Keep an eye on our blog or follow us on Twitter for news about ArangoDB.

More »

Posted in General | Leave a comment

AQL object literal simplification

ArangoDB’s devel branch recently saw a change that makes writing some AQL queries a bit simpler.

The change introduces an optional shorthand notation for object attributes in the style of ES6’s enhanced object literal notation.

For example, consider the following query that groups values by age attribute and counts the number of documents per distinct age value:

The object declaration in the last line of the query is somewhat redundant because one has to type identical attribute names and values:

In this case, the new shorthand notation simplifies the RETURN to:

In general, the shorthand notation can be used for all object literals when there is an attribute name that refers to a query variable of the same name.

It can also be mixed with the longer notation, e.g.:

Posted in Query Language | Leave a comment