On building AQL Query Strings

00Query Language, SecurityTags:

I recently wrote two recipes about generating AQL query strings. They are contained in the ArangoDB cookbook by now:

After that, Github user tracker1 suggested in Github issue 1457 to take the ES6 template string variant even further, using a generator function for string building, and also using promises and ES7 async/await.

We can’t use ES7 async/await in ArangoDB at the moment due to lacking support in V8, but the suggested template string generator function seemed to be an obvious improvement that deserved inclusion in ArangoDB.

Basically, the suggestion is to use regular JavaScript variables/expressions in the template string and have them substituted safely.

With regular AQL bind parameters, a query looks like this:

This is immune to parameter injection, because the query string and the bind parameter value are passed in separately. But it’s not very ES6-y.

More info

Lockfree protection of data structures that are frequently read

04Architecture, SecurityTags:

Motivation

In multi-threaded applications running on multi-core systems, it occurs often that there are certain data structures, which are frequently read but relatively seldom changed. An example of this would be a database server that has a list of databases that changes rarely, but needs to be consulted for every single query hitting the database. In such situations one needs to guarantee fast read access as well as protection against inconsistencies, use after free and memory leaks.

Therefore we seek a lock-free protection mechanism that scales to lots of threads on modern machines and uses only C++11 standard library methods. The mechanism should be easy to use and easy to understand and prove correct. This article presents a solution to this, which is probably not new, but which we still did not find anywhere else.

The concrete challenge at hand

Assume a global data structure on the heap and a single atomic pointer P to it. If (fast) readers access this completely unprotected, then a (slow) writer can create a completely new data structure and then change the pointer to the new structure with an atomic operation. Since writing is not time critical, one can easily use a mutex to ensure that there is only a single writer at any given time. The only problem is to decide, when it is safe to destruct the old value, because the writer cannot easily know that no reader is still accessing the old values. The challenge is aggravated by the fact that without thread synchronization it is unclear, when a reader actually sees the new pointer value, in particular on a multi-core machine with a complex system of caches.

If you want to see our solution directly, scroll down to “Source code links“. We first present a classical good approach and then try to improve on it. More info

Securing your Foxx with API Keys

02API, Foxx, SecurityTags: ,

ArangoDB’s Foxx allows you to easily build an API to access your data sources. But now this API is either public or restricted to users having an account, but those still get unlimited access.

In many use cases you do not want to expose your data in this fashion, but you want to expose it with a more controllable access pattern and want to restrict the requests one user could issue in a certain time period. Popular examples for these API restrictions are Twitter or Facebook. This allows you to offer all of your data but only in limited chunks, and then possibly charge your customers to increase the chunk limit they can request.

All this is done via API keys, which are bound to a user and has become a common pattern to monetize the data you have collected. More info

Do you like ArangoDB?
icon-githubStar this project on GitHub.
close-link