Small Things in ArangoDB 2.8: Explain Improvements, POW, Arangoimp

Explain Improvements

Explaining AQL queries becomes even easier in ArangoDB 2.8. While previous versions required writing a hard-to-memorize command like

to explain an AQL query from the ArangoShell, 2.8 reduces this task to a mere

Apart from that, explain in 2.8 is smarter when confronted with very lengthy query strings, and with queries that contain huge hard-coded string, array, or object values. More »

Posted in General, Query Language, Releases | Leave a comment

Bi-Weekly Newsletter #41 | ArangoDB 2.8 release

Big news this week – ArangoDB 2.8 is generally available for download. This is a huge step forward for our still young project, great improvements and new features await exploration. To name just some of the biggest ones…

  • Graph Pattern Matching enables you to traverse a graph using plain AQL
  • Arrax Indexes allow quick access to data stored in arrays, e.g. indexing tags in articles
  • Automatic deadlock detection for transactions.

With 2.8 you will benefit from dozens of internal changes that improve performance and robustness of the database, e.g. more than 30 existing AQL functions have been backed with a C++ implementation.

More than that, we are delighted that we can share some new case studies of clients using ArangoDB in production. Flightstats Inc. (OR) – the leading provider of real-time global flight data and Liaison Technologies (GA), a global data management and integration company, shared how they use ArangoDB. More »

Posted in General | Leave a comment

Killing a long-running query

Suppose there is an AQL query that’s executing in the server for a long time already and you want to get rid of it. What can be done to abort that query?

If a connection to the server can still be established, the easiest is to use the ArangoShell to fetch the list of currently executing AQL queries and send a kill command to the server for the correct query.

To start, we can fetch the list of all running queries and print their ids, query strings and runtimes. This is only inspection and does not abort any query:

Here’s an example result for the list of running queries:

To now kill a query from the list, we can pass the query’s id to kill:

If a query was actually killed on the server, that call should return without an error, and the server should have logged a warning in addition. More »

Posted in General, Query Language | Leave a comment

ArangoDB 2.8 w/ AQL Graph Traversals, Array Indexes, Aggregation

We welcome 2016 with our first big news yet – the release of ArangoDB 2.8!

Now you can use new AQL keywords to traverse a graph even more convenient – a big deal for those who like to get the maximum out of their connected data. ArangoDB is getting faster with every iteration, in this release we have implemented several AQL functions and arithmetic operations in super-fast C++ code, optimizer rules and indexing improved further to help you getting things done faster. Download ArangoDB 2.8 here.

Array Indexes

The added Array Indexes are a major improvement to ArangoDB that you will love and never want to miss again. Hash indexes and skiplist indexes can now be defined for array values as well, so it’s freaking fast to access documents by individual array values. Let assume you want to retrieve articles that are tagged with “graphdb”, you can now use an index on the tags array:

An added hash-index on tags (ensureHashIndex("tags[*]")) can be used for finding all documents having "graphdb" somewhere in their tags array using the following AQL query:

Have fun with these new indexes!


AQL Graph Traversal

Next, the mentioned AQL graph traversals. The query language AQL adds the keywords GRAPH, OUTBOUND, INBOUND and ANY for use in graph traversals. Using plain AQL in ArangoDB 2.8 you can create a shopping list for your friends birthday gifts, related to products they already own and up to 5 ideas ordered by price.

More »

Posted in General, Releases | 4 Comments

Maintenance Release – ArangoDB 2.7.5

Still waiting for the 2.8 release announcement…
So in the meantime, let’s have a look at the latest maintenance release of ArangoDB 2.7.

Here are the changes that come with version 2.7.5:

  • backported added automatic deadlock detection for transactions

    In case a deadlock is detected, a multi-collection operation may be rolled back automatically and fail with error 29 (deadlock detected). Client code for operations containing more than one collection should be aware of this potential error and handle it accordingly, either by giving up or retrying the transaction.

  • improved internal datafile statistics for compaction and compaction triggering conditions, preventing excessive growth of collection datafiles under some workloads. This should also fix issue #1596.

  • Foxx export cache should no longer break if a broken app is loaded in the web admin interface.

  • Foxx: removed some incorrect deprecation warnings.

  • Foxx: mocha test paths with wildcard characters (asterisks) now work on Windows

Posted in Releases | Leave a comment

AQL optimizer improvements for 2.8

With the 2.8 beta phase coming to an end it’s time to shed some light on the improvements in the 2.8 AQL optimizer. This blog post summarizes a few of them, focusing on the query optimizer. There’ll be a follow-up post that will explain dedicated new AQL features soon.

Array indexes

2.8 allows creating hash and skiplist indexes on attributes which are arrays. Creating such index works similar to creating a non-array index, with the exception that the name of the array attribute needs to be followed by a [*] in the index fields definition:

Now if the tags attribute of a document in the posts collection is an array, each array member will be inserted into the index:

The index on tags[*] will now contain the values arangodb, database, aql and nosql for the first document, arangodb, v8 and javascript for the second, and javascript, v8 and nodejs for the third.
More »

Posted in Performance | Leave a comment

Running ArangoDB on a Mac with Docker and Kitematic / Docker-Machine

When I work with ArangoDB on my Mac, I usually install ArangoDB via homebrew and do tests on the latest new developments based on the devel-branch, compiling ArangoDB right from source.

To test a feature in a special version I use docker images that need a virtual machine on MacOS. I struggled with Boot2Docker several times and recently tried Docker Machine – with the Web UI Kitematic – currently in beta.

Here’s how to start with ArangoDB and Docker Machine:

First you need to install Docker Machine as described here. In my case, the installation process detects an earlier Boot2Docker Installation and migrates it to a new Docker Machine VM. Nice.

Screen Shot 2016-01-18 at 16.34.38

During the installation process you can choose to install the Docker Machine and Kitematic (current 0.8). I installed both to have a try. After installation I chose the fancy UI stuff and opened Kitematic.

Searching for ArangoDB you get the official Docker Hub Image as the first result – and by default, the latest stable release.

Screen Shot 2016-01-15 at 16.47.17

Just one click on “create” and your ArangoDB container is up and running! More »

Posted in Documentation | Leave a comment

ArangoDB 2.8 (beta 3)

ArangoDB 2.8 (beta3) is available for testing.

The last beta release of ArangoDB 2.8 – at least for 2015 – comes with the following bugfixes and improvements:

  • web interface: fixed a graph display bug concerning dashboard view

  • web interface: fixed several bugs during the dashboard initialize process

  • web interface: included several bugfixes: #1597, #1611, #1623

  • added --create-collection-type option to arangoimp

    This allows specifying the type of the collection to be created when --create-collection is set to true.

  • AQL query optimizer now converts LENGTH(collection-name) to an optimized expression that returns the number of documents in a collection More »

Posted in Releases | Leave a comment

ArangoDB 2.7.3 – Maintenance release

ArangoDB 2.7.3 – a maintenance release – is ready for download.

Changes:

  • fixed disappearing of documents for collections transferred via sync or syncCollection if the collection was dropped right before synchronization and drop and (re-)create collection markers were located in the same WAL file

  • fixed an issue where overwriting the system sessions collection would break the web interface when authentication is enabled

Posted in General | Leave a comment

ArangoDB 2.8 (beta 2)

The second beta of ArangoDB 2.8 is available for testing. Now it’s your turn – give it a try, report bugs and provide us with your early feedback on the new features (Array Indexes and graph traversal in AQL). Do you like what you see?

Here’s what changed recently (beta 1 / beta 2):

  • added AQL query optimizer rule “sort-in-values”

    This rule pre-sorts the right-hand side operand of the IN and NOT IN operators so the operation can use a binary search with logarithmic complexity instead of a linear search. The rule is applied when the right-hand side operand of an IN or NOT IN operator in a filter condition is a variable that is defined in a different loop/scope than the operator itself. Additionally, the filter condition must consist of solely the IN or NOT IN operation in order to avoid any side-effects.

  • changed collection status terminology in web interface for collections for which an unload request has been issued from in the process of being unloaded to will be unloaded.

  • unloading a collection via the web interface will now trigger garbage collection in all v8 contexts and force a WAL flush. This increases the chances of performing the unload faster.

  • added the following attributes to the result of collection.figures() and the corresponding HTTP API at PUT /_api/collection/<name>/figures:

    • documentReferences: The number of references to documents in datafiles that JavaScript code currently holds. This information can be used for debugging compaction and unload issues.
    • waitingFor: An optional string value that contains information about which object type is at the head of the collection’s cleanup queue. This information can be used for debugging compaction and unload issues.
    • compactionStatus.time: The point in time the compaction for the collection was last executed. This information can be used for debugging compaction issues.
    • compactionStatus.message: The action that was performed when the compaction was last run for the collection. This information can be used for debugging compaction issues.

    Note: waitingFor and compactionStatus may be empty when called on a coordinator in a cluster.

  • the compaction will now provide queryable status info that can be used to track its progress. The compaction status is displayed in the web interface, too.

  • better error reporting for arangodump and arangorestore

  • arangodump will now fail by default when trying to dump edges that refer to already dropped collections. This can be circumvented by specifying the option --force true when invoking arangodump

  • fixed cluster upgrade procedure

  • the AQL functions NEAR and WITHIN now have stricter validations for their input parameters limit, radius and distance. They may now throw exceptions when invalid parameters are passed that may have not led to exceptions in previous versions.

  • deprecation warnings now log stack traces

  • Foxx: improved backwards compatibility with 2.5 and 2.6

    • reverted Model and Repository back to non-ES6 “classes” because of compatibility issues when using the extend method with a constructor

    • removed deprecation warnings for extend and controller.del

    • restored deprecated method Model.toJSONSchema

    • restored deprecated type, jwt and sessionStorageApp options in Controller#activateSessions

Posted in Releases | Leave a comment