Community Notebook Challenge
Calling all Community Members! 🥑
Today we are excited to announce our Community Notebook Challenge.
What is our Notebook Challenge you ask? Well, this blog post is going to catch you up to speed and get you excited to participate and have the chance to win the grand prize: a pair of custom Apple Airpod Pros.
(more…)Detecting Complex Fraud Patterns with ArangoDB
Introduction
This article presents a case study of using AQL queries for detecting complex money laundering and financial crime patterns. While there have been multiple publications about the advantages of graph databases for fraud detection use cases, few of them provide concrete examples of implementing detection of complex fraud patterns that would work in real-world scenarios.
This case study is based on a third-party transaction data generator, which is designed to simulate realistic transaction graphs of any size. The generator disguises complex financial fraud patterns of two kinds:
- Circular money flows: a big amount of money is going through different nodes and comes back to the source node.
- Indirect money transfers: a big amount of money is sent from source node to a target node over a multi-layered network of intermediate accounts.
Best Practices for AQL Graph Queries
Estimated reading time: 8 minutes
The ArangoDB Query Language(AQL) was designed to accomplish a few important goals, including:
- Be a human-readable query language
- Client independency
- Support complex query patterns
- Support all ArangoDB data models with one language
The goal of this guide is to ensure Read more
From Zero to Advanced Graph Query Knowledge with ArangoDB
Thinking about your data as a highly connected set of information is a powerful way to gain insights, solve problems and bring products faster into the hands of your users.
Unlike other databases, relationships take the first priority in graph databases and with ArangoDBs multi-model approach for graphs, documents and key/value pairs you can even switch between models or combine them in a single query.
The graph concept is booming but still new to many. So we invested a few bazillion coffees and some night shifts to come up with a good plan for a Graph Course:
Small Things in ArangoDB 2.8: Explain Improvements, POW, Arangoimp
Explain Improvements
Explaining AQL queries becomes even easier in ArangoDB 2.8. While previous versions required writing a hard-to-memorize command like
Killing a long-running query
Suppose there is an AQL query that’s executing in the server for a long time already and you want to get rid of it. What can be done to abort that query?
If a connection to the server can still be established, the easiest is to use the ArangoShell to fetch the list of currently executing AQL queries and send a kill command to the server for the correct query. Read more
On building AQL Query Strings
I recently wrote two recipes about generating AQL query strings. They are contained in the ArangoDB cookbook by now:
After that, Github user tracker1 suggested in Github issue 1457 to take the ES6 template string variant even further, using a generator function for string building, and also using promises and ES7 async/await.
We can’t use ES7 async/await in ArangoDB at the moment due to lacking support in V8, but the suggested template string generator function seemed to be an obvious improvement that deserved inclusion in ArangoDB.
Basically, the suggestion is to use regular JavaScript variables/expressions in the template string and have them substituted safely.
With regular AQL bind parameters, a query looks like this:
var bindVars = { name: "test" };
var query = `FOR doc IN collection
FILTER doc.name == @name
RETURN doc._key`;
db._query(query, bindVars);
This is immune to parameter injection, because the query string and the bind parameter value are passed in separately. But it’s not very ES6-y.
AQL object literal simplification
ArangoDB’s devel
branch recently saw a change that makes writing some AQL queries a bit simpler.
The change introduces an optional shorthand notation for object attributes in the style of ES6’s enhanced object literal notation.
For example, consider the following query that groups values by age
attribute and counts the number of documents per distinct age
value:
FOR doc IN collection
COLLECT age = doc.age WITH COUNT INTO length
RETURN { age: age, length: length }
The object declaration in the last line of the query is somewhat redundant because one has to type identical attribute names and values:
RETURN { age: age, length: length }
In this case, the new shorthand notation simplifies the RETURN
to:
RETURN { age, length }
In general, the shorthand notation can be used for all object literals when there is an attribute name that refers to a query variable of the same name.
It can also be mixed with the longer notation, e.g.:
RETURN { age, length, dateCreated: DATE_NOW() }
Introducing RETURN DISTINCT for AQL queries
Last week saw the addition of the RETURN DISTINCT
for AQL queries. This is a new shortcut syntax for making result sets unique.
For this purpose it can be used as an easier-to-memorize alternative for the already existing COLLECT
statement. COLLECT
is very flexible and can be used for multiple purposes, but it is syntactic overkill for making a result-set unique.
The new RETURN DISTINCT
syntax makes queries easier to write and understand.
Here’s a non-scientific proof for this claim:
Compare the following queries, which both return each distinct age
attribute value from the collection:
FOR doc IN collection
COLLECT age = doc.age
RETURN age
With RETURN DISTINCT
:
FOR doc IN collection
RETURN DISTINCT doc.age
Clearly, the query using RETURN DISTINCT
is more intuitive, especially for AQL beginners. Apart from that, using RETURN DISTINCT
will save a bit of typing compared to the longer COLLECT
-based query.
Internally both COLLECT
and RETURN DISTINCT
will work by creating an AggregateNode
. The optimizer will try the sorted and the hashed variants for both, so they should perform about the same.
However, the result of a RETURN DISTINCT
does not have any guaranteed order, so the optimizer will not insert a post-SORT
for it. It may do so for a regular COLLECT
.
As mentioned before, COLLECT
is more flexible than RETURN DISTINCT
. Notably, COLLECT
is superior to RETURN DISTINCT
when the result set should be made unique using more than one criterion, e.g.
FOR doc IN collection
COLLECT status = doc.status, age = doc.age,
RETURN { status, age }
This is currently not achievable via RETURN DISTINCT
, as it only works with a single criterion.
Diffing Two Documents in AQL
I just stumbled upon a comment in the ArangoDB blog asking how to create a diff of two documents with AQL.
Though there is no built-in AQL function to diff two documents, it is easily possible to build your own like in the following query.
Read more on how to diff two documents in AQL.
Get the latest tutorials,
blog posts and news:
Thanks for subscribing! Please check your email for further instructions.