Massive Inserts into ArangoDB With NodeJS

03GeneralTags: , ,

Nothing performs faster than arangoimport and arangorestore for bulk loading or massive inserts into ArangoDB. However, if you need to do additional processing on each row inserted, this blog will help with that type of functionality.

If the data source is a streaming solution (such as Kafka, Spark, Flink, etc), where there is a need to transform data before inserting into ArangoDB, this solution will provide insight into that scenario as well. Read more

ArangoDB 2.6 – API changes, additions and changed behavior

00API, Documentation, ReleasesTags: ,

ArangoDB 2.6 comes with new and changed APIs as well as changed behavior regarding document keys and several graph functions.

If you use Travis-CI for your tests you can download the Travis-CI ArangoDB build here: Travis-CI/ArangoDB-2.6.0-alpha2.tar.gz

The changes so far:

APIs added

  • added batch document removal and lookup APIs:

    These APIs can be used to perform multi-document lookup and removal operations efficiently. The arguments to these APIs are the name of the collection plus the array of document keys to fetch or remove.

    The endpoints for these APIs are as follows:


    Example call to fetch documents:

    The documents will be returned in an attribute documents of the HTTP response. documents is an array containing all documents found. Only those documents that were actually found will be returned. Documents that were searched but do not exist will not be returned and do not trigger any errors. More info

LoopBack Connector for ArangoDB

00API, Community, nodejsTags:

ArangoDB can be used as a backend data source for APIs that you compose with the popular open-source LoopBack Node.js framework.

strongloop

In a recent blog article on StrongLoop, Nicholas Duffy explains how to use his new loopback-connector-arango connector to access ArangoDB:

Getting Started with the Node.js LoopBack Connector for ArangoDB

The tutorial uses the loopback-connector-arango which is available as npm and a demo application which is available from Github. More info

Bulk Document Lookups

03APITags:

ArangoDB 2.6 comes with a specialized API for bulk document lookups. The new API allows fetching multiple documents from the server using a single request, making bulk document retrieval more efficient than when using one request per document to fetch.

Provided the documents keys are known, all the client application needs to do is to call the collection’s lookupByKeys method:

Additionally, the server-side REST API method for bulk document lookups can be invoked directly via HTTP as follows:

Jan compared the functionality with single document requests in his latest blog post.

Exporting Data for Offline Processing (in PHP)

00APITags:

A few weeks ago I wrote about ArangoDB’s specialized export API.

The export API is useful when the goal is to extract all documents from a given collection and to process them outside of ArangoDB.

The export API can provide quick and memory-efficient snapshots of the data in the underlying collection, making it suitable for extract all documents of the collection. It will be able to provide data much faster than with an AQL query that will extract all documents.

In this post I’ll show how to use the export API to extract data and process it with PHP.

Please read the full blog post Exporting Data for Offline Processing.

More Efficient Data Exports with new Export API

03API, PerformanceTags: ,

ArangoDB 2.6 provides a specialized export API for exporting all documents from a collection and shipping them to a client application. It is rather limited but faster than the general-purpose AQL cursor API and can store its snapshots using less memory.

export_api

A side effect of the speedup is that the first results will arrive much earlier in the client application. This will help in reducing client connection timeouts in case clients are enforcing them on temporarily non-responding connections. More info

Do you like ArangoDB?
icon-githubStar this project on GitHub.
close-link