ArangoDB 2.6 provides a specialized export API for exporting all documents from a collection and shipping them to a client application. It is rather limited but faster than the general-purpose AQL cursor API and can store its snapshots using less memory.
A side effect of the speedup is that the first results will arrive much earlier in the client application. This will help in reducing client connection timeouts in case clients are enforcing them on temporarily non-responding connections.
The following table shows the execution times for fetching the first 1,000 documents from collections of different sizes, both with via the cursor API and the export API. Figures for the cursor API are shown for ArangoDB 2.5 and 2.6 (the version in which it was rewritten):
Read more about the new export API in Jan’s blog post on more efficient data exports.
3 Comments
Really nice! Any plans on extending the export API yet? Like a consistent snapshot of an entire database? Alternatively, a read-only maintenance mode would also do, and maybe even work in clusters.
Sorry for the delay, but I stumbled upon your comment just now.
So far the export API is pretty limited. It is designed purely to get a single collection’s documents out as efficiently as possible. There are no concrete thoughts yet about extending the export API into the directions you mentioned, though I won’t exclude that this is a future option.
Regarding the snapshotting:
For creating a snapshot of an entire database, there is arangodump. AFAIK it should create a consistent snapshot of the database even if data modifications happen while its dumping. There are some restrictions though: the dump won’t be consistent in a cluster and there might be issues if collections are dropped while the dump is ongoing. Finally, a full arangodump run may take a long time when run on a bigger database. But for smaller instances arangodump should be fine already.
Sorry for the late reply… Yes, we released a new arangoexport feature. Please read more here: https://www.arangodb.com/2017/02/arangoexport-tool-exporting-data-arangodb/