Milestone 1 ArangoDB 3.3: Datacenter to Datacenter Replication

02General, ReleasesTags: ,

Every company needs a disaster recovery plan for all important systems. This is true from small units like single processes running in some container to the largest distributed architectures. For databases in particular this usually involves a mixture of fault-tolerance, redundancy, regular backups and emergency plans. The larger a data store, the more difficult is it to come up with a good strategy.

Therefore, it is desirable to be able to run a distributed database in one datacenter and replicate all transactions to another datacenter in some way. Often, transaction logs are shipped over the network to replicate everything in another, identical system in the other datacenter. Some distributed data stores have built-in support for multiple datacenter awareness and can replicate between datacenters in a fully automatic fashion.

This post gives an overview over the first evolutionary step of ArangoDB towards multi-datacenter support, which is asynchronous datacenter to datacenter replication.

Read more

Setting up Datacenter to Datacenter Replication in ArangoDB

00Architecture, cluster, General, how to, Releases, ReplicationTags: , ,

Please note that this tutorial is valid for the ArangoDB 3.3 milestone 1 version of DC to DC replication!

Interested in trying out ArangoDB? Fire up your cluster in just a few clicks with ArangoDB Oasis: the Cloud Service for ArangoDB. Start your free 14-day trial here

This milestone release contains data-center to data-center replication as an enterprise feature. This is a preview of the upcoming 3.3 release and is not considered production-ready.

In order to prepare for a major disaster, you can setup a backup data center that will take over operations if the primary data center goes down. For a server failure, the resilience features of ArangoDB can be used. Data center to data center is used to handle the failure of a complete data center.

Data is transported between data-centers using a message queue. The current implementation uses Apache Kafka as message queue. Apache Kafka is a commonly used open source message queue which is capable of handling multiple data-centers. However, the ArangoDB replication is not tied to Apache Kafka. We plan to support different message queues systems in the future.

The following contains a high-level description how to setup data-center to data-center replication. Detailed instructions for specific operating systems will follow shortly. Read more

ArangoDB 3.2 GA
RocksDB, Pregel, Fault-Tolerant Foxx & Satellite Collections

00General, ReleasesTags: , , , ,

We are pleased to announce the release of ArangoDB 3.2. Get it here. After an unusually long hackathon, we eliminated two large roadblocks, added a long overdue feature and integrated an interesting new one into this release. Furthermore, we’re proud to report that we increased performance of ArangoDB on average by 35%, while at the same time reduced the memory footprint compared to version 3.1. In combination with a greatly improved cluster management, we think ArangoDB 3.2 is by far our best work. (see release notes for more details)

One key goal of ArangoDB has always been to provide a rock solid platform for building ideas. Our users should always feel safe to try new things with minimal effort by relying on ArangoDB. Todays 3.2 release is an important milestone towards this goal. We’re excited to release such an outstanding product today. Read more

Updated GraphQL Sync in ArangoDB 3.2

01General, ReleasesTags: ,

Just in time for the upcoming 3.2.0 release, we have updated the graphql-sync module for compatibility with graphql-js versions 0.7.2, 0.8.2, 0.9.6 and 0.10.1. The graphql-sync module allows developers to implement GraphQL backends and schemas in strictly synchronous JavaScript environments like the ArangoDB Foxx framework by providing a thin wrapper around the official GraphQL implementation for JavaScript.

As a long-term database solution, ArangoDB is committed to API stability and avoids upgrades to third-party dependencies that would result in breaking changes. This means ArangoDB will continue to bundle the graphql-js 0.6.2 compatibility version of graphql-sync.

Read more

ArangoDB 3.2 beta release:
Pluggable Storage Engine with RocksDB, Distributed Graph Processing and a ClusterFoxx

01General, ReleasesTags: , , , , , ,

We’re excited to release today the beta of ArangoDB 3.2. It’s feature rich, well tested and hopefully plenty of fun for all of you. Keen to take it for a spin? Get ArangoDB 3.2 beta here.

With ArangoDB 3.2, we’re introducing the long-awaited pluggable storage engine and its first new citizen, RocksDB from Facebook

  • RocksDB: You can now use as much data in ArangoDB as you can fit on your disk. Plus, you can enjoy performance boosts on writes by having only document-level locks (more info below).
  • Pregel: Furthermore, we implemented distributed graph processing with Pregel for discovering hidden patterns, identify communities and perform in-depth analytics of large graph data sets.
  • ClusterFoxx: Another important upgrade is what we internally and playfully call the ClusterFoxx. The Foxx management internals have been rewritten from the ground up to make sure multi-coordinator cluster setups always keep their services in sync and new coordinators are fully initialised even when all existing coordinators are unavailable.
  • Enterprise: Working with some of our largest customers, we’ve added further security and scalability features to ArangoDB Enterprise like LDAP integration, Encryption at Rest, and the brand new Satellite Collections.

The goal of the whole ArangoDB 3 release cycle has been to scale the multi-model idea to new heights. Getting ‘ready’ for large scale applications is not done overnight and it’s definitely not possible without the help of a strong community. We’d like to invite all of you to lend us a helping hand to make ArangoDB 3.2 the best release ever. Please push this beta to its limits: test it for your use cases and compare the performance of the new features like RocksDB. Let us know on Github any bug that you find. Don’t worry about hurting our feelings: we want to fix any problems.

Join the Beta Bug Hunt Challenge and win a $200 Amazon Gift Card as first prize. You can find more details about this reward program at the end of this post. Read more

RocksDB Integration in ArangoDB – FAQs

00Community, General, ReleasesTags: , ,

The new release of ArangoDB 3.2 is just around the corner and will include some major improvements like distributed graph processing with Pregel or a powerful export tool. But most importantly we integrated Facebook’s RocksDB as the first pluggable storage engine in ArangoDB. With RocksDB you will be able to use as much data in ArangoDB as fits on your disc.

As this is an important change and many questions reached us from the community we wanted to share some answers on the most common questions. Please find them below

Will I be able to go beyond the limit of RAM?

Yes. By defining RocksDB as your storage engine you will be able to work with as much data as fits on your disc.

What is the locking behaviour with RocksDB in ArangoDB?

Read more

Alpha3 of ArangoDB 3.2: Support for Distributed Graph Processing

00ReleasesTags: , , ,

The next alpha release of the upcoming ArangoDB 3.2 is available for testing. You can download and install alpha3 here.

Moving forward

As ArangoDB 3.2 will include several new features and improvements, we realized that the release model that we currently follow has room for improvement. Going forward we will introduce milestone releases with ArangoDB 3.3. For this major release you will see a bit more alphas. You can read detailed info about the new structure model here.

Pregel computing model

In this alpha we introduce support for incremental graph processing algorithms in a single mode server as well as in the cluster. Read more

Introducing milestone release model and why is it better

00General, ReleasesTags:

When developing ArangoDB, we want to share with you new features as early as possible. For example, the pregel implementation that will become part of ArangoDB 3.2 was ready for testing weeks before the release date of the first beta release and the final release of 3.2.

Therefore we decided to create intermediate releases, called alpha releases, which contain new feature as early as they became stable. The benefits for the community and also for our developers team are: Read more

Do you like ArangoDB?
icon-githubStar this project on GitHub.
close-link