ArangoDB | Pronto Move Shard – Multi-Model NoSQL Database

In July Adobe announced that they plan the End-of-Life for flash at around 2020.
As HTML5 progressed and due to a long history of critical security vulnerabilities this is – technologically speaking – certainly the right decision. However I tended to also become a bit sad.

Flash was the first technology that brought interactivity to the web. We tend to forget how static the web was in the early 2000s. Flash brought life to the web and there were plenty of stupid trash games and animations which I really enjoyed at the time. As a homage to the age of trashy flash games I created a game which resembles the games of this era:

Game info

To start the game select a “shard” in the top right corner. Then the shard will appear as a yellow “dot” and the servers as “cards” on the table. Finally upon pressing the “Pronto move shard” button the dealer will move the shard around and once finished you have to guess where it is. More on shards and servers in the following sections. Make sure to enable sound (and put your headphones on if you are in an open office).

Please note that the game was not created in a responsive fashion. If you want you can open it in a new window.

Any similarities to a german trash tv show from the 80s are totally intended.

For the best and the most authentic experience you should have the `Comic Sans MS` font on your system. Windows and Mac should have it out of the box.

What is the game about?

The version above is the standalone version. The real version is only playable on an ArangoDB Cluster:

It will move around data in your cluster and you have to guess where the shard was being moved to. Please note that this is not at all a fake process. It will indeed move data around within the cluster!

A short introduction to some ArangoDB features

Foxx

The game has been implemented as a so called Foxx App. ArangoDB allows you to run microservices within the database:

Once installed the service is being hosted inside the database. This offers several possibilities:

1. Data locality: Your logic will sit alongside the data and execute a lot faster as you save network overhead
2. You have access to the internal ArangoDB APIs

This game is only possible because we have access to the ArangoDB cluster API and access to the Agency. Data locality is in this case not really relevant as we are not accessing any data. However this the use case most users will use when doing Foxx apps.

Cluster

In a clustered setup ArangoDB organizes its data in so called `shards`. The data is being partitioned using some key and stored on multiple database servers. When creating collections (they are our “tables”) you can control the `numberOfShards` and the `replicationFactor`.

`numberOfShards` will determine how many shards are being created and `replicationFactor` specifies how many replicas across the cluster will be held. By setting the `replicationFactor` you allow the cluster to be able to fail over to a different machine in case of an outage and achieve `high availability`.

Supervision

As a user you don’t really have to worry about failovers because we have the so called `Supervision` running within the `Agency` that will manage the cluster all by itself and will organize failovers in case a cluster node goes down. Apart from the failover case there is however also a managed shard movement. This comes in handy if you for example want to upgrade a machine or if you want to manually move traffic away from some nodes.

This is what this game will use internally and when deployed on the cluster it will look like this:

For the game any shard is considered moveable when there are at least 3 possible servers that the shard could be moved to. When you have a shard that has a `replicationFactor` of 3 and you have 5 servers in total the shard will not be considered moveable. If you add another server the shard will be moveable.

After pressing the “Pronto move shard” button the Foxx endpoint starting the move will choose a random free server.

It will then periodically check if the move job has been finished and if not it will fake some movements across the servers (in the background there is of course just ONE move job happening).

When the job is finally finished it will simulate a few more movements (it is not that simple 😉 ) and ensure the shard is properly placed. Then you can make your guess.

Please note that it is not advisable to play this game on a production cluster. The game was made up very quickly just for illustration purposes. There are many edge cases that are not handled in the game (like servers going down while moving). Also moving shards around will put a lot of pressure on your cluster. Especially if there is a lot of data in the shard.

Code

The code is available on github (please note that the code quality also resembles the early days). Also thanks to Michael – our graph expert – that I could use his photo for this game.

Your move

The game is installable via our foxx store. So go ahead, install an ArangoDB cluster via DC/OS, via our starter, or manually and install `pronto-move-shard` and make your guess. Be sure to reach out in case of questions.

If you like foxx and did something with it feel free to create a pull request to our registry, so that others can install your experiments too.

For more information about the cluster architecture and how you can achieve high availability be sure to check out the chapter about the cluster architecture in our documentation.