Performance analysis with pyArango: Part III Measuring possible capacity with usage Scenarios

00General, how to, PerformanceTags: , , , , ,

So you measured and tuned your system like described in the Part I and Part II of these blog post series. Now you want to get some figures how many end users your system will be able to serve. Therefore you define “scenarios” which will be typical for what your users do.
One such a user scenario could i.e. be:

  • log in
  • do something
  • log out

Since your users won’t nicely queue up and wait for other users to finish their business, the pace you need to test your defined system is “starting n scenarios every second”. Many scenarios simulating different users may be running in parallel. If your scenario would require 10 seconds to finish, and you’d start 1 per second, that means that your system needs to be capable to process 10 users in parallel. If it can’t handle that, you will see that more than 10 sessions are running in parallel, and the time required to handle such a scenario will lengthen. You will see the server resource usage go up and up, and finally have it burst in flames.
Read more

Performance analysis with pyArango: Part II
Inspecting transactions

00GeneralTags: , ,

Following the previous blog post on performance analysis with pyArango, where we had a look at graphing using statsd for simple queries, we will now dig deeper into inspecting transactions. At first, we split the initialization code and the test code.

Initialisation code

We load the collection with simple documents. We create an index on one of the two attributes: Read more

Performance analysis using pyArango Part I

00GeneralTags: , ,

Usually, your application will persist of a set of queries on ArangoDB for one scenario (i.e. displaying your user’s account information etc.) When you want to make your application scale, you’d fire requests on it, and see how it behaves. Depending on internal processes execution times of these scenarios vary a bit.

We will take intervals of 10 seconds, and graph the values we will get there:

  • average – all times measured during the interval, divided by the count.
  • minimum – fastest requests
  • maximum – slowest requests
  • the time “most” aka 95% of your users may expect an answer within – this is called 95% percentile

Read more

Reaching and harnessing consensus with ArangoDB

00Architecture, cluster, GeneralTags: ,
nihil novi nisi commune consensu
nothing new unless by the common consensus

– law of the polish-lithuanian common-wealth, 1505

A warning aforehand: this is a rather longish post, but hang in there it might be saving you a lot of time one day.

Introduction

Consensus has its etymological roots in the latin verb consentire, which comes as no surprise to mean to consent, to agree. As old as the verb equally old is the concept in the brief history of computer science. It designates a crucial necessity of distributed appliances. More fundamentally, consensus wants to provide a fault-tolerant distributed animal brain to higher level appliances such as deployed cluster file systems, currency exchange systems, or specifically in our case distributed databases, etc. Read more

Updated Sync & Async Java Drivers with ArangoDB 3.1

00Drivers, JavaTags: ,

The upcoming 3.1 release comes with a binary protocol – VelocyStream – to transport VelocyPack (internal storage format of ArangoDB introduced with the 3.0 release) data between ArangoDB and client applications. VelocyPack stores a superset of JSON, is more compact and has a fast attribute lookup. On the other hand, VelocyStream allows to send VelocyPack in an optimized form over the network. We think it would be the right time to update our official Java Driver to modernize it and to let it be the first to fully support VelocyStream. Read more

AQL optimizer improvements for 2.8

00PerformanceTags: , ,

With the 2.8 beta phase coming to an end it’s time to shed some light on the improvements in the 2.8 AQL optimizer. This blog post summarizes a few of them, focusing on the query optimizer. There’ll be a follow-up post that will explain dedicated new AQL features soon. Read more

AQL Function Speedups in 2.8

00PerformanceTags: , ,

While working on the upcoming ArangoDB 2.8, we have reimplemented some AQL functions in C++ for improved performance. AQL queries using these functions may benefit from using the new implementation of the function.

The following list shows the AQL functions for which a C++ implementation has been added in 2.8. The other C++-based AQL function implementations added since ArangoDB 2.5 are also still available. Here’s the list of functions added in 2.8: Read more

Using Multiple Indexes Per Collection

03Documentation, PerformanceTags: ,

The query optimizer in ArangoDB 2.8 has been improved in terms of how it can make use of indexes. In previous versions of ArangoDB, the query optimizer could use only one index per collection used in an AQL query. When using a logical OR in a FILTER condition, the optimizer did not use any index for the collection in order to ensure the result is still correct.

This is much better in 2.8. Now the query optimizer can use multiple indexes on the same collection for FILTER conditions that are combined with a logical OR. Read more

Do you like ArangoDB?
icon-githubStar this project on GitHub.
Star ArangoDB on GitHub