ArangoDB has an HTTP interface to talk to its clients. Sometimes people want to secure this connection and use SSL or TLS instead. That is where we are using OpenSSL. It provides all the methods to implemented HTTPS on top of an HTTP server. It worked well and the corresponding code is only some 300 lines of C++ code. The biggest obstacle was the documentation. You can basically only learn from examples. That’s what we did. However, we finally encountered a bizarre bug. ArangoDB uses a number of threads to handle I/O in an asynchronous manner. The underlying library for I/O is libev. We span three threads by default each with its own event loop. With HTTP everything is working fine and even HTTPS was no problem.
Until we made a mistake during testing. We started ArangoDB as HTTPS server and ran our unittests. Everything showed green until we connected with an HTTP client to the HTTPS port by accident. The client could not connect and returned an error message – as expected. Meanwhile the unittests started to fail for a few seconds and then recovered. What! This is a completely different socket connection. How is it possible that one connection influences the other?
First idea: We managed to somehow mangle the sockets. So we started to print out file descriptors for accept, read and write, close. As there are a lot of unittests, the output is quite messy. But after hours of debugging, we convinced yourself that everything looked fine.