Presenting our bleeding edge stack

Hey everyone! It’s been sometime since our last post so I'd like to present the technologies we are using to build Hacker Experience 2. But before diving in details I must remind you of our philosophy: “Above all, programming should be fun”. And you know that “no risk no fun”.

We rely on a bunch of brand new technologies, many of which have no more than 3 years in the market. Learning them is fun, and the risk is part of the game. So, let’s start the journey!

One of our goals is to reach as many platforms as possible. You should be able to play Hacker Experience 2 on your browser, computer, smartphone, video game or even on your good ol’ TTY, as long as you have a working Internet connection. The best way to do that is creating a core entity that holds the entire game logic and can behave as an API. The interface (your browser, desktop, mobile) just connects to the API and displays the information in a nice UI.

Since we will release HE2 as open source, I should say that our public API uses Websocket instead of HTTP, which provides much less overhead on the wire. But Roy Fielding’s followers won’t be disappointed. Although no HTTP verbs, one could say our API is REST-y.

The core entity (“core” from now on) is divided into several microservices, each of which has it’s very own purpose. (I’ll talk later about Microservices and MicroservicePremium in greater detail). The first service you hit when sending a websocket request is the Dispatcher, which will validate the payload and route the request to the relevant service, according to the REST-y topic you specified.

A single request incur in several messages being sent, usually in a ping-pong style, across each service. The man behind this is RabbitMQ, a fast and reliable message broker. (We also use ZeroMQ for internal logging)

Back to the services. The Dispatcher and all other services that subscribe to it are built using Python 3. Specifically, we use Python’s new asynchronous programming library, asyncio released on version 3.4. It allow us to do non-blocking I/O, so, for example, while one request is waiting for the Database to reply, or for another service’s response, it can process a new request. Whenever the response is ready, that first request gets to be processed.

One of the oldest on the stack, we use PostgreSQL as our RDBMS. The default setup is not fun, so we use streaming replication with WAL (Write-Ahead Logging). Then, our application sends all write queries to the master, and read-only queries to the slave. This way we can tune each server to it’s best use case.

Between the core processing and the database there is a cache layer, because you know, cache. We started with Redis Cluster (unstable at the time) and then switched to Aerospike. Aerospike provides better clustering capabilities and, for our use case, is at least as fast as Redis. We can also store data on SSD (but indexes on RAM) more easily.

The cool part comes now: logs and processes are fundamental features of the game. They are stored at Elasticsearch (now only Elastic) and processed by Elixir. This gives us blazing fast querying capabilities, as well as much easier scaling. And here is the obligatory comment from any programmer introduced to Elixir: it’s a wonderful language . (And so is Erlang!)

The best part of relying on a microservice architecture is that we can choose the best language for the job. Just as we did with Elixir, we also use Ruby at the cache service. That’s because Ruby was the most suitable Aerospike client we found (the Erlang one misses fundamental features).

Of course that managing dozens, maybe hundreds of services is no piece of cake. But it gets manageable, and even fun (ha!) with Docker, Phusion Baseimage, Consul, ELK, collectd, Mesos and Marathon. Phew. I probably forgot some, but you got the gist.

This was just the core! We are also bleeding-edge at the UI.

Our web interface is being built on top of Node.js (which can be run on both Desktop and Browser). We also rely on Facebook’s charming technology trio: React + Flux + Immutable.js.
So far, React has proven to be extremely easier to manage our application views.

For mobile development we are using Facebook’s **React Native, released last March. Currently it only supports iOS, but I believe the Android version will be out soon. If not, heads up Scala!

I hope I was able to give you a glimpse of what Hacker Experience 2 is becoming. And all this will be released openly. I’m sure we all will learn a lot with it. Join the journey, subscribe!

Author image
São Carlos, Brazil Website
Former PHP victim, currently working with Elixir, Elm, distributed systems and some hardcore systems administration.