FiftyThree achieves 10,000 reqs/sec load balancing with the right tools plus Neo4j
FiftyThree is the creator of the Paper, the award-winning iPad app powered by Neo4j.
When we began we had no idea how much traffic we would get. So we did what any experienced engineering team would do when faced with potentially huge load and millions of dollars on the line: we guessed.
Based on the total number of Paper users, downloads per month, average time spent in the app, and our API polling frequency, we estimated that we could face peak traffic of around 1000 requests per second, and meltdown traffic of over 5000 requests per second. These numbers became our goal.
Smooth Start: The Database and Load Balancer
We began by testing our database for the baseline performance of our foundational layer. Using open-source tools like siege and httperf, we recorded dismal metrics under 200 reqs/sec. We worked with the Neo4j support team, however, to fine-tune our Neo4j configuration. We were then able to achieve close to 2000 reqs/sec across the cluster.
We were encouraged by a quick but important discovery: for our type of traffic, high-CPU instances were much more effective than high-memory ones. Our database fits into memory and will for quite a while, even at scale. Neo4j runs in a JVM with a memory size fixed at service start, so we had maximized memory settings to use all available RAM. Our experiments showed, however, that we could get the same effect with less memory and more CPU.
Once we knew the baseline numbers for requests directly to our database, we tested with HAProxy in front and saw no performance loss. Since we knew our caching layer would only improve our ability to handle traffic, we were satisfied with our core components.
Special thanks to Neo4j engineer Max De Marzi for lending us his expertise.