Goodbye, Heroku! Hello, Docker, Kubernetes on AWS!

It has been a couple of months since my last post. We’ve been lucky to get some rockstar developers to join us in our journey. One major infra change that we did was to move from Heroku to Kubernetes. We will also write a series of posts on how we’ve setup our kubes, but this post is about why we moved off Heroku.

Heroku was fantastic for us in the early stages. It allowed us to write services in different languages. It allowed us to setup custom domains and SSL easily enough. It allowed us to push updates with just a git push. Most importantly, deploying to Heroku helped maintain all dependencies cleanly — we didn’t fall into the trap of installing custom libraries on the production machine by hand and then forget about it.

So why did we move off Heroku? The main reasons were:

  • We were running our services in us-east region with Heroku whereas our customers were in India. This caused some noticeable latency issues when we started getting serious users. Heroku only supports us-east and eu regions currently.
  • We wanted to setup a staging environment (we’re growing up!). We had deployed our services as separate apps in Heroku (we had 5 IIRC). It seemed error prone to maintain two environments by hand. Interestingly, Heroku came up with Pipelines around the time we were migrating.
  • Usage was getting to a point when we couldn’t afford downtime. We needed to have rolling deploys with no downtime. While this is theoretically possible with Heroku, it would tie us too tightly to Heroku.
  • We were getting feedback from prospects that bigger companies would want a private deployment of our product within their infrastructure (or within the geographical region). This would be impossible to do with Heroku.

Given all of these considerations, we started looking at alternatives. We figured we needed three pieces: a container mechanism, a container orchestration system and a public cloud provider. After doing a little bit of research, we decided to go with Docker, Kubernetes and AWS.

Why AWS and not Google? We had a bunch of AWS credits (thanks, folks!). I was very familiar with AWS. At my previous workplace at Qubole, we used to hammer EC2 and S3 and I was super-impressed by how robust AWS was. Finally, we knew AWS was launching in India soon (and they did). As an aside, AWS seems to be routing credits to early stage startups through events, shared spaces and investors and getting them hooked early — Google really needs to up its sales game.

Some of you may be wondering what the hell is a Docker and a Kubernetes and why should I care? This is a little beyond the scope of this article — but I will use a food-based analogy to summarize.

www.abrowntable.com

The service you write is the food you prepare and eat. In the above picture, it happens to be noodles. It could very well be a hot curry. Or some breakfast cereal. Often times, you end up preparing and eating a couple of different dishes. Similarly, your services could be written in Python, Go, Java or whatever.

The bowl that holds on the food while you eat is the container. The same bowl can hold a variety of dishes. Bowls do come in different sizes. Usually, people don’t create a new bowl when they cook something new. Similarly, you shouldn’t write a bunch of new bash scripts from scratch to package a new service. Docker is the biggest player here and we went with it, but there are a couple of alternatives like Rocket. Whenever we want to deploy a new version of an app, we would create a new docker image.

Finally, the table on which the bowls sit is the orchestration system. The system must support placement of multiple bowls. The system must allow for adding an extra bowl of rice before removing the old one so that hungry people aren’t left waiting. This is termed a rolling upgrade. If the soup is really popular, it must support adding another bowl of soup. This is replica management and load balancing. Finally, containers need to be able to discover and talk to each other — there really isn’t an analogy for this :)

We compared Kubernetes and Docker Swarm and it was a difficult choice — both were attacking the same problem and both were in relatively early stages. Kubernetes seemed more mature and it seemed like more organizations were running Kubernetes in production than Docker Swarm. So we went with Kubernetes. It would be interesting to see how this ecosystem develops.

It has been almost 2 months since we migrated to Docker / Kubernetes running in AWS Singapore and we’ve been super happy. Kubernetes has been pretty solid and our services seems much snappier — I suspect Heroku had additional overheads. We managed to setup a staging environment within a day after getting production environment running. We also run kubernetes on our laptops for development — this was probably the one hard part and there wasn’t much on the web on how to do this. These are all topics for future posts — so stay tuned.

In conclusion, I’d say Heroku was a great choice for us to get started. It allowed us to not worry about deployment and upgrades and I highly recommend it for early stage startups. It was time for us to move on, though and so it’s goodbye, Heroku!