Host your own Heroku-style platform with Deis and Azure

The Heroku workflow is fantastic. If done right, it allows developers to quickly build, release, run and scale applications without having to worry about the environment, rollbacks or releases. It's so fantastic, that the guys from Deis.io build a powerful open-source Heroku-style platform on top of the slim Linux distro CoreOS and the application container system Docker. The guys behind Deis flew over from Boulder to San Francisco, joining me and some of my fellow open source Microsofties for a few days to make the deployment of your own Heroku-Style cluster as simple as possible. The result are a few scripts that automate large parts of the creation of the cluster. Awesome, eh?

What the heck is Deis

Deis is a lightweight application platform that deploys and scales Twelve-Factor apps as Docker containers across a cluster of CoreOS machines. It features an automated Build/Release/Run workflow: Deis takes a git push and creates a new Docker image for the application. It then couples said image with the app's configuration to create a new numbered release. The release numbering is done every single time either code or configuration changes, making rollbacks dead-simple. The run stage then takes the release and rolls it out. The old app containers are only collected after the new containers are live and serving traffic – providing zero-downtime deploys. Oh, and Deis runs anywhere and is open source. In summary, it's delightful to work with.

Creating a Deis Cluster (in minutes)

Deis does have a few daunting requirements - ideally, it runs on a CoreOS cluster. Setting up clusters is rarely fun, but we created a small Python Script that does all of the heavy lifting for you. Go ahead and grab the requirements (the Deis repo, Python 2.7 with pip and the Azure SDK for Python):

$ git clone https://github.com/deis/deis.git && cd ./deis/contrib/azure
$ brew install python
$ sudo pip install azure

The script will automate most of the cluster setup, but we do need to authorize it with a certificate. Enter the folder contrib/azure and run ./generate-mgmt-cert.sh - the command will use details in cert.conf to create a management certificate, so feel free to change it to your liking. To let Azure know that this is your certificate, log in to the Azure portal's certificate management. Select upload and select the just created azure-cert.cer file. While in the portal, also save the ids of the subscription and the name of the blob storage container you'd like to create the cluster in. If you don't have one yet, create one.

Cluster Configuration

Ready to create a cluster? You might want to configure a few basic things. The script create-azure-user-data will use defaults in ../coreos/user-data.example to create a configuration for you. If you want to stick with the defaults, go and run ./create-azure-user-data $(curl -s https://discovery.etcd.io/new) to create a configuration file.

Running the script

Our script is authorized, we have a configurtion file - in short, we're good to go!

./azure-coreos-cluster [cloud service name]
     --subscription [subscription id]
     --azure-cert azure-cert.pem
     --num-nodes 3
     --location "[location]"
     --vm-size Large
     --pip
     --deis
     --blob-container-url https://[blob container].blob.core.windows.net/vhds/
     --data-disk
     --custom-data azure-user-data

By default, the script will provision a 3 node cluster but you can increase this with the num-nodes parameter. Note that for scheduling to work properly, clusters must consist of at least 3 nodes and always have an odd number of members. The --pip command is only required when you want to configure DNS.

Once the script is done, you'll have a running CoreOS cluster with three machines and one Cloud Service (working as a load balancer). To get its public IP, go back to the Azure Portal and check out your load balancer Cloud Service. We'll also need the public IP of one of its boxes (which you can find under Input Endpoints)

A wild cluster appears (let's configure it)

Deis has a command line tool that helps finishing the installation of all required parts. Let's install it.

$ mkdir ~/bin       //Skip if the directory already exists
$ cd ~/bin && curl -sSL http://deis.io/deisctl/install.sh | sh -s
$ sudo ln -fs $PWD/deisctl /usr/local/bin/deisctl

With deisctl installed, go back to the deis/contrib/azure directory to inform the tool about our created cluster - by adding the domain and SSH private key. A quick word about domains and DNS: Ideally, you want a domain with a wildcard DNS record (*.domain.com), being the home to all of your apps that are going to run on that cluster. Being lazy, we'll just go with the Cloud Service's domain here though, but do check out the Deis documentation if you want to be serious with it.

$ export DEISCTL_TUNNEL=104.40.93.17:22001 // One of the Input Endpoints
$ ssh-add ./ssh-cert.key
$ deisctl config platform set domain=mycluster.cloudapp.net
$ deisctl config platform set sshPrivateKey=ssh-cert.key

We're basically done at this point, but we can now run the heavy-lifting stuff - putting Deis in control of the cluster. Give these commands a bit of time.

$ deisctl install platform
$ deisctl start platform

Once you see “Deis started.”, your Deis platform is running on your cluster and you're now ready to go! At this point, you can treat the whole cluster like your own version of DEIS. There's a ton of possebilities from here, so go and check out the full manual for tutorials on the many things you can do.