My server is a bash script

Someone Else's Server

Snowflakes, Phoenixes, & Forklifts

Everything in the cloud is running on virtual machines. In turn, these run run on real hardware in someone's data center. In other words – "someone else's server".

This is true, but it also misses a crucial point: modern cloud providers let you automate intrastructure in a way that traditional datacenters never could. Server configuration as code is called infrastructure as code. In contrast, replicating existing infrastructure in the cloud is sometimes called lift and shift, or a forklift migration, because all you're really doing is lifting the server up and dropping it somewhere else. There is a misconception is that every cloud project is just a forklift migration; therefore all cloud migrations are a waste of time & money, and that the cloud itself is meaningless hype. Hence this page: railing against the term someone else's server.

Initially I set up this server as an example – a complete server defined in Bash. Everything on this server is created (and re-created) by bash scripts. A single command from my laptop will create a new fully operational server, complete with content, ready to enter production, in just a few minutes.

Full source code is in a Github repository.

Creating a web server in 3 easy steps

The following assumes creates a web server running on the DigitalOcean cloud infrastructure platform (and assumes that an account has already been created, and DigitalOcean's doctl command line tool has been set up. There are two bash scripts documented here. One runs on a local machine and interfaces with the Digital Ocean API to provision the server. The second script is run on the newly created server, in the cloud. Both scripts should be easily portable to other Unix environments, probably even Windows 10's Linux subsystem.

1. Provisioning a machine with code

Before the server starts, I generate an ssh root key on my local machine, using the standard ssh-keygen tool. The public half of the key is uploaded to DigitalOcean, and this key is added to the droplet as it's provisioned. (Root login via password is automatically disabled).

  export HOST=tharsis
  export KEY=~/.ssh/root_tharsis_digital_ocean

  rm -f $KEY
  ssh-keygen -q -N '' -f $KEY
  export KEYID=$(doctl compute ssh-key import $HOST \
    --public-key-file $KEY.pub \
    --format ID \
    --no-header)

Now we tell DigitalOcean to provision the new droplet using the doctl compute droplet create command. In this case we're creating the smallest sized droplet (a single CPU with 1GB RAM and 25GB of SSD storage). This costs about 0.7c an hour, or $USD 5 a month to run. (Cloud servers are like lightblubs, they're cheap to run but it's still important to remember to shut-down any cloud services you aren't using, unless you want an unexpected bill).

  export IPV4=$(doctl compute droplet create $HOST \
    --size s-1vcpu-1gb \
    --image ubuntu-17-10-x64 \
    --region sgp1 \
    --ssh-keys $KEYID \
    --user-data-file sundog-userdata.sh \
    --format PublicIPv4 \
    --no-header \
    --wait)
  doctl compute ssh-key delete $KEYID --force
  echo $IPV4 > ip

2. Configuring the server with user data

Most of the server configuration is done via the second script, which is placed into user data, (via the --user-data-file option). When the droplet first starts up (and only on first startup), this user data script is executed.

The first action on the machine is to set up a bare git repository for the web page content. This lets us edit & test web content locally, before pushing it to the server.

    mkdir /srv/git
    mkdir /srv/git/tharsis.git
    git init --bare /srv/git/tharsis.git
    mkdir /srv/www
    git clone /srv/git/tharsis.git /srv/www

Secondly, we use the server's package manager to download & and install a web server (Nginx), and create a bare-bones config file for it. Ubuntu has a rather complex config file setup, where config files for sites are created in the sites-available folder, and symlinked into sites-enabled.

  apt-get -y update
  apt-get -y install nginx

  echo "server {
    listen  80;
    server_name tharsis.io;
    root  /srv/www;
    index index.html;
    location / {
      try_files \$uri \$uri/ =404;
    }
  }" > /etc/nginx/sites-available/tharsis.io

  ln -s /etc/nginx/sites-available/tharsis.io /etc/nginx/sites-enabled/tharsis.io
  rm /etc/nginx/sites-enabled/default
  service nginx restart

The machine server can take a while to boot up, install the web server, etc. I create a dummy web page that is available once the user-data script has run to completion. The next part of the process needs to wait until this is complete before pushing pages to the server.

  git clone /srv/git/tharsis.git /srv/www
  # temp index.html, needed for test 200 response.
  export HOSTNAME=$(curl -s http://169.254.169.254/metadata/v1/hostname)
  export PUBLIC_IPV4=$(curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address)
  echo "<body>hostname: $HOSTNAME<br>IP: $PUBLIC_IPV4</body>" > /srv/www/index.html

3. Pushing content to the server

The next part of the script running on the local machine has to wait for the server to boot-up before pushing content (including this page). To do this we use a small loop that attempts to read the index page from the web server, waiting until it receives a response.

  echo -n "Machine created at $IPV4.  Waiting for response...";
  until $(curl --output /dev/null --silent --head --fail http://$IPV4); do
    printf '.'
    sleep 5
  done

After the server starts up we point a local git repository at the server (i.e. adding a new remote), and push the content.

  export PROJECT=~/Documents/Projects/Pavonis
  export GIT_SSH_COMMAND="ssh -i $KEY -q -o StrictHostKeyChecking=no"
  cd $PROJECT
  git remote set-url origin root@$IPV4:/srv/git/tharsis.git
  git push origin master
  ssh -i $KEY root@$IPV4 "rm /srv/www/*; cd /srv/www; git pull"

A small footnote: This code bypasses the check that the ECDSA signature of the host machine is valid. It is possible that if the connection was compromised before the host key was added to known_hosts then all subsequent communications could be intercepted. Fixing this will have to wait for later.

DigitalOcean, and bash

DigitalOcean is one of the smaller cloud providers, but they're popular among developers for being simple, inexpensive, fast and having great documentation and support. Many product offerings and core concepts are very similar across all cloud providers. EC2 for example is very similar to DigitalOcean droplets, S3 storage works in much the same ways as DigitalOcean spaces, and AWS elastic IPs work just like DigitalOcean flexible IPs.

Bash would not generally be the go-to tool for large scale (or even small scale) cloud provisioning. Bash is ok for a demo project like this, but there are many (better) tools for managing virtual machines and cloud infrastructure, for instance: Vagrant, Terraform, Puppet, Chef and Ansible, each with their own adherants, evangelists, zealots and apologists. But I'm using Bash because Bash is simple, Bash is everywhere (even on some Windows machines), and because I wanted some experience using command-line manipulation of cloud APIs. And just because I could.

For a more detailed guide to the infrastructure as code approach, I recommend Amazon's white paper Architecting for the Cloud.

Doing anything in code always takes longer than planned, especially when you take debugging into account. I'd estimate it took me several hours to get the bash script working exactly the way I wanted, and numerous droplets were created and destroyed in the process.

On the other hand, this would have gone a lot faster if I'd documented what I'd done when I manually set up my previous web server! I'd often find myself using Google to refresh my memory. Where are the Nginx config files again? How do I reset the remote origin of a git repository?

As much as it is code, this script is also a how-to and a cheat-sheet for setting up a server in the cloud. Like any programming task, next time it'll go much faster.

Version control is the new backup

The trouble with manual configuration is that it's not easily repeatable. If the hardware on which a server is running fails (unlikely), or the server gets hacked (more likely), or you screw up (much more likely), then setting up the server again requires just as much manual work. These manually set-up and maintained servers are called Snowflake Servers (as in a special snowflake). On the other hand, automating a server as (a script or a template) allows this code to be re-run. This creates what has been termed a Phoenix Server (because you can burn it down, and have it rise from the ashes).

Snowflake servers are bad. Manual configuration is bad. Forklift migrations, and treating clouds like bare metal is doing it wrong!

With a server configuration defined entirely in code, you can put it under (distributed) version control. Version control offers many advantages: having code under distributed version control obviates the need for backups of specific servers; the code exists in multiple repositories, shared by developers and in shared repositories. As long as the code exists, copies of the server can be recreated in minutes. Servers become disposable, fungible resources. Machines no longer have to be restored from backup; they can be rebuilt (often more quickly) from the latest production branch.

New branches allow developers to quickly create exact copies of production machines for testing, rather than trying to coordinate the synchronisation of production, test, and development (which, let's face it, never works.)

New configurations (if they pass tests), can then become a production branch, and a new production server can be deployed. In the event of any problems, the configuration can be restored to a previous state and an old server deployed in minutes.