This was originally posted on the Local Fourth blog as part of my participation in a community media innovation project at the Medill School of Journalism.
You’re in the middle of a big project with tight deadlines. Parts of your infrastructure are a little, well, jenky. Do you take the time to make things cleaner and more coherent, or do you focus on coding, hoping that your stack holds together until you have time to clean it up? Will a new tool pay off or is it just a distraction from perhaps more tedious, but more crucial work that needs to be done.
This was my situation early this week when I started looking at our deployment procedure when moving developing on our workstations to making our work public on our webhost. I was working on a bug where things that worked on our development machines weren’t working on the webhost. Instead of just setting up a new instance for testing on our webhost, I decided to invest the time in exploring a new-to-me tool called Fabric.
Fabric is a Python library and tool for scripting commands to be run on a remote server over SSH. You can store it in the root of your Django or other Python project and run tasks on the remote server. For instance, to build up a new staging instance on our webserver, I type ‘fab staging mkinstance’ on my shell. In this example, staging and mkinstance are just tasks that you’ve defined as Python functions. staging is a task that sets context variables about this particular instance, such as the home directory where the other tasks will execute on the remote server. mkinstance just calls other tasks to create a new virtualenv, install required Python packages, download the most recent version of the source from git and more. I may go into the details of our Fabric configuration in the future, but, more importantly, I want to explain why I think exploring this tool to improve our deployment was a good way to spend my time early this week.
Avoid rm -rf . “Oh #$%!(*&!!”
I would say that even savvy system administrators who know better, who will lecture their junior admins about what not to do at great length, sometimes hop in a shell to run a quick command only to carelessly wreak disaster. I recently caught myself typing commands into shells on the wrong host. There had to be a better way. Fabric modules compel me to explicitly type the action that I intend to take and the instance I want to take it on. fab staging destroyeverything has a different resonance to ssh somehost.com && cd /home/ghing/webapps/ && rm -rf myproject.
A deployment reasoning tool
Automating your deployment process in a script helps you break down the deployment process. What are the steps that need to be taken? What order should they be run in? What are the dependencies between the steps? Writing these things out in code helps identify redundancies or potential problems in the process.
Code as documentation
An artist who taught game development to high school students told me that she introduced students to the idea of programming by having them work in pairs and take turns writing pseudocode to instruct their partner about how to move around in a space. I’d imagine one of the big takeaways is the difference between code and natural language. The latter can make for some beautiful journalism, but it can make describing a process convoluted. After I finished our Fabric module, I wondered how I would have written free form paragraphs to describe our deploy process, or explain them out loud. Even with few comments, its clear what’s happening in the deploy process. In around the same time it would take to write a step-by-step description of the process, I have one in code that can also do all the steps for me.
Further Reading
Refactoring fabfile.py for fast, robust Django deployment by Christopher Groskopf got me stoked about using Fabric, and it’s a good place to look for good practices for deploying Django in the cloud. It was, however, a little daunting as an entry point to the different tools.
Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip does a good job of describing the connection between these tools and how they can be successfully used together for deployment.
Deploying Django with Fabric helped get me started in writing my Fabric file and understanding the conventions of the tool. Some of the information on this page may be a little different with more recent versions of Fabric.
Deploying with Django’s Sites Framework on Webfaction (virtualenv, mod_wsgi, git) helped me figure out some of the non-Fabric stuff to make deployment consistent on our webhost.