Streamline your Django workflow

September 29, 2011 Tags: django, django-community

Ruby gems make your python more awesome.

This post is about a couple of improvements I made to my Django development environments recently. It started with a simple issue. Whenever I want to work on something, I need to:

  • Run the Django development server,
  • Run compass watch on my CSS files,
  • Run the command that automatically runs my tests when a file changes in the project.

That's 3 terminal windows to open (I don't use tabs :) and twice as much commands to type. Tedious, to say the least. Luckily a friend / colleague of mine, Bertrand, pointed me to a tool to manage such repetitive tasks: Foreman. To quote their documentation:

Foreman is a tool to manage Procfile-based applications.

Which means (for us), it reads a file that describes different processes and runs them in parallel, merging the output of every process.

First, let's install the gem.

Making gems play nice with virtualenv

If you like virtualenv, you'll probably don't want to install ruby gems system-wide. To tell gem to install stuff in your virtualenv, just add:

export GEM_HOME="$VIRTUAL_ENV/gems"
export GEM_PATH=""
export PATH=$PATH:$GEM_HOME/bin

to your ~/.virtualenvs/postactivate hook. Now you can gem install stuff and it'll be isolated from your system.

As a side note, gem compiles the documentation when it installs something. If you don't need it and want to save a couple of seconds, add gem: --no-rdoc --no-ri to your ~/.gemrc.

Requirements.txt, the Ruby way

Next, we'll specify our dependencies for the project. For python we have pip requirements, for Ruby there is Bundler. Create a file named Gemfile with the following content:

source :rubygems
gem 'foreman'

Then run:

gem install bundler
bundle install

I use Compass which is a Ruby gem too, so my Gemfile looks like:

source :rubygems
gem 'foreman'
gem 'compass-less-plugin'
gem 'rb-inotify'

Running bundle install installs everything and generates a Gemfile.lock file describing your installation, with pinned version numbers (what you usually do — or should do — in requirements.txt). This ensures you get the same packages if you recreate your environment from scratch, so make sure you keep it under version control too.

Foreman and Django

Now we have everything in place, let's create our configuration file for foreman. Create a Procfile with the following content:

web: django-admin.py runserver --settings=project.settings

Now run foreman start and here is what you should see:

14:12:31 web.1     | started with pid 19591
14:12:32 web.1     | Validating models...
14:12:32 web.1     |
14:12:33 web.1     | 0 errors found
14:12:33 web.1     | Django version 1.3.1, using settings 'project.settings'
14:12:33 web.1     | Development server is running at http://127.0.0.1:8000/
14:12:33 web.1     | Quit the server with CONTROL-C.

foreman successfully launches runserver and prefixes the output with the name we specified in the Procfile. But let's add more stuff. Like I wrote above I run my tests whenever a file changes in my project, and I use gorun for this:

Gorun uses inotify so it's linux-only but there are other tools for OSX and windows. To use gorun, just add to your settings.py:

DIRECTORIES = (
    ('', 'django-admin.py test app1 app2 --settings=project.settings'),
)

In a shell, what you need to do is cd to your project and run gorun settings.py. Let's add this to our Procfile:

web: django-admin.py runserver --settings=project.settings
tests: cd project && gorun settings.py

An now, when we run foreman start again, we see the combined output of both runserver and gorun:

14:24:50 web.1     | started with pid 21066
14:24:50 tests.1   | started with pid 21069
14:24:51 tests.1   | DIRECTORY /path/to/project
14:24:51 tests.1   | Waiting for stuff to happen...
14:24:51 web.1     | Validating models...
14:24:51 web.1     |
14:24:51 web.1     | 0 errors found
14:24:51 web.1     | Django version 1.3.1, using settings 'project.settings'
14:24:51 web.1     | Development server is running at http://127.0.0.1:8000/
14:24:51 web.1     | Quit the server with CONTROL-C.

And finally, compass. I've started using it since watching Idan Gazit's Djangocon.eu talk and like Idan, I don't want to do plain CSS anymore :)

So I have a bunch of SASS files in my project and I generate the CSS file on the fly using compass watch. The command looks like this:

compass watch --force --no-line-comments --output-style compressed \
    --require less --sass-dir <project>/<app>/static/<app>/css \
    --css-dir <project>/<app>/static/<app>/css \
    --image-dir /static/ <project>/<app>/static/<app>/css/screen.scss

It's a pretty awful line but it does what I want: take the screen.scss file provided by my project's "core" app and generate the CSS. I put the compass-compiled CSS under source control to avoid the compass dependency during deployments.

To make your Procfile more generic, you can define environment variables in a .env file and use them in the Procfile:

PROJ=project
APP=app

And the final Procfile looks like:

compass: compass watch <the compass line above>
web: django-admin.py runserver --settings=$PROJ.settings
tests: cd $PROJ && gorun settings.py

Now you can directly jump to your project, run foreman start and start coding!

14:37:18 compass.1  | started with pid 22145
14:37:18 web.1      | started with pid 22148
14:37:18 tests.1    | started with pid 22151
14:37:19 tests.1    | DIRECTORY /path/to/project
14:37:19 tests.1    | Waiting for stuff to happen...
14:37:19 compass.1  | >>> Compass is watching for changes. Press Ctrl-C to Stop.
14:37:19 web.1      | Validating models...
14:37:19 web.1      |
14:37:19 web.1      | 0 errors found
14:37:19 web.1      | Django version 1.3.1, using settings 'project.settings'
14:37:19 web.1      | Development server is running at http://127.0.0.1:8000/
14:37:19 web.1      | Quit the server with CONTROL-C.

As you can see it's really easy to setup and saves a lot of time and typos as you add more moving parts to your projects.

Comments

September 29, 2011Adrian

I've struggled with the same issue...compass, django, coffeescript, etc...Foreman is exactly what I've been looking for. Thanks!

October 5, 2011Markus Gattol

Another thing that's quite handy in this regard (i.e. additional stuff for Profile) is https://github.com/jessemiller/HamlPy

I am a new happy foreman user btw... :-)

October 6, 2011Liam

Thanks for the post, I've really found it useful. I've been creating Python scripts to run CoffeeScript, Compass, and a simple HTTP server, but Foreman has already made this much easier. Awesome!

November 28, 2011Philipp Bosch

I'm also using foreman to run Django's dev server, Compass, Coffeescript, etc. One problem I ran into quickly is that if I use the pdb debugger in my code, its output will be included in foreman's output but I'm unable to interact with the debugger. Did you by any chance find a solution for this?

November 28, 2011Bruno

@Philipp: Foreman seems to be read-only unfortunately. For debugging I'd suggest something like the Werkzeug debugger or just falling back to plain shell.

Add a comment

Comments are closed for this entry.