Thursday, December 20, 2012

How to use Shovel with Django

Here is my attempt to use Shovel (Rake, for Python) in Django.

Shovel can be used as an alternative to Django built-in Command. The reasons to use Shovel instead of the Command are:

1. "Rake, for Python" sounds cool.
2. Shovel is light-weight.
3. I can find all tasks in one place (the shovel/ directory).
4. I hate writing a list of make_option.
5. I hate inheriting LabelCommand, BaseCommand, NoArgsCommand, etc.

Okay, you get it... The reasons to use Shovel are actually: (i) "Rake, for Python" sounds cool, and (ii) I am lazy.

I didn't use the term "replacement for Django Command" because we cannot simply invoke Shovel task in the Django codebase using (But we can have work-around, like making a Command to proxy the call)

Let's get back to my attempt to mix the two. I placed the shovel/ folder inside the Django project.

  - shovel/ 

And I setup the Django environment at the top of the Shovel task file.
try: # Load the django environment
    import imp
    import os
    import sys

    me = os.path.abspath(os.path.dirname(__file__))
    module_info = imp.find_module('context', [me])
    imp.load_module('context', *module_info)
    print >> sys.stderr, 'Cannot setup Python environment from'
At last, the setup injects a directory to Python sys.path and use Django function to load the
def setup_django_env(path):
    from import setup_environ
        module_info = imp.find_module('settings', [path])
        settings = imp.load_module('settings', *module_info)

    except ImportError:
        msg = "Error: Can't find '' in configured PYTHON_PATH", path
        print >> sys.stderr, msg

# assume shovel/ directory is placed at the same level of
This is a common technique to integrate Django with other Python gears as well. By applying this technique, I can freely use any Django functions / models in Shovel.

NOTE 1: The name of Shovel task file cannot collide with any module/app name of the Django project.

NOTE 2: We can place the shovel/ folder outside the Django project but we have to use proper Python path and import statement (e.g. use `from themix.things.models import Thing` instead of `from things.models import Thing`).

My attempt can be found at -

Friday, January 20, 2012

git-fix-whitespace series 1: Knowing about `git diff -p`

In the last post, the first requirement of the project ( has been settled. The next requirement of this project is to read the git-diff patch in order to find any line changes that violate the whitespace rules specified in git config.

In a typical git-diff patch (see below), there are a few major parts.
  1. Line 1 provides metadata about the modified file. It gives the file path to the file, rooted at the git repository.
  2. Line 2 provides metadata about the git index and the file object discretionary access control list.
  3. Line 3 and 4 provides metadata about which file path is old and which is new.
  4. Line 5, 14, and 24 are metadata that tells which part of the file is modified. Let's take an example to explain this - "@@ -33,8 +33,7 @@ def sanitize_diff(git_diff):". "-33,8" tells that there is a diff hunk with 8 lines starts at line 33 of the old version of the file. "+33,7" tells that there is a diff hunk with 7 lines starts at line 33 of the new version of the file. As a result, the new version of the file gets one line fewer than the old version.
  5. The rest of the patch is the content in the modified file. Those lines are either prefixed by ' ', '-', or '+'. ' ' means no modification, '-' means removed line, and '+' means added line. Note, there is no line replacement since it is represented by '-' lines followed by '+' lines (see line 28-31).
After knowing the structure of a git-diff patch, the next step is to read and write the modified file.

Sunday, January 15, 2012

git-fix-whitespace series 0: GitPython vs libgit2

This is the first post of the git-fix-whitespace series. In this series, I will put some notes about working on the project - (NOTE: This series is a by-product of the git-fix-whitespace project, since my blog need some updates :P)

First of all, let me introduce the rationale of working on this git-fix-whitespace project. Most of the time, as a Python developer, I hate tab indentation and trailing whitespaces (read pep8). I know there are existing tools to "proof-read" a file using certain whitespacing rules. However, I insist to create my own tool to achieve the goal. Reason: I just wanna have a pet project that I LOVE to keep working on.

The very first requirement of this project is to support the configuration directives of git (see `man git-config` and look for `core.whitespace`). Hence, I need to find a tool that can read the git configuration files (both the user level git config file ~/.gitconfig and repository level git config file /.git/config). By using Google, I got GitPython and libgit2/pygit2. At first, I try to read the libgit2 python binding to see if I can read whitespace configuration by simple api calls. But it seems that it did not support that yet (as of 2012-Jan-15). Then I move on to GitPython. Gotcha! There's simple api call to read the core.whitespace configurations :) As a result, git-fix-whitespace got a dependency on GitPython at the moment.
© 2009 Emptiness Blogging. All Rights Reserved | Powered by Blogger
Design by psdvibe | Bloggerized By LawnyDesignz