Python descriptor, Django CharField with encryption

This post is part of the pyfun series, I will try to *log* some of the features that I think they make Python funny :)

One of the most recent topics in my reading list is Python descriptor.

An object attribute with “binding behavior”, one whose attribute access has been overridden by methods in the descriptor protocol. Those methods are __get__(), __set__(), and __delete__(). If any of those methods are defined for an object, it is said to be a descriptor. - http://docs.python.org/2/howto/descriptor.html

When I finish the howto on python.org, I don't really understand what is it and thus my read-later list kept expanding with a lot of related articles; until I came across this post (If you don't know what is Python descriptor, I recommend you to read the post first since I'm not here to re-post the details with my poor English).

The purpose of this post is to extend the recommended reading to provide another example use of Python descriptor - a encryption/decryption wrapper of a Django `CharField`.

One of the major purpose to implement a Python descriptor is to provide getter and setter to attributes. In some traditional programming languages, we have to implement/generate a set of getter and setter to protect the read/write access of attributes. Or, we can use a generic attribute class that has the protection but the access of the attributes looks like `object.attribute.get()` and `object.attribute.set(xxx)`. Python descriptor solves both of the mentioned problems.

To encrypt/decrypt a `CharField`, it is obvious to override its `get()`/`set()` functions. We can simply do so by extending the `CharField` just like this snippets. However, I would like to demonstrate the use of Python descriptor (yep, I'm abusing it here).

At first, we need the descriptor with encryption and decryption. The cipher we use here is a simple 32-bytes XOR without padding (which is simply uesless in most of the cases).

class EncryptedAttr(object):
    '''Descriptor that encrypt content on write, decrypt on read'''
    def __init__(self, attr, secret_key):
        self.attr = attr
        self.key = secret_key

    def encrypt(self, v):
        '''A simple XOR chiper'''
        return ''.join(chr(ord(a) ^ ord(b)) for (a, b) in zip(self.key, v))

    def decrypt(self, v):
        '''A simple XOR chiper'''
        return ''.join(chr(ord(a) ^ ord(b)) for (a, b) in zip(self.key, v))

    def __get__(self, obj, klass):
        '''Get `attr` from owner, and decrypt it'''
        cipher_text = getattr(obj, self.attr, None)
        if not cipher_text:
            return ''

        return self.decrypt(cipher_text)

    def __set__(self, obj, value):
        '''Encrypt value, and set to owner via `attr`'''
        if not value:
            setattr(obj, self.attr, '')
            return

        cipher_text = self.encrypt(value)
        setattr(obj, self.attr, cipher_text)

The descriptor requires a Django model attribute name and a secret key in its constructor. The attribute name is used to look up the wrapped attribute of the Django model in its `__get__()` and `__set__()` functions. To use it, we just assign it as an attribute to the model class.

class Secret(models.Model):
    wrapped = models.CharField(max_length=32)
    content = EncryptedAttr('wrapped', 'This is the 32-bytes secret key.')


# Let's make a secret
payload = 'The secret must be 32-bytes long'  # Because we use a 32-bytes XOR
s = Secret()
s.content = payload

s.wrapped
>>> '32-bytes blah blah blah blah ...'

s.content
>>> 'The secret must be 32-bytes long'

In this example, the CharField `wrapped` attribute is not expected to be accessed directly. When we assign plain text to `content`, the plain text is encrypted and stored to `wrapped`. The `content` attribute does not hold anything at all. On the other hand, when we read from `content` attribute, it actually decrypts the cipher text from `wrapped`.

You may get the sample Django project to play around at https://github.com/mrkschan/encrypted-field.

How to use Shovel with Django

Here is my attempt to use Shovel (Rake, for Python) in Django.

Shovel can be used as an alternative to Django built-in Command. The reasons to use Shovel instead of the Command are:

1. "Rake, for Python" sounds cool.
2. Shovel is light-weight.
3. I can find all tasks in one place (the shovel/ directory).
4. I hate writing a list of make_option.
5. I hate inheriting LabelCommand, BaseCommand, NoArgsCommand, etc.

Okay, you get it... The reasons to use Shovel are actually: (i) "Rake, for Python" sounds cool, and (ii) I am lazy.

I didn't use the term "replacement for Django Command" because we cannot simply invoke Shovel task in the Django codebase using django.core.management.call_command(). (But we can have work-around, like making a Command to proxy the call)

Let's get back to my attempt to mix the two. I placed the shovel/ folder inside the Django project.

djangoproject/
  - manage.py
  - settings.py
  - urls.py
  - shovel/ 

And I setup the Django environment at the top of the Shovel task file.

try: # Load the django environment
    import imp
    import os
    import sys

    me = os.path.abspath(os.path.dirname(__file__))
    module_info = imp.find_module('context', [me])
    imp.load_module('context', *module_info)
except:
    print >> sys.stderr, 'Cannot setup Python environment from context.py'

At last, the setup injects a directory to Python sys.path and use Django function to load the settings.py.

def setup_django_env(path):
    from django.core.management import setup_environ
    try:
        module_info = imp.find_module('settings', [path])
        settings = imp.load_module('settings', *module_info)

        setup_environ(settings)
    except ImportError:
        msg = "Error: Can't find 'settings.py' in configured PYTHON_PATH", path
        print >> sys.stderr, msg
        sys.exit(1)

# assume shovel/ directory is placed at the same level of settings.py
setup_django_env(os.getcwd())

This is a common technique to integrate Django with other Python gears as well. By applying this technique, I can freely use any Django functions / models in Shovel.

NOTE 1: The name of Shovel task file cannot collide with any module/app name of the Django project.

NOTE 2: We can place the shovel/ folder outside the Django project but we have to use proper Python path and import statement (e.g. use `from themix.things.models import Thing` instead of `from things.models import Thing`).

My attempt can be found at - https://github.com/mrkschan/shovel-django-mix.