Showing posts with label python. Show all posts
Showing posts with label python. Show all posts

Sunday, February 23, 2014

Python descriptor, Django CharField with encryption

This post is part of the pyfun series, I will try to *log* some of the features that I think they make Python funny :)

One of the most recent topics in my reading list is Python descriptor.
An object attribute with “binding behavior”, one whose attribute access has been overridden by methods in the descriptor protocol. Those methods are __get__(), __set__(), and __delete__(). If any of those methods are defined for an object, it is said to be a descriptor. - http://docs.python.org/2/howto/descriptor.html
When I finish the howto on python.org, I don't really understand what is it and thus my read-later list kept expanding with a lot of related articles; until I came across this post (If you don't know what is Python descriptor, I recommend you to read the post first since I'm not here to re-post the details with my poor English).

The purpose of this post is to extend the recommended reading to provide another example use of Python descriptor - a encryption/decryption wrapper of a Django `CharField`.

One of the major purpose to implement a Python descriptor is to provide getter and setter to attributes. In some traditional programming languages, we have to implement/generate a set of getter and setter to protect the read/write access of attributes. Or, we can use a generic attribute class that has the protection but the access of the attributes looks like `object.attribute.get()` and `object.attribute.set(xxx)`. Python descriptor solves both of the mentioned problems.

To encrypt/decrypt a `CharField`, it is obvious to override its `get()`/`set()` functions. We can simply do so by extending the `CharField` just like this snippets. However, I would like to demonstrate the use of Python descriptor (yep, I'm abusing it here).

At first, we need the descriptor with encryption and decryption. The cipher we use here is a simple 32-bytes XOR without padding (which is simply uesless in most of the cases).

class EncryptedAttr(object):
    '''Descriptor that encrypt content on write, decrypt on read'''
    def __init__(self, attr, secret_key):
        self.attr = attr
        self.key = secret_key

    def encrypt(self, v):
        '''A simple XOR chiper'''
        return ''.join(chr(ord(a) ^ ord(b)) for (a, b) in zip(self.key, v))

    def decrypt(self, v):
        '''A simple XOR chiper'''
        return ''.join(chr(ord(a) ^ ord(b)) for (a, b) in zip(self.key, v))

    def __get__(self, obj, klass):
        '''Get `attr` from owner, and decrypt it'''
        cipher_text = getattr(obj, self.attr, None)
        if not cipher_text:
            return ''

        return self.decrypt(cipher_text)

    def __set__(self, obj, value):
        '''Encrypt value, and set to owner via `attr`'''
        if not value:
            setattr(obj, self.attr, '')
            return

        cipher_text = self.encrypt(value)
        setattr(obj, self.attr, cipher_text)

The descriptor requires a Django model attribute name and a secret key in its constructor. The attribute name is used to look up the wrapped attribute of the Django model in its `__get__()` and `__set__()` functions. To use it, we just assign it as an attribute to the model class.

class Secret(models.Model):
    wrapped = models.CharField(max_length=32)
    content = EncryptedAttr('wrapped', 'This is the 32-bytes secret key.')


# Let's make a secret
payload = 'The secret must be 32-bytes long'  # Because we use a 32-bytes XOR
s = Secret()
s.content = payload

s.wrapped
>>> '32-bytes blah blah blah blah ...'

s.content
>>> 'The secret must be 32-bytes long'

In this example, the CharField `wrapped` attribute is not expected to be accessed directly. When we assign plain text to `content`, the plain text is encrypted and stored to `wrapped`. The `content` attribute does not hold anything at all. On the other hand, when we read from `content` attribute, it actually decrypts the cipher text from `wrapped`.

You may get the sample Django project to play around at https://github.com/mrkschan/encrypted-field.

Thursday, October 17, 2013

Partial function call

This post is part of the pyfun series, I will try to *log* some of the features that I think they make Python funny :)

Again, I was reading the Scala tutorial and find that she has built-in support of partial function call (see http://twitter.github.io/scala_school/basics.html#functions). This reminded me that Python does also has functools.partial(), which can be used as function shortcuts.

Let's see this example of Django.

# Let's have a Coupon that can either be fixed amount discount or percentage off
# But, we don't want to have model inheritance and table join to get the data
class Coupon(models.Model):
    code = models.CharField(max_length=8)
    type = models.CharField(max_length=11, choices=['fixedamount', 'percentage'])
    amount = models.DecimalField(max_digits=8, decimal_places=2)
    currency = models.CharField(max_length=3, choices=['USD', 'CAD'], default='')


# To create a Coupon based on certain conditions, you can have this
kwargs = {'code': code}

if condition_a:
    kwargs.update({'type': 'fixedamount', 'currency': 'USD'})
elif condition_b:
    kwargs.update({'type': 'percentage'})

kwargs.update({'amount': x if condition_c else y})
coupon = Coupon.objects.create(**kwargs)


# Or with functools.partial()
FixedamountCoupon = functools.partial(Coupon.objects.create, type='fixedamount')
PercentageCoupon = functools.partial(Coupon.objects.create, type='percentage')

if condition_a:
    coupon = functools.partial(FixedamountCoupon, code=code, currency='USD')
elif condition_b:
    coupon = functools.partial(PercentageCoupon, code=code)

coupon = functools.partial(coupon, amount=x if condition_c else y)
coupon = coupon()

Yes, we just shortcuted two types of Coupon using functools.partial(), created "sub-class" of Coupon. Furthermore, if the underlying function accepts positional arguments, we can also shortcut those arguments.

Clean code FTW.

Friday, October 11, 2013

Scala Traits in Python?

Sometimes, I would question myself, am I qualify as a seasoned Python engineer? The answer is I'm still a junior. The next question is, how to qualify as a seasoned one?

I have two things in mind that may be part of the answer to the second question:

1. Know when to use certain Python features.
2. Know what kind of library or framework is suitable for particular task.

In this pyfun series, I will try to *log* some of the features that I think they make Python funny :)

Let's get back to the primary subject of this post. Scala, which compiles to JAVA bytecode and runs on JVM, is one of the latest big hit. So, I'm going through some tutorials of it to check out her magics. One of the topic is Traits.

Traits, is similar to Java interface. When I read some samples about it, I wonder how can it be done in Python. From stackoverflow, http://stackoverflow.com/q/6240118, there is a suggestion that to use Python class to simulate the thing. And, starting from that idea, I wonder can we do that in runtime instead of just static definition. Here, we come to Python type().

Most of the time, type() can be used as Enum in C (see http://stackoverflow.com/a/1695250), or dynamic objects like those in Javascript (https://gist.github.com/mrkschan/6936112). It is because type() is essentially a dynamic form of the class statement (http://docs.python.org/2/library/functions.html#type). In other words, we can use type() to create Python class that simulates Traits (JAVA interface) at runtime.

Here is an example.

# See original Scala version at http://twitter.github.io/scala_school/basics.html#trait
Car = type('Car', (object,), {'brand': ''})
Shiny = type('Shiny', (object,), {'refraction': 0})

BMW = type('BMW', (Car, Shiny,), {'brand': 'BMW', 'refraction': 100})
my_bmw = BMW()

print my_bmw.brand, my_bmw.refraction

# We can have constructor as well
def bmw_init(self, refraction):
    self.refraction = refraction

BMW = type('BMW', (Car, Shiny,), {'brand': 'BMW', '__init__': bmw_init})
c1, c2 = BMW(10), BMW(100)
print c1.refraction, c2.refraction

As we can see, we can use type() to create a set of "interfaces". And, use type() to create class that implement the "interface". These "interfaces" can have its name changed according to runtime conditions. type() is Python magic and it's fun :)

However, I did not come up with a use case for runtime defined Traits yet :P

Thursday, December 20, 2012

How to use Shovel with Django

Here is my attempt to use Shovel (Rake, for Python) in Django.

Shovel can be used as an alternative to Django built-in Command. The reasons to use Shovel instead of the Command are:

1. "Rake, for Python" sounds cool.
2. Shovel is light-weight.
3. I can find all tasks in one place (the shovel/ directory).
4. I hate writing a list of make_option.
5. I hate inheriting LabelCommand, BaseCommand, NoArgsCommand, etc.

Okay, you get it... The reasons to use Shovel are actually: (i) "Rake, for Python" sounds cool, and (ii) I am lazy.

I didn't use the term "replacement for Django Command" because we cannot simply invoke Shovel task in the Django codebase using django.core.management.call_command(). (But we can have work-around, like making a Command to proxy the call)

Let's get back to my attempt to mix the two. I placed the shovel/ folder inside the Django project.

djangoproject/
  - manage.py
  - settings.py
  - urls.py
  - shovel/ 

And I setup the Django environment at the top of the Shovel task file.
try: # Load the django environment
    import imp
    import os
    import sys

    me = os.path.abspath(os.path.dirname(__file__))
    module_info = imp.find_module('context', [me])
    imp.load_module('context', *module_info)
except:
    print >> sys.stderr, 'Cannot setup Python environment from context.py'
At last, the setup injects a directory to Python sys.path and use Django function to load the settings.py.
def setup_django_env(path):
    from django.core.management import setup_environ
    try:
        module_info = imp.find_module('settings', [path])
        settings = imp.load_module('settings', *module_info)

        setup_environ(settings)
    except ImportError:
        msg = "Error: Can't find 'settings.py' in configured PYTHON_PATH", path
        print >> sys.stderr, msg
        sys.exit(1)

# assume shovel/ directory is placed at the same level of settings.py
setup_django_env(os.getcwd())
This is a common technique to integrate Django with other Python gears as well. By applying this technique, I can freely use any Django functions / models in Shovel.

NOTE 1: The name of Shovel task file cannot collide with any module/app name of the Django project.

NOTE 2: We can place the shovel/ folder outside the Django project but we have to use proper Python path and import statement (e.g. use `from themix.things.models import Thing` instead of `from things.models import Thing`).

My attempt can be found at - https://github.com/mrkschan/shovel-django-mix.

Friday, January 20, 2012

git-fix-whitespace series 1: Knowing about `git diff -p`

In the last post, the first requirement of the project (https://github.com/mrkschan/git-fix-whitespace) has been settled. The next requirement of this project is to read the git-diff patch in order to find any line changes that violate the whitespace rules specified in git config.

In a typical git-diff patch (see below), there are a few major parts.
  1. Line 1 provides metadata about the modified file. It gives the file path to the file, rooted at the git repository.
  2. Line 2 provides metadata about the git index and the file object discretionary access control list.
  3. Line 3 and 4 provides metadata about which file path is old and which is new.
  4. Line 5, 14, and 24 are metadata that tells which part of the file is modified. Let's take an example to explain this - "@@ -33,8 +33,7 @@ def sanitize_diff(git_diff):". "-33,8" tells that there is a diff hunk with 8 lines starts at line 33 of the old version of the file. "+33,7" tells that there is a diff hunk with 7 lines starts at line 33 of the new version of the file. As a result, the new version of the file gets one line fewer than the old version.
  5. The rest of the patch is the content in the modified file. Those lines are either prefixed by ' ', '-', or '+'. ' ' means no modification, '-' means removed line, and '+' means added line. Note, there is no line replacement since it is represented by '-' lines followed by '+' lines (see line 28-31).
After knowing the structure of a git-diff patch, the next step is to read and write the modified file.

Sunday, January 15, 2012

git-fix-whitespace series 0: GitPython vs libgit2

This is the first post of the git-fix-whitespace series. In this series, I will put some notes about working on the project - https://github.com/mrkschan/git-fix-whitespace. (NOTE: This series is a by-product of the git-fix-whitespace project, since my blog need some updates :P)

First of all, let me introduce the rationale of working on this git-fix-whitespace project. Most of the time, as a Python developer, I hate tab indentation and trailing whitespaces (read pep8). I know there are existing tools to "proof-read" a file using certain whitespacing rules. However, I insist to create my own tool to achieve the goal. Reason: I just wanna have a pet project that I LOVE to keep working on.

The very first requirement of this project is to support the configuration directives of git (see `man git-config` and look for `core.whitespace`). Hence, I need to find a tool that can read the git configuration files (both the user level git config file ~/.gitconfig and repository level git config file /.git/config). By using Google, I got GitPython and libgit2/pygit2. At first, I try to read the libgit2 python binding to see if I can read whitespace configuration by simple api calls. But it seems that it did not support that yet (as of 2012-Jan-15). Then I move on to GitPython. Gotcha! There's simple api call to read the core.whitespace configurations :) As a result, git-fix-whitespace got a dependency on GitPython at the moment.
 
© 2009 Emptiness Blogging. All Rights Reserved | Powered by Blogger
Design by psdvibe | Bloggerized By LawnyDesignz