Python: cleaning the premises (), or do the inhabitants have a lighter cousin?

advertisements

I'm working in Django. In Django, when you're rendering a template you send it a context dictionary to get replaced. Because I'm lazy/DRY, i often use locals() as a shortcut instead of sending a dictionary that looks like {'my_var': my_var, 'var2': var2}.

This usually works beautifully, and saves a lot of mind-numbing repetition.

I'm using django-notifications to send emails when certain events happen - say you receive a private message. Django-notifications comes with a built-in queuing feature that I am integrating now.

However, the problem is that django-notifications pickles the context dictionary at queue time. This is where the locals() trick fails -- the dictionary from locals has a LOT of crap in it beyond the local variables (e.g. it has import and int()). The dictionary created by locals doesn't pickle.

I see three options: 1) rewrite the queueing method of django-notifications to render the template before it is stored (straightforward but a bit tedious and breaks upgradability) 2) stop using the locals trick and start repeating myself 3) Try to find a lighter-weight version of locals (or a way to pickle locals).

I'm here hoping someone has leads in the direction of #3.

In case it might be relevant, here is the error I get when I try to pickle using the locals() shortcut:

TypeError: can't pickle ellipsis objects

Further, the dictionary output of locals():

{
    '__builtins__':
    {
        'bytearray': <type 'bytearray'>,
         'IndexError': <type 'exceptions.IndexError'>,
         'all': <built-in function all>,
         'help': Type help() for interactive help,
         or help(object) for help about object.,
         'vars': <built-in function vars>,
         'SyntaxError': <type 'exceptions.SyntaxError'>,
         'unicode': <type 'unicode'>,
         'UnicodeDecodeError': <type 'exceptions.UnicodeDecodeError'>,
         'isinstance': <built-in function isinstance>,
         'copyright': Copyright (c) 2001-2010 Python Software Foundation.
All Rights Reserved.

Copyright (c) 2000 BeOpen.com.
All Rights Reserved.

Copyright (c) 1995-2001 Corporation for National Research Initiatives.
All Rights Reserved.

Copyright (c) 1991-1995 Stichting Mathematisch Centrum,
         Amsterdam.
All Rights Reserved.,
         'NameError': <type 'exceptions.NameError'>,
         'BytesWarning': <type 'exceptions.BytesWarning'>,
         'dict': <type 'dict'>,
         'input': <built-in function input>,
         'oct': <built-in function oct>,
         'bin': <built-in function bin>,
         'SystemExit': <type 'exceptions.SystemExit'>,
         'StandardError': <type 'exceptions.StandardError'>,
         'format': <built-in function format>,
         'repr': <built-in function repr>,
         'sorted': <built-in function sorted>,
         'False': False,
         'RuntimeWarning': <type 'exceptions.RuntimeWarning'>,
         'list': <type 'list'>,
         'iter': <built-in function iter>,
         'reload': <built-in function reload>,
         'Warning': <type 'exceptions.Warning'>,
         '__package__': None,
         'round': <built-in function round>,
         'dir': <built-in function dir>,
         'cmp': <built-in function cmp>,
         'set': <type 'set'>,
         'bytes': <type 'str'>,
         'reduce': <built-in function reduce>,
         'intern': <built-in function intern>,
         'issubclass': <built-in function issubclass>,
         'Ellipsis': Ellipsis,
         'EOFError': <type 'exceptions.EOFError'>,
         'locals': <built-in function locals>,
         'BufferError': <type 'exceptions.BufferError'>,
         'slice': <type 'slice'>,
         'FloatingPointError': <type 'exceptions.FloatingPointError'>,
         'sum': <built-in function sum>,
         'getattr': <built-in function getattr>,
         'abs': <built-in function abs>,
         'exit': Use exit() or Ctrl-D (i.e. EOF) to exit,
         'print': <built-in function print>,
         'True': True,
         'FutureWarning': <type 'exceptions.FutureWarning'>,
         'ImportWarning': <type 'exceptions.ImportWarning'>,
         'None': None,
         'hash': <built-in function hash>,
         'ReferenceError': <type 'exceptions.ReferenceError'>,
         'len': <built-in function len>,
         'credits':     Thanks to CWI,
         CNRI,
         BeOpen.com,
         Zope Corporation and a cast of thousands
    for supporting Python development.  See www.python.org for more information.,
         'frozenset': <type 'frozenset'>,
         '__name__': '__builtin__',
         'ord': <built-in function ord>,
         'super': <type 'super'>,
         '_': None,
         'TypeError': <type 'exceptions.TypeError'>,
         'license': Type license() to see the full license text,
         'KeyboardInterrupt': <type 'exceptions.KeyboardInterrupt'>,
         'UserWarning': <type 'exceptions.UserWarning'>,
         'filter': <built-in function filter>,
         'range': <built-in function range>,
         'staticmethod': <type 'staticmethod'>,
         'SystemError': <type 'exceptions.SystemError'>,
         'BaseException': <type 'exceptions.BaseException'>,
         'pow': <built-in function pow>,
         'RuntimeError': <type 'exceptions.RuntimeError'>,
         'float': <type 'float'>,
         'MemoryError': <type 'exceptions.MemoryError'>,
         'StopIteration': <type 'exceptions.StopIteration'>,
         'globals': <built-in function globals>,
         'divmod': <built-in function divmod>,
         'enumerate': <type 'enumerate'>,
         'apply': <built-in function apply>,
         'LookupError': <type 'exceptions.LookupError'>,
         'open': <built-in function open>,
         'quit': Use quit() or Ctrl-D (i.e. EOF) to exit,
         'basestring': <type 'basestring'>,
         'UnicodeError': <type 'exceptions.UnicodeError'>,
         'zip': <built-in function zip>,
         'hex': <built-in function hex>,
         'long': <type 'long'>,
         'next': <built-in function next>,
         'ImportError': <type 'exceptions.ImportError'>,
         'chr': <built-in function chr>,
         'xrange': <type 'xrange'>,
         'type': <type 'type'>,
         '__doc__': "Built-in functions,
         exceptions,
         and other objects.\n\nNoteworthy: None is the `nil' object; Ellipsis represents `...' in slices.",
         'Exception': <type 'exceptions.Exception'>,
         'tuple': <type 'tuple'>,
         'UnicodeTranslateError': <type 'exceptions.UnicodeTranslateError'>,
         'reversed': <type 'reversed'>,
         'UnicodeEncodeError': <type 'exceptions.UnicodeEncodeError'>,
         'IOError': <type 'exceptions.IOError'>,
         'hasattr': <built-in function hasattr>,
         'delattr': <built-in function delattr>,
         'setattr': <built-in function setattr>,
         'raw_input': <built-in function raw_input>,
         'SyntaxWarning': <type 'exceptions.SyntaxWarning'>,
         'compile': <built-in function compile>,
         'ArithmeticError': <type 'exceptions.ArithmeticError'>,
         'str': <type 'str'>,
         'property': <type 'property'>,
         'GeneratorExit': <type 'exceptions.GeneratorExit'>,
         'int': <type 'int'>,
         '__import__': <built-in function __import__>,
         'KeyError': <type 'exceptions.KeyError'>,
         'coerce': <built-in function coerce>,
         'PendingDeprecationWarning': <type 'exceptions.PendingDeprecationWarning'>,
         'file': <type 'file'>,
         'EnvironmentError': <type 'exceptions.EnvironmentError'>,
         'unichr': <built-in function unichr>,
         'id': <built-in function id>,
         'OSError': <type 'exceptions.OSError'>,
         'DeprecationWarning': <type 'exceptions.DeprecationWarning'>,
         'min': <built-in function min>,
         'UnicodeWarning': <type 'exceptions.UnicodeWarning'>,
         'execfile': <built-in function execfile>,
         'any': <built-in function any>,
         'complex': <type 'complex'>,
         'bool': <type 'bool'>,
         'ValueError': <type 'exceptions.ValueError'>,
         'NotImplemented': NotImplemented,
         'map': <built-in function map>,
         'buffer': <type 'buffer'>,
         'max': <built-in function max>,
         'object': <type 'object'>,
         'TabError': <type 'exceptions.TabError'>,
         'callable': <built-in function callable>,
         'ZeroDivisionError': <type 'exceptions.ZeroDivisionError'>,
         'eval': <built-in function eval>,
         '__debug__': True,
         'IndentationError': <type 'exceptions.IndentationError'>,
         'AssertionError': <type 'exceptions.AssertionError'>,
         'classmethod': <type 'classmethod'>,
         'UnboundLocalError': <type 'exceptions.UnboundLocalError'>,
         'NotImplementedError': <type 'exceptions.NotImplementedError'>,
         'AttributeError': <type 'exceptions.AttributeError'>,
         'OverflowError': <type 'exceptions.OverflowError'>
    },
     'notification': <module 'notification.models' from '/home/b/webapps/myapp/notification/models.pyc'>,
     'u': <User: abcd>,
     'User': <class 'django.contrib.auth.models.User'>
}


Please allow me to propose an alternative to using locals()

class Object(object):
    pass

def foo():
    result = Object()
    result.my_var = 'bar'
    your_var = 'not to be returned'
    result.var2 = 'baz' + result.my_var + len(your_var)
    return result.__dict__

Another option would be to just add objects to the context as you create them.

def foo():
    result = {}
    result['my_var'] = my_var = 'bar'
    your_var = 'not to be returned'
    result['var2'] = var2 = 'baz' + my_var + len(your_var)
    return result

Although, make sure that the multiple assignment occurs every time you modify the variable (or at least the very last time).

edit: You should be using that second version. I'm sure the additional keystrokes of using the context dict (result or ctx or whatever) doesn't feel very DRY, but let me tell you that this is actually a good thing. The Zen of Python states "Explicit is better than implicit." When you return the value of locals(), you are returning who knows what. Everything that was in the global namespace, everything that's in a nested namespace, every intermediate calculation you may have made, all of those things are implicitly returned to the caller.

If you create a container for the value you wish to return, you are making explicit your intent, drawing a sharp line between

this_is_for_me = 'foo'

and

result['this_is_for_you'] = 'bar'

The difference is clear in the code what is public, for use by others; and private, subject to change, implementation specific. It's also clear to the caller, because those private values are simply unavailable.

And if you are integrating an external library that never-the-less uses locals() magic, you should be very hesitant about where and how you use it. I would probably prefer to patch the code that does this over any other option available.