The Python future statement
from __future__ import feature provides a nice way to ease the transition to new language features. Is it is possible to implement a similar feature for Python libraries:
from myproject.__future__ import feature?
It's straightforward to set a module wide constants on an import statement. What isn't obvious to me is how you could ensure these constants don't propagate to code executed in imported modules -- they should also require a future import to enable the new feature.
This came up recently in a discussion of possible indexing changes in NumPy. I don't expect it will actually be used in NumPy, but I can see it being useful for other projects.
As a concrete example, suppose that we do want to change how indexing works in some future version of NumPy. This would be a backwards incompatible change, so we decide we to use a future statement to ease the transition. A script using this new feature looks something like this:
import numpy as np from numpy.__future__ import orthogonal_indexing x = np.random.randn(5, 5) print(x[[0, 1], [0, 1]]) # should use the "orthogonal indexing" feature # prints a 2x2 array of random numbers # we also want to use a legacy project that uses indexing, but # hasn't been updated to the use the "orthogonal indexing" feature from legacy_project import do_something do_something(x) # should *not* use "orthogonal indexing"
If this isn't possible, what's the closest we can get for enabling local options? For example, is to possible to write something like:
from numpy import future future.enable_orthogonal_indexing()
Using something like a context manager would be fine, but the problem is that we don't want to propagate options to nested scopes:
with numpy.future.enable_orthogonal_indexing(): print(x[[0, 1], [0, 1]]) # should use the "orthogonal indexing" feature do_something(x) # should *not* use "orthogonal indexing" inside do_something
The way Python itself does this is pretty simple:
Note that the only things allowed before a future statement are strings, comments, blank lines, and other future statements, which means it doesn't need to fully parse the code to do this. That's important, because future statements can change the way the code is parsed (in fact, that's the whole point of having them…); strings, comments, and blank lines can be handled by the lexer step, and future statements can be parsed with a very simple special-purpose parser.
Then, if any future statements are found, Python sets a corresponding flag bit, then re-seeks to the top of the file and calls
compile with those flags. For example, for
from __future__ import unicode_literals, it does
flags |= __future__.unicode_literals.compiler_flag, which changes
In this "real compile" step, the future statements are treated as normal imports, and you will end up with a
__future__._Feature value in the variable
unicode_literals in the module's globals.
Now, you can't quite do the same thing, because you're not going to reimplement or wrap the compiler. But what you can do is use your future-like statements to signal an AST transform step. Something like this:
flags =  for line in f: flag = parse_future(line) if flag is None: break flags.append(flag) f.seek(0) contents = f.read() tree = ast.parse(contents, f.name) for flag in flags: tree = transformers[flag](tree) code = compile(tree, f.name)
Of course you have to write that
parse_future function to return 0 for a blank line, comment, or string, a flag for a recognized future import (which you can look up dynamically if you want), or
None for anything else. And you have to write the AST transformers for each flag. But they can be pretty simple—e.g., you can transform
Subscript nodes into different
Subscript nodes, or even into
Call nodes that call different functions based on the form of the index.
To hook this into the import system, see PEP 302. Note that this gets simpler in Python 3.3, and simpler again in Python 3.4, so if you can require one of those versions, instead read the import system docs for your minimum version.
For a great example of import hooks and AST transformers being used in real life, see MacroPy. (Note that it's using the old 2.3-style import hook mechanism; again, your own code can be simpler if you can use 3.3+ or 3.4+. And of course your code isn't generating the transforms dynamically, which is the most complicated part of MacroPy…)