Re-imagining WSGI and Pylons

Think of environ and start_response as Services

It is easy to think of pool and render as services (even though we haven't formally defined "service" yet) but I want you to also think of environ and start_response as being servies. After all, environ is an object that provides information about the request, the server and the WSGI variables, and start_response() is a service that lets you set response information. You can look at it like this: It just so happens that environ and start_response "services" have been standardised in the WSGI spec whereas pool and render haven't.

There's one problem with this "pass services as arguments" approach though: different applications need different services and we wouldn't want an API which was different for every different possible combination of services an application could use. We'll come back to this in a minute. First let's look at another problem with most web frameworks: thread-locals.

Thread-Local Hell

HTTP connections are generally very slow at getting data to and from the server so dedicating an entire process to each request is very inefficient. Instead, the same process can handle multiple requests at the same time using threads. This is great for performance put introduces a technical challenge: if you aren't careful, different threads can change each other's data, after all they are sharing the same code. The vast majority of the time you'll never notice this problem as a web developer because the framework you are using takes care of it.

In the case of Pylons, global variables are used along with StackedObjectProxies to solve the problem. Here's some psuedo-code demonstarting the sort of approach Pylons takes:

from pylons import request, response, pool, render

def get_posts_app():
    connection = pool.connect()
    response.status = '200 OK'
    response.content_type', 'text/html'
    return render('blog_posts.html', posts=get_posts(connection))

As you can see, rather than passing the services as arguments to the application, you import StackedObjectProxies from Pylons and use them as global variables. Pylons ensures that whichever thread you access the "service" from, you get the correct data for that thread. This magic all happens behind the scenes using thread-locals to make programming a multithreaded web application as easy as programming any other sort of code. Other frameworks use variations of this idea too and other components within Pylons (such as the SQLAlchemy session use their own implementation of the same idea too).

There are some problems with the thread-local approach though:

• Thread-locals used as globals are hard to understand
• They hide what is really going (which breaks the principle of least surprise)
• They make it very hard to use code that relies on the StackedObjectProxies outside of a web request because the proxy objects aren't initialised until a request starts so any code treating them as normal object won't work.

With these thought's in mind, why not avoid the threading problem completely by passing an object containing all the services we need explictily from one part of an application to another? Let's call that object state as it represents the state of the application and all its services for the particular thread which is executing. Our code instead looks like this:

def get_posts_app(state):
    connection = state.pool.connect()
    state.response.status = '200 OK'
    state.response.content_type', 'text/html'
    return state.render('blog_posts.html', posts=get_posts(connection))

This new application just takes the one argument, state, which is an object which contains all the "services" (including the WSGI environ and WSGI start_response() callable) and can be customised depending on the services the application requires. It just means there is a bit more typing (you have to type state in front of each service) and you have to adapt existing code to explicitly take a state argument rather than relying on the right data to magically appearing in the function you are using it from. Whilst this is a bit of hassle, the vast removal of complexity is well, well worth it.

Having a single state argument to the application also solves the problem of different applications requiring different services.

So that's the two leaps of faith:

1. Treating environ and start_response() as just one of many services
2. Having an explicit state object containing all services, passed explicitly around that application as a function argument rather than a myriad of thread-local hacks

[If you've been following this for a long time, this is pretty much what I was pushing for right at the start of Pylons in this post from 2005. I just didn't describe it as a state and I didn't describe the attributes exposed as being services]

Application State vs Request State

Objects that get created when an application is first loaded into memory make up the application state. For example, database connections, template renders etc. Objects which are set up on each request make up the request state, for example Request and Response objects, an object which gives access to a session store, authentication information etc. Things which have request state are created at the start of a request and destroyed at the end of a request. Things which have application state are destroyed when the server is shut down.

Now in practice objects which have application state and those which have request state might be related. Think about our database connection pool again. Although the individual connections are created when the application is loaded and thus have application state, you might want to ensure that the same connection is used throughout a particular request so that if a problem occurs, all the changes from that request can be rolled back. This means that the object which manages a connection for a request has request state. This situation happens rather a lot.

Introducing Service Objects

With the distinction between application state and request state firmly established, let's think of what a "service" object might look like to provide an object as an attribute of state. (Hint: it looks a lot like Django middleware by coincidence):

class DatabasePoolService(object):

    def __init__(self, dsn):
        # Save the input arguments so that they can be accessed if needed elsewhere
        self.dsn = dsn
        # Set up the connection pool
        self.pool = make_connection_pool(dsn)

    def start(self, state, key):
        # The request is starting, create a connection for this request
        state['key'] = RequestSpecificConnection(self.pool.connect())

    def stop(self, state, key):
        # Commit the changes and relase the connection back to the pool
        state['key'].commit()
        state['key'].release()

    def error(self, state, key):
        # An error occurred, rollback the changes and return the connection to the pool
        state['key'].rollback()
        state['key'].release()

An instance of this class will be created when the application is first loaded and remain in memory for the entire lifetime of the application. When it is created it initialises a connection pool which it saves as self.pool. On each request the service's start() method is called to set up any request-specific objects and add them as attributes of the state object passed as an argument. If the key argument is also passed the request-specific object should be added as that key to the state.

Using a Service Object

Now, say as a user you want to use this new database code, this is all you need to do:

# Create the state (AttributeDict is any object that behaves like a
# dictionary but whose keys can be accessed as attributes)
state = AttributeDict()

# Create a service
database_service = DatabasePoolService('mysq://james@mysql.example.com:password/test')

# Start the serivce
database_service.start(state, 'pool')

# Run the application
try:
    get_posts(state)
except:
    # Handle errors
    database_service.error(state, 'pool')

# Stop the services
database_service.stop(state, 'pool')

If you have lots of services you might put them all in a dictionary and initialis them all at once:

# Create the state
state = AttributeDict()

# Create a service
services = {
    'pool': DatabasePoolService('mysq://james@mysql.example.com:password/test'),
    'template': TemplateService('/path/to/templates'),
}

# Start the serivces
for key, service in services.items():
    service.start(state, key)

# Run the application
try:
    get_posts(state)
except:
    # Handle errors
    for key, service in services.items():
        service.error(state, key)

# Stop the services
for key, service in services.items():
    service.start(state, key)
    database_service.stop(state, key)

All you need to do now is set up the environ and start_response "services" and to replace the call to get_posts(state) with some code which handles dispatch and you have the basis of an entire framework, standardised around the concept of a service:

def create_application(dsn, template_dir)
    # Create a service
    services = {
        'pool': DatabasePoolService(dsn),
        'template': TemplateService(template_dir),
    }

    def handle_request(environ, start_response):

        # Create the state
        state = AttributeDict()
        # Set up the application state
        state.app = services
        # Set up environ and start_response
        state.environ = environ
        state.start_response = start_response

        # Start the serivces
        for key, service in services.items():
            service.start(state, key)

        # Run the application
        try:
            if state.environ.get('PATH_INFO') == '/posts':
                return get_posts(state)
            else:
                return handle_404(state)
        except:
            # Handle errors
            for key, service in services.items():
                service.error(state, key)
        finally:
            # Stop the services
            for key, service in services.items():
                service.start(state, key)
                database_service.stop(state, key)

    return handle_request

By adding new service objects to this architecture you can easily recreate a framework as sophisticated as Pylons.

It is also very suprising just how many things which are currently written as WSGI middleware can actually be re-implemented much more simply as services.

As you can probably spot, handle_request is a valid WSGI application so you could easily build a WSGI middleware stack around it rather than returning it directly.

In fact, it isn't necessary to have all the services defined in one place either. In my production version of this concept services are created only for the requests that need them (making the possibility of running this entire framework as a CGI script more plausible as only those services which are required for the particular request are instantiated). In the production version the services have dependencies amongst each other, for example the error handler service requires the mail service etc. There are also tools to extract objects from the WSGI environment to turn them into services (so that you can use Beaker session store via the state object for example). You can also create services at different parts of the middleware stack so that WSGI middleware itself can use services.