Using yield Statements in WSGI Middleware can be Very Harmful

Fixing the Problem

To still be able to iterate over results without needing to buffer output but avoid the execution order problems created by using a generator directly you can put the code containing the yield statement inside another function and return the result of calling that function as shown below:

>>> class Middleware3(object):
...     def __init__(self, app):
...         self.app = app
...     def __call__(self, environ, start_response):
...         try:
...             print 1
...             def output():
...                 for data in self.app(environ, start_response):
...                     yield data
...             return output()
...         finally:
...             print 2
...
>>> for data in Middleware3(returning_app)(dummy_environ, dummy_start_response):
...     print data
...
1
2
3
4
5

As you can see this new code results in execution happening the correct order.

Using Generators in WSGI Applications

Although using yield in WSGI middleware can be dangerous if not done properly, using it in WSGI applications is a perfectly good idea and can be used to stream data back to the browser. Here's another application which yields a new number every second.

>>> import time
>>>
>>> def yielding_app(environ, start_response):
...     start_response('200 OK', [])
...     yield 3
...     time.sleep(1)
...     yield 4
...     time.sleep(1)
...     yield 5
...

If you run the applicaiton you'll see there is always a pause between the numbers 3 and 4 being printed and the numbers 4 and 5 being printed but that the order the different parts of the middleware are executed in remains the same as it did when using the returning_app. Middleware2 still produces an unexpected order as it did before and Middleware1 and Middleware2 still work correctly.

>>> for data in Middleware1(yielding_app)(dummy_environ, dummy_start_response):
...     print data
...
1
2
3
4
5
>>> for data in Middleware2(yielding_app)(dummy_environ, dummy_start_response):
...     print data
...
1
3
4
5
2
>>> for data in Middleware3(yielding_app)(dummy_environ, dummy_start_response):
...     print data
...
1
2
3
4
5
>>>

More Complex Middleware Chains

If you have written middleware similar to Middleware2 before and not noticed a problem it may be because the other middleware which wraps it did not have any try... finally blocks or that it behaved in such a way that the order of execution didn't cause a problem. Let's define another middleware classcalled OuterMiddleware to demonstrate some of the side-effects of middleware using yield statements.

>>> class OuterMiddleware(object):
...     def __init__(self, app):
...         self.app = app
...     def __call__(self, environ, start_response):
...         try:
...             print 1
...             return self.app(environ, start_response)
...         finally:
...             print 3
...
>>>

Let's also redefine the existing middleware without try... finally blocks so that you can see it is the fact that they are generators that is causing a problem, not the fact that they include a try... finally block. Obviously without a try... finally block there is no way to print a value after the middleware returns so these examples will only print 5 numbers. I've also updated the other middleware so that the correctly written ones still output consequitive numbers:

>>> def yielding_app(environ, start_response):
...     start_response('200 OK', [])
...     time.sleep(1)
...     yield 4
...     time.sleep(1)
...     yield 5
...     time.sleep(1)
...
>>> class Middleware1(object):
...     def __init__(self, app):
...         self.app = app
...     def __call__(self, environ, start_response):
...         print 2
...         return self.app(environ, start_response)
...
>>> class Middleware2(object):
...     def __init__(self, app):
...         self.app = app
...     def __call__(self, environ, start_response):
...         print 2
...         app_iter = self.app(environ, start_response)
...         for data in app_iter:
...             yield data
...         if hasattr(app_iter, 'close'):
...             app_iter.close()
...
>>> class Middleware3(object):
...     def __init__(self, app):
...         self.app = app
...     def __call__(self, environ, start_response):
...         print 2
...         def output():
...             for data in self.app(environ, start_response):
...                 yield data
...         return output()
...
>>>

Now let's see how the new middleware chains behave:

>>> for data in OuterMiddleware(Middleware1(yielding_app))(dummy_environ, dummy_start_response):
...     print data
...
1
2
3
4
5
>>> for data in OuterMiddleware(Middleware2(yielding_app))(dummy_environ, dummy_start_response):
...     print data
...
1
3
2
4
5
>>> for data in OuterMiddleware(Middleware3(yielding_app))(dummy_environ, dummy_start_response):
...     print data
...
1
2
3
4
5
>>>

As you can see Middleware1 and Middleware3 still behave correctly but look what is happening in the example containing Middleware2. The finally block of OuterMiddleware is executed before either the try block of Middleware2 or the application itself.

I hope this post has demonstrated that you have to be extremly careful when using yield statements in WSGI middleware, even when the middleware you produce appears to function correctly under certain circumstances. If you have weird execution order problems in your WSGI stack it is possible that one of the middleware components is using yield somewhere it shouldn't. If you are in any doubt it is best to avoid yield statements altogether.