1 Aug, 2024
Back in the CKAN project around 2011 time we made some very good decisions:
That top decision is still very unusual in the word of Python frameworks today.
Design a set of functions that each represent an operation a user would like to perform as part of a use case.
Each function takes JSON-compatible input and returns JSON-compatible output.
All inputs are validated, and no model or data function is allowed to be called outside one of these functions.
Once you do this, you can test just that set of functions.
Then you can use them as a JSON API, or to prepare data for HTML templates or for a command line tool, or for anything really.
There is then less need to test everything else - no bug is ever going to be too big once you have confidence in this layer.
The problem comes up time and again when talking about Django Fat Models and how to avoid a big ball of mud in Django projects. The solution is the logic layer even though it doesn't feel very Djangoy. Here's an example discussion from 2021:
https://news.ycombinator.com/item?id=23324560
In modern Python this can be simplified further because you can specify Typeddicts for the types of the JSON fields and write JSON serialisers and deserialisers by looking up the types themselves.
You still need to manually validate any extra restrictions on the data though (such as that a date is in the future), so you still need some validation functions.
One nice way of implementing the data part of the logic layer is to return JSON from a relational database.
This is nice because for the API use case you can often return the JSON directly from the API without deserialising it or pass it directory to another system.
And returning JSON directly from the database is also more efficient than what most ORMs do by returning a single set of rows for the flattened data structure with lots of duplicate entries and then and building the correct data structure in memory.
See SQLite Complex Nested JSON.
The stages of proceeding an operation in the logic layer are:
A full solution also requires:
If you use asyncio you can actually run your background tasks in the same process as the web server. Likewise the webhooks or long polling.
All this can be implemented in pure Python in just the standard library and I'm building a reference implementation. The limitation is that Python doesn't have rsa in the standard library so we have to use HMAC-based hashing for the JWTs in OIDC/OAuth.
Another way of keeping costs low is to use shared hosting or provision a t4g.nano instance as a Spot Instance and give it an ENI in an auto-scaling group so it keeps its public IPV6 address. This can cost about $1.5 per month for the instance and $0.64 for 8GB storage.
Spot instances are regularly terminated by AWS but you usually get 2 minutes warning. This is enough time to spin up another EC2 spot instance and to attach the ENI to the new instance (along with its IP address) before the old one is terminated. You can use livestream on an SQLite database to stream backups to S3 which you can restore onto the new instance.
Finally, not all people an access an IPv6 address (but most can, and most browsers will prefer IPv6 if possible) so you might need an IPv4 to IPv6 proxy too (which would cost about $5/month on top). But in theory you could use just one of these for all your $2/month applications, not just one.
There is the start of a script you could use for serving a Python app with HTTPS from Let’s Encrypt here:
Start of Debian Bookworm Lets Encrypt script for Sanic.
A great way of implementing plugins is with Protocols and egg entry points. Pylons used egg entry points.
Be the first to comment.
Copyright James Gardner 1996-2020 All Rights Reserved. Admin.