This is a quick post to shout out about things I've been reading that I think are pretty interesting. Touching on writing Python like Rust, multiple interpreters with Python 3.12 and cool stuff with Django 4.2.
Real Multithreading is Coming to Python - Learn How You Can Use It Now - Martin Heinz
First up was this really interesting article from Martin
Heinz on the coming "Per-interpreter GIL" (Global Interpreter lock) that is
coming in Python 3.12. The GIL is a restriction within the Python language that
prevents true multi-threading, it enforces that only one thread can be in
control of the Python interpreter at a time. This often leads us to reach for
multiprocessing
for running code across multiple processors. This model doesn't spawn threads
but rather creates new subprocesses that side-step the GIL and allow us to run
code in parallel. This has a number of disadvantages compared to true
multi-threading, the biggest one which I've seen in the wild is processes don't
share memory like threads so by creating multiple processes you essentially copy
the data you need for each process. In a HPC context this can be a significant
problem because often you want to crunch a big dataset across multiple cores but
you have to use more memory than the actual size of your dataset because you
have to make copies of the data when using multiprocessing.
Update: On this point above about processes and memory use, this isn't strictly
true on Linux due to copy-on-write. So when we create multiple processes they
don't actually copy the memory for each process unless they actually try and
change the value. Thanks John Hodrien for pointing this out
What Martin digs into in his post is how the coming "Per-interpreter GIL" change in Python 3.12 can actually be used to do true multi-threading! He notes crucially that this change coming in Python 3.12 isn't for end users and that a more user friendly version (if the PEP is accepted) will arrive in 3.13. But this is still a really interesting development and worth poking if you're interested in high performant Python code.
Writing Python like its Rust - Jakub Beránek
This post for me managed to capture something I've felt since starting to play with Rust and having to move back to other languages like R and Python. One of the best things about Rust is the way it forces you to think differently about how you'd write other code, and to strive for "correctness" even without the Rust compiler. I know I'll be referring back to this post for a while!
Writing a chat application in Django 4.2 using async StreamingHttpResponse, Server-Sent Events and PostgreSQL LISTEN/NOTIFY - Víðir Valberg Guðmundsson
I've written some Django stuff in my time but nothing quite as clever as this. This post looks as some really cool features in Django 4.2 and how you can integrate these with the Server-side events (SSE) API to building real-time applications. This is pushing the envelope in terms of things I thought were do-able with Django so it's great to have this post that walks through these topics and does so with plenty of detail.
Memory allocation - Sam Rose
This is a fun post looking at memory and memory allocation and how simple allocators work. As someone not from a computer science background I really love posts like this that help fill out my understanding of what is actually going on with things like memory. This post also comes with some great interactive widgets and cute little dog call outs, whats not to like!