This article is part of a series on Django Readonly Field. Here, we see how it works.
Exploring the project
First, you can see that the GitHub project contains quite a few files, but the really interesting part for today is the
django_readonly_field subdirectory. So that’s only 3 files and 59 lines of code according to my coverage data.
Let’s look at those files.
__init__.py, the entry point
You can see that there’s not much done here. The version is useful to have here for the lib to easily introspect its own version if need be. The
default_app_config is interesting. According to the Django documentation, this variable will be used if the module it’s in is placed in the list of
INSTALLED_APPS. So now we kind of have a idea how the app is to be used.
The value seems to be a class defined in
apps.py. Let’s go and check that.
apps.py, switching the compiler
Let’s dive in !
From the Django documentation above, we learn that the
AppConfig.ready method is called as part of
django.start() which is called at the very beginning of a django process.
So the main step here is that we replace the string in
This string is used here.
It is the main link between the
DatabaseWrapper object which represent a connection between Django and its Database, and the SQLCompiler class which is how django transforms methods calls into SQL.
The idea is that we’ll use our very own compiler that will remove the fields marked as readonly from write queries.
The thread problem
One thing that was hard to foresee is that the object we’re modifying here (actually, the whole
DatabaseWrapper) is completely recreated in every thread (it’s local to the thread), and Django will use a new thread for every connection. This means that the object we’re modifying here will be thrown away shortly, except in the tests that are mono-threaded.
Consequently, we need to make the same modification in every new version of the
DatabaseWrapper. We’ll do this by modifying the
load_backend function that is defined here and used here. The main advantage of modifying this function in particular is that it’s not thread local (there will be only one instance of this function, whatever the number of threads used) but all the threads will use this function.
The next question is how do we make
load_backend return a patched
If we follow the original code:
load_backendis called and returns an object (more precisely a module);
- in this object, a
DatabaseWrapperis called and returns a
DatabaseWrapper.__init__(), it defines the
compiler_modulewe want to change.
So we use Python duck-typing capabilities :
- Our own
custom_load_backendneeds to return an object. Instead of a module like the original function, we’ll return a class;
- The returned object needs to contain an object named
DatabaseWrapper, which is the case, but instead of a class in the original function, our object is a staticmethod;
- When that object is called, it’s supposed to return the
DatabaseWrapper. In the original function, this would simply a call to the class constructor. In our cas, it’s a call to the static method. The static method will instanciate a real
DatabaseWrapper, then patch it and then return it.
The magic of Python duck-typing is that python has no need to know that it just manipulated a class and a staticmethod instead of a module and a class like it was originally written for, because both had the same interfaces.
So now, all that’s left for us to see is what exactly our
compiler.py, the real work
The SQL compiler module is expected to contain classes named in a very specific way. The classes that we’re not modifying, we’ll just import them and don’t touch them.
For the ones we’ll modify, we’ll actually subclass them and add the a mixin that reads as follow:
readonly_field_names will explore the Model associated to the query, and if there’s a class named
ReadonlyMeta defined here, it will read its
readonly property containing the names of the fields we want to be readonly. We make a
frozenset out of it, because that’s the best structure Python provides for our use (unordered iterable with unique values that we never have to update in the course of the program). These field names are returned.
as_sql method is the entrypoint for all things SQL, so that’s where we’re going to hit. Sadly, the format of the fields is not the same for
SQLUpdateCompiler, so we’ll let the subclass add all the necessary details.
The subclasses are just doing the expected work : removing the fields from the query :
There’s not much to say here. A few tricks are used :
my_list[:] = new_listis quite the same as
my_list = new_listexcept that the former re-uses the same list, while the latter creates a new list in memory.
- The 2 forms of generators are used here. The comprehension generator and the one that uses
yield. If you don’t know about those, go and learn about it, it’s one of Python’s really nice features.
Well that’s the whole of it ! We patch the compiler to remove the readonly fields from the sql requests. We do this both in the main thread and in the subthreads created afterwards. And just with this, it works.
But this is only a part of Django Readonly Field. In order for it to be usable, we need many other parts. If you’re interested in other articles on, say, the tests, the CI, the scaffolding, the packaging etc, let me know !