The Proliferation of Registries

As I’ve discussed before, one of the core design tenets of Courant News was the ability for news orgs to customize and add on to our core platform without having to modify the code of the platform itself. While it is possible to create a cohesive platform, it is more difficult to allow outside code to hook into it without actually modifying the platform itself.

One common way, adopted by the Django built-in admin app, as well as  a number of common Django reusable apps like django-tagging and django-mptt, is that of a registry system. I’ve been joking with one of my Courant cohorts, Robert Baskin (@rsbaskin), on twitter about registries, and I thought it was time to let everyone else in on the discussion.

An Example

One way to manage functionality across various content types is to define the set of content types in your settings file. A recent example is django-shorturls, which lets you generate short URLs for your content by specifiying the content types to expose with a given prefix/abbreviation.

# in settings.py
SHORTEN_MODELS = {
    'A': 'myapp.animal',
    'V': 'myapp.vegetable',
    'M': 'myapp.mineral'
}

This works wonderfully when you have full control of the codebase, but if we were to include django-shorturls in Courant itself, it would require news orgs to modify our code* to add in their own custom content types or tweak how our standard configuration works.

You can solve this problem by creating a registry system. Each model can then register itself with the URL shortening app, passing along the prefix parameter of its choice. For example:

# in myapp/models.py
from django.db import models
from courant.core.shorturls import shorturls

class Planet(models.Model):
    name = models.CharField(max_length=100)
    ...
shorturls.register(Planet, 'P')

In the courant.core.shorturls app, it keeps track of all the models that register with it, and then can use this registry in its internal code. It also means that if you want to change the prefix for, say, the built-in Article model from ‘A’ to ‘S’ (for story), you could do this:

# in myapp/models.py or any other place that will get automatically run by Django, such as an __init__.py
from courant.core.news.models import Article
from courant.core.shorturls import shorturls

shorturls.unregister(Article)
shorturls.register(Article, 'S')

In this manner you can tweak the default configuration, while also hooking your own new content types into our functionality.

The Proliferation Problem

The short URL app example above is a nice, clear example of the types of situations where registries make sense. But as I mentioned in the beginning of this post, there are a number of other apps that use registries, both within Courant and in the general Django community. There’s the Django admin, django-tagging, django-mptt (hierarchical relationships), and django-comment-utils on the outside. We also have the get tag which I described previously and a registry for the search system (upcoming post once a few bugs are fixed). Our Article model currently looks like this (much simplified):

# courant/core/news/models.py
from django.db import models
from courant.core.discussions.moderation import moderator, CourantModerator
from courant.core.gettag import gettag
from courant.core.search import search

class Article(models.Model):
    heading = ...
    ...
moderator.register(Article, CourantModerator)
gettag.register(Article, name_field='heading')
search.register(Article,
                fields=('heading', 'subheading', 'summary', 'body'),
                filter_fields=('section', 'display_type','status'),
                date_field='published_at',
                use_delta=True)

We haven’t actually written the short URLs app I described above, but likely will, in which case you can tack on yet another registration call there. As you can see, this starts to rapidly build up for the more commonly used models, although that’s admittedly a rather small percentage of all of our models.

For those who follow Rob or I on twitter, you may have noticed us joking about meta-registries, which would be a registry to help manage all of these individual registries. How that would actually work is up for debate, and is really nothing more than an inside joke (though not so inside anymore).

Alternatives?

In light of this potential problem of runaway registry creation, we’ve been considering some other options. For some of the more complicated registries, like for search, we’d most likely be better served by going to a declarative syntax like models themselves. django-haystack has taken this approach, and I actually much prefer it in many respects, and just haven’t yet gotten around to building something similar to work with django-sphinx (our search tool of choice; explanation of that decision for that future post on search). It still requires a small registry of its own, but the registration call is handled in a separate file from the model itself, similar to how the admin system works.

Another option could be to tack additional options onto the models’ internal Meta classes and customize the python metaprogramming that Django does to build python objects from your model definitions. A registration process would still be occuring behind the scenes, but you wouldn’t be required to interact with it directly. For the above Article example, it might look like this (again, the surrounding parts of the model are much simplified):

# courant/core/news/models.py
from django.db import models
from courant.core.discussions.moderation import CourantModerator

class Article(models.Model):
    heading = ...
    ...

    class Meta:
        # standard Django meta options
        ordering = '-published_at'

        # custom Courant meta options
        short_url_prefix = 'A'
        get_tag = {'name_field': 'heading'}
        moderator = CourantModerator
    ...

This is maybe slightly cleaner because you don’t need all the imports at the top of the file, but I’m not sure it gains you much in the end. I personally rather like the explicitness of importing everything and manually registering it. Note that the get_tag meta option in this example uses a python dictionary because there are actually a number of other optional parameters that you can pass to it, and you wouldn’t want to have a meta option for every single possible parameter. My biggest hesitation with going this route is that it probably involves mucking with metaprogramming and doing some behind-the-scenes magic, which I don’t really think is a favorable cost/benefit tradeoff.

Conclusion

I hope that explains some of my twitter ramblings over the past couple weeks, and gives some additional insight into how Courant will enable customization and extension of the platform without forking the code. Please post comments if you have any questions or thoughts or opinions.

* Technically, this example resides in the settings file, which the news org has full control over anyways and thus would have no problem modifying. But I think it makes more sense to define the prefixes with the models they are related to, and not hidden away in a setting.

Leave a Reply