Courant News: Model Customization

For those who read through the Courant News documentation, you might have noticed that to customize articles we created a model inheritance system. However, upon further reflection, we feel that this is not the right approach for many reasons, and have devised a new system.

For any coder, especially one familiar with Django, using the model inheritance system to create custom article types is no big deal. However, for non-coders, writing any code can be a huge barrier to use of the functionality. Besides writing code, it requires running `manage.py syncdb` and then restarting/refreshing the server process to pick up the new code. This goes totally against our “everything through the admin” philosophy, though at the time we were willing to make that sacrifice for the flexibility that the inheritance system affords coders.

History

Back in January, Rob, Paul and I discussed what we called a “metadata” system to allow for site owners to customize their models through the admin in a similar manner to WordPress’s custom fields or Drupal’s CCK. We put the project on the backburner because we had more pressing and fundamental features to worry about, but it kept coming up in discussions in the following months. Rob had been a big proponent of the metadata system, and Paul was a fan of the inheritance system; I kept bouncing between the two camps, until I decided in March to go the inheritance route because it was somewhat easier and I was in a rush to launch the YDN.

Now, with more time to reflect and learn about Django/Python advanced functionality, I have decided to reverse course. The inheritance system, while mostly working, was having an issue with the get tag that I could not forsee a resolution of, and I realized that customizing through the admin is just so much better on so many levels. The main concern with the metadata system before was a concern about performance impact of such EAV systems. Then I saw Adrian Holovaty’s presentation at PyCon 2009 about how they mostly solved the performance problem for such a system at EveryBlock, and that concern mostly went out the window for me.

Dynamic Models

Talking with Rob today, we decided to call this feature Dynamic Models, but it is fundamentally the same as our original metadata system vision. Most of the core Courant News models (Article, Issue, Section, Event, etc.) hook into the system, and your own site-specific models can use it too if appropriate.

The feature allows for the definition of sets of “Dynamic Types” or configurations of a given model, each of which has a set of additional fields for that model. So you could have a Review type for Article, which has an integer Rating field for storing the review rating, or you could tweak the Standard type to add some more fields to all Events.  It currently supports text, integer, and flag/boolean value types, though it should be able to support any field type that Django supports (only ManyToMany would be really challenging).

Technically, it will make two simple queries the first time you try to access a dynamic field on a given model instance variable, at which time it will load all dynamic fields for that model at once. Since you will likely use all or most of the dynamic fields if you are going to use them at all, we do slightly more work up front to avoid repeatedly hitting the database on each variable access. Likewise, when saving the model instance it will only do an extra database hit if you’ve accessed or changed one of these dynamic field values. Overall, the performance impact should be absolutely minimal, and I doubt it would be noticable, especially with normal site caching techniques in place.

I’ve tested the system pretty extensively from a Django/Python shell, as only the definition of the dynamic types and fields are currently feasible with the stock Django admin. The dynamic models system will be built into the new Courant admin system which is in the planning stages, but until then you’ll only be able to set dynamic field data on models through the shell. Nobody should be using Courant in a production capacity until the new admin anyways, so I think that that is a fair compromise. More news on the admin soon, as I’m in the process of putting mocks together and finishing up the spec.

Note: I haven’t committed the code yet because my local dev setup decided to start giving me grief just as I finished things, and I don’t have time tonight to reinstall everything. I’ll try to do that tomorrow, confirm that it all works, and then I’ll commit it along with more technical documentation on how it works behind the scenes.

Updated: Decided to commit the code anyways for the benefit of the readers. Since noone should be using Courant in production, this should be fine for now. I won’t be making a habit of committing unfinished code, but I think it is fine for this case. I’ll be working on it more tomorrow and Monday.

Update #2 (July 3, 2009): I more-or-less finished this feature and have committed the code. Docs can be seen here. Dynamic fields don’t yet show in forms in admin, but that is a work item slated for Nando. Management of dynamic types and fields is currently possible through normal Django admin CRUD functionality.

One Response to Courant News: Model Customization

Leave a Reply