Max Cutler

For those who read through the Courant News doc­u­men­ta­tion, you might have noticed that to cus­tomize arti­cles we cre­ated a model inher­i­tance sys­tem. How­ever, upon fur­ther reflec­tion, we feel that this is not the right approach for many rea­sons, and have devised a new system.

For any coder, espe­cially one famil­iar with Django, using the model inher­i­tance sys­tem to cre­ate cus­tom arti­cle types is no big deal. How­ever, for non-coders, writ­ing any code can be a huge bar­rier to use of the func­tion­al­ity. Besides writ­ing code, it requires run­ning ‘manage.py syncdb‘ and then restarting/refreshing the server process to pick up the new code. This goes totally against our “every­thing through the admin” phi­los­o­phy, though at the time we were will­ing to make that sac­ri­fice for the flex­i­bil­ity that the inher­i­tance sys­tem affords coders.

His­tory

Back in Jan­u­ary, Rob, Paul and I dis­cussed what we called a “meta­data” sys­tem to allow for site own­ers to cus­tomize their mod­els through the admin in a sim­i­lar man­ner to WordPress’s cus­tom fields or Drupal’s CCK. We put the project on the back­burner because we had more press­ing and fun­da­men­tal fea­tures to worry about, but it kept com­ing up in dis­cus­sions in the fol­low­ing months. Rob had been a big pro­po­nent of the meta­data sys­tem, and Paul was a fan of the inher­i­tance sys­tem; I kept bounc­ing between the two camps, until I decided in March to go the inher­i­tance route because it was some­what eas­ier and I was in a rush to launch the YDN.

Now, with more time to reflect and learn about Django/Python advanced func­tion­al­ity, I have decided to reverse course. The inher­i­tance sys­tem, while mostly work­ing, was hav­ing an issue with the get tag that I could not forsee a res­o­lu­tion of, and I real­ized that cus­tomiz­ing through the admin is just so much bet­ter on so many lev­els. The main con­cern with the meta­data sys­tem before was a con­cern about per­for­mance impact of such EAV sys­tems. Then I saw Adrian Holovaty’s pre­sen­ta­tion at PyCon 2009 about how they mostly solved the per­for­mance prob­lem for such a sys­tem at Every­Block, and that con­cern mostly went out the win­dow for me.

Dynamic Mod­els

Talk­ing with Rob today, we decided to call this fea­ture Dynamic Mod­els, but it is fun­da­men­tally the same as our orig­i­nal meta­data sys­tem vision. Most of the core Courant News mod­els (Arti­cle, Issue, Sec­tion, Event, etc.) hook into the sys­tem, and your own site-specific mod­els can use it too if appropriate.

The fea­ture allows for the def­i­n­i­tion of sets of “Dynamic Types” or con­fig­u­ra­tions of a given model, each of which has a set of addi­tional fields for that model. So you could have a Review type for Arti­cle, which has an inte­ger Rat­ing field for stor­ing the review rat­ing, or you could tweak the Stan­dard type to add some more fields to all Events.  It cur­rently sup­ports text, inte­ger, and flag/boolean value types, though it should be able to sup­port any field type that Django sup­ports (only Many­ToMany would be really challenging).

Tech­ni­cally, it will make two sim­ple queries the first time you try to access a dynamic field on a given model instance vari­able, at which time it will load all dynamic fields for that model at once. Since you will likely use all or most of the dynamic fields if you are going to use them at all, we do slightly more work up front to avoid repeat­edly hit­ting the data­base on each vari­able access. Like­wise, when sav­ing the model instance it will only do an extra data­base hit if you’ve accessed or changed one of these dynamic field val­ues. Over­all, the per­for­mance impact should be absolutely min­i­mal, and I doubt it would be not­i­ca­ble, espe­cially with nor­mal site caching tech­niques in place.

I’ve tested the sys­tem pretty exten­sively from a Django/Python shell, as only the def­i­n­i­tion of the dynamic types and fields are cur­rently fea­si­ble with the stock Django admin. The dynamic mod­els sys­tem will be built into the new Courant admin sys­tem which is in the plan­ning stages, but until then you’ll only be able to set dynamic field data on mod­els through the shell. Nobody should be using Courant in a pro­duc­tion capac­ity until the new admin any­ways, so I think that that is a fair com­pro­mise. More news on the admin soon, as I’m in the process of putting mocks together and fin­ish­ing up the spec.

Note: I haven’t com­mit­ted the code yet because my local dev setup decided to start giv­ing me grief just as I fin­ished things, and I don’t have time tonight to rein­stall every­thing. I’ll try to do that tomor­row, con­firm that it all works, and then I’ll com­mit it along with more tech­ni­cal doc­u­men­ta­tion on how it works behind the scenes.

Updated: Decided to com­mit the code any­ways for the ben­e­fit of the read­ers. Since noone should be using Courant in pro­duc­tion, this should be fine for now. I won’t be mak­ing a habit of com­mit­ting unfin­ished code, but I think it is fine for this case. I’ll be work­ing on it more tomor­row and Monday.

Update #2 (July 3, 2009): I more-or-less fin­ished this fea­ture and have com­mit­ted the code. Docs can be seen here. Dynamic fields don’t yet show in forms in admin, but that is a work item slated for Nando. Man­age­ment of dynamic types and fields is cur­rently pos­si­ble through nor­mal Django admin CRUD functionality.