WordPress as a Web Citizen: The Need for (Better) APIs

Prior to the 3.0 release of WordPress, I often dismissed WordPress as “just a tool for blogging.” Much to the credit of the core developers and community, WordPress has become much better suited for projects that are more complex than reverse-chronological blogging.

But when I try to use WordPress as a CMS, one of my biggest remaining stumbling blocks is the difficulty of integrating with other systems.

Friends like Daniel and Andrew have been experimenting with using WordPress as a repository for all of their digital activity. Leveraging custom post types, plugins, post formats, and more, they have created rich means of capturing different types of activity and ideas. As a data aggregation and repository tool, I think that’s wonderful and want to pursue similar efforts for my own site eventually.

However, such efforts are mostly one-way: WordPress plugin code uses the APIs of other services to transfer data to or from WordPress. If WordPress is the only system in your personal or corporate data ecosystem, that may be fine. But for many many companies, there are legacy and complementary systems that need to interact with WordPress.

My background is in newspapers, where many newsrooms are struggling with how, and even if, to tie together legacy behemoth CMSs with ad-hoc Django/Rails projects and WordPress blog networks. There’s also third party special-purpose content services like CoverItLive and Storify, for which the primary means of “integration” seems to be dropping HTML/JS snippets in the CMS or WordPress posts.

What is really needed is a robust way for these heterogeneous networks of systems to communicate with each other. Most of the aforementioned services offer read+write APIs for just that purpose. But WordPress is the odd man out, and I do not understand why that has been allowed to continue for years.

The recommended “WordPress way” to handle this is to use the XML-RPC API. That would indeed be a good response if the API supported more than the basic blogging capabilities. But if you are using WordPress as a CMS and leveraging all the wonderful features added in and since WordPress 3.0, then you are S.O.L.

The WordPress XML-RPC API is essentially the descendant of the classic MetaWeblog, MovableType, and Blogger blog APIs. But while WordPress core has plowed ahead with new features every point release, the XML-RPC API is essentially frozen in time at 3.0, with minor bugfixes over the years.

One of the biggest consumers of the API today are the mobile apps, to which I started contributing this year. One of the biggest mobile app feature requests is support for custom post types, which were added to core in 3.0 but are still not supported by XML-RPC almost three releases later. Coming close behind is the need for better media management and support for custom taxonomies.

I’ve built python-wordpress-xmlrpc over the past year to facilitate integrations with Django (and other Python-based) projects. I know of a number of individuals and companies who have fallen back on custom WordPress plugins that add new XML-RPC methods to support their needs. That’s great for them, but results in the fragmentation of the ecosystem, and limits opportunities for shared knowledge and libraries. Everyone would be better off if we could all use the same API instead of each creating our own variants. And since you can’t run plugins on WordPress.com, that’s not even an option for commercial integrations wanting to support that market.

This summer, WordPress sponsored a Google Summer of Code project to rectify this situation, and we now have a series of patches to expose most of the core functionality through XML-RPC. These patches still require a fair amount of polish and testing before being commit-ready, but they are a good starting point.

Having all the raw features accessible via the XML-RPC API would be great, but it’s also only a start. There remain some critical usability flaws that limit the usefulness of the API as a means of cross-system integration.

There’s still no way to page through the posts archive, forcing very expensive database operations to pull the full archive for large sites, often blowing past memory limits. The media management methods don’t return IDs, making it difficult to change media after they are uploaded. There’s no way to differentiate published and modified dates on posts. None of the methods support ‘since <date>’ parameters to allow efficient discovery of changes for diff-based syncing. The use of actions and filters in the method implementations are incredibly inconsistent, if they exist at all, making it very difficult for plugins to modify the behavior without wholesale replacing the built-in methods. And the list goes on…

As you may notice, many of these issues already have tickets filed in the core trac, and most of them even have patches. Yet they sit there mostly untouched. I’ve offered to help on some of them, and am willing to do much more work on XML-RPC patches, but I won’t waste my time if they aren’t going to be acted upon.

Those of us building cross-system integrations and working on the official mobile apps need a better API. There’s really no downside, especially when there are people like me who are willing to put in serious work on patches and testing. I’ve watched the core devs agonize over internal (“core”) WordPress API design decisions, but they seem not to care nearly as much about the public-facing XML-RPC API.

So I’ll be petitioning for XML-RPC to be one of the focuses/priorities for the WordPress 3.4 release cycle. Hopefully someone will agree and we won’t see another release go by without improving WordPress’ interoperability.

One Response to WordPress as a Web Citizen: The Need for (Better) APIs
  1. Andrew Nacin

    But while Word­Press core has plowed ahead with new fea­tures every point release, the XML-RPC API is essen­tially frozen in time at 3.0, with minor bug­fixes over the years.

    Sorry, that made me laugh. The XML-RPC API is frozen in time from much earlier than that.

    Seriously, though — we would absolutely love it if someone breathed life with the XML-RPC API. Most patches out there aren’t yet good enough for core, particularly those prototyped as part of the GSoC project. Make them so and I will make sure we can run with them. Until now, I don’t think I’ve seen the ones you referenced in your post, either.

    Until it gets to the point where it’s not a joke, I doubt you’ll ever see us consider it when making core design designs. But if core developers are approached at the right parts of the cycle, I’m sure a number would be willing to shepherd a number of XML-RPC improvements as early as 3.4. Let’s talk further about it.

    I will say, though, that register_meta() in 3.3 was designed with XML-RPC access in mind, and specifically enabling plugins (and core, certainly) to allow access to some meta keys while protecting others from user modification.

Leave a Reply