As part of the release, I spent the time to write proper documentation; be sure to check out the examples section! The library is now also compatible with Python 3 (in addition to Python 2.6+).

If you are currently using a previous release (1.x) of the library, be warned that 2.0 is not backwards compatible. I’ve removed all but one use of legacy Blogger/metaWeblog methods, which has caused the posts module and several others to change significantly. See the changelog for full details.

And, of course, you can download it on PyPI or directly through GitHub.

New Methods

The most significant change was the addition of a suite of new methods covering post and taxonomy functionality.

These replace the legacy legacy metaWeblog API methods for creating and editing posts, which did not support custom post types or taxonomies. You should no longer need to use any non-wp.* methods, please use the WordPress API in all of your apps and integrations!

Bugfixes

Fortunately, most of the items from my bugs hitlist have been fixed this cycle.

One of the most requested was #15098, support for adding a featured image/thumbnail to posts. This has been added to both the new post methods and the legacy metaWeblog post methods. Now the mobile apps can finally support post thumbnails, a long-requested feature by mobile users.

Check out the full list of XML-RPC tickets from this cycle.

Unit Tests

The WordPress core team takes backwards compatibility very seriously, and that’s even more important for a public/external API like XML-RPC. The best way to protect against breaking API changes is to have a robust test suite for it. One of the commit bars for this cycle was that core patches needed corresponding unit tests.

The core unit test suite now includes 134 XML-RPC tests, covering 631 assertions. While this is a large step forward, it’s hopefully just the beginning.

Documentation

Having written XML-RPC clients in Python, C#, and Java for talking to WordPress, I’ve banged my head against the vagaries of the WordPress XML-RPC API many many times. A large part of this is because there has been a dearth of documentation on the specifics of method parameters and return values.

Although WordPress ostensibly implements “standard” APIs (Blogger, metaWeblog, and MovableType), the specs for those APIs have largely disappeared from the internet, and WordPress has expanded or modified them to cover WordPress-specific features. This makes life difficult for developers who don’t want to dive into WordPress core code.

Over the past several months, I’ve been documenting the entirety of the XML-RPC API, including all of the legacy methods. For now you can find it on my Codex userpage, but I hope to work with the Codex team to get it onto the “real” Codex before the final 3.4 release.

Conclusion

Between support for custom post types, custom taxonomies, post thumbnails, and all of the other bugfixes, the XML-RPC API has made significant strides forward this release. This has opened the door for the official mobile apps to become more capable, as well as enable new scenarios for other external services and API consumers.

I’d like to thank Peter Westwood (aka westi) for being our fearless leader and carving time out of his busy schedule to mentor Marko Heijnen and I this cycle. Thanks also to Joseph Scott for helping to drive the bugfix work at the beginning of the cycle. And thanks to Andrew Nacin and Ryan Boren for their support and beating the security drum.

If anyone finds bugs, please report them! Once 3.4 is out the door, we’ll talk about scope for 3.5, and I have several ideas on where XML-RPC can head in future releases. Especially if core is serious about adopting a REST(ful) API like the one being built for WordPress.com. But more on that another day!

The new design is based on a theme called PixelPower by Curt Ziegler. My previous design was also one of his themes, and I’m certainly a fan of his minimalistic style.

PixelPower employs responsive web design techniques, so the site should now scale in size from my large desktop monitor down to smartphones, without the need for fragile plugins and browser sniffing.

The new theme also supports WordPress post formats, which will enable me to start micro-blogging. My goal has been to make this site the repository of all my personal activity on the web, and this is one more step along that path. Over the coming months I intend to archive my tweets and bookmarks here and start treating this site as my canonical data warehouse.

Alongside the new theme, I am using the excellent WP Resume plug-in by Ben Balter. It uses custom post types to capture the resume information in a structured format, and comes with an intelligent default template and markup structure (hResume+HTML5). Certainly beats maintaining a text resume by hand!

So expect the volume on this blog to increase moderately, as I start posting content here that I might have otherwise posted on Twitter or various mailing lists. Here’s to an exciting 23rd year!

This update mostly refines and previous release by making parameter names and return values more consistent across the API.

I also reorganized the code to separate the plugin initialization hooks from the new server class implementation. This is mostly for organizational purposes, but also makes it so that the bulk of the code isn’t parsed and loaded by WordPress unless the request is actually to xmlrpc.php.

The plugin is still missing wp.newPost and wp.editPost, as I have not yet come up with a design for them that I am satisfied with. These methods are currently the planned focus for our core dev team’s (led by westi, with me and MarkoHeijnen as collaborators) first two-week cycle for 3.4. If/when they are ready, I’ll include them in a 0.7 release for the plugin.

I’d love to get feedback and more people testing the new functionality, so if you are so-inclined please do let me know and share your thoughts.

What is it?

This plugin is primarily a refinement of the work done by Prasath during the WordPress Web Services GSoC 2011 project, as discussed in my last post. This involved a lot of reformatting the code to be self-consistent and closer to the WordPress core code style, as well as simplifying or reorganizing some of the logic.

In an effort to reduce code duplication, I’ve taken a cue from the bbPress XML-RPC implementation and extracted common return-value preparation code into prepare_<datatype> methods. Where sensible, I’ve also removed some input validation that already occurs in the core APIs that are subsequently called.

I had to remove three methods from the original wp-webservices plugin: wp.newPost, wp.editPost, and wp.deletePost. These methods are pretty crucial, but I was unsatisfied with the approach taken in the GSoC project and want to spend more time devising a more sensible implementation before including them in the plugin.

Highlight: wp.getPosts

One of the biggest pain points for the WordPress mobile apps and other XML-RPC integrations is the inability to retrieve anything beyond the latest 10 posts. The metaWeblog.getRecentPosts method lets you specify the number of posts to return, but does not support paging or any sort of filtering, which means that many shared hosts fall over (or time out) if you try to retrieve all the posts at once.

The new wp.getPosts method has two parameters that allow for much greater control over the return values: filter and fields.

The “filter” (associative) array allows you to more precisely control what posts will be returned. It can accept the following keys:

• post_type: the type of post to return (post, page, or any custom post type)
• post_status: only return posts with specific status(es)
• numberposts: the number of posts to return
• offset: combined with numberposts, allows paged return of results
• orderby and order: if you’d prefer an ordering other than reverse-chronological, you can use these parameters to specify the ordering (see wp_get_recent_posts docs for valid values)

The “fields” array allows you to specify the extra set of fields to return in the response for each post. In many scenarios, the calling application does not require all of the fields, so asking for a subset saves on bandwidth and potentially avoids some additional queries on the server.

“fields” can contain individual field names, but can also contain “conceptual group” names, which are sets of associated fields. The available conceptual groups for posts in v0.5 are “post” (most of the basic fields), “taxonomies” (akin to calling wp.getPostTerms), “custom_fields”, and “enclosure”.

Other Features

See the readme.txt file for a full list of methods, including methods for user management, custom taxonomies, and basic information on custom post types. I’ll cover many of these methods in future blog posts.

Next Steps

This release is just the first step in a process of trying to get these methods into core. Tomorrow I’ll be updating patches on all of the relevant tickets and soliciting feedback on certain design and implementation questions. If you have an interest in this, please watch the core Trac (see my last post for specific tickets).

A new branch of python-wordpress-xmlrpc is also forthcoming that includes support for the new methods.

As of today (12/22/2011), there are 72 open tickets in the WordPress core Trac XML-RPC component. While I’d love to see them all fixed, there is limited amount of time and so we must prioritize. This post represents my current priorities and plans.

GSoC Project

The first area of focus will be refining the work from the WordPress Web Services 2011 Google Summer of Code project, which resulted in the wp-webservices plugin. These new methods expand the wp.* method namespace and deprecate the usage of metaWeblog/MT/Blogger API methods for common tasks.

I’ll first release a new plugin based on wp-webservices but with updated implementations. The goal is to allow people to test the new methods on 3.3 or earlier, and to make it easier than applying two dozen patches against trunk. I’ll also have a new branch of python-wordpress-xmlrpc that can talk to the new methods.

(Note: priorities are my opinion, not necessarily that of WordPress core team)

Bugfixes

The remainder of the tickets are a mix of feature requests and bugs. Many already have patches, though they may need to be updated to apply against trunk. An example of some easy fixes that would make the lives of the mobile app developers (and other XML-RPC users) so much better:

Of the 51 open non-GSoC XML-RPC tickets, 34 currently have patches.

Next Steps

My first task is to get a release of the improved wp-webservices plugin out the door. I started work on it this summer when the GSoC project was winding down, and I’ll spend the next few days doing some more refining. My goal is to have code available by Tuesday, and updated patches on all the GSoC tickets by Wednesday.

Starting this weekend I’ll also start idling in #wordpress-dev again, where I’ll be pestering nacin, westi and josephscott about triaging the old tickets and getting some of the low-hanging fruit committed.

But when I try to use WordPress as a CMS, one of my biggest remaining stumbling blocks is the difficulty of integrating with other systems.

Friends like Daniel and Andrew have been experimenting with using WordPress as a repository for all of their digital activity. Leveraging custom post types, plugins, post formats, and more, they have created rich means of capturing different types of activity and ideas. As a data aggregation and repository tool, I think that’s wonderful and want to pursue similar efforts for my own site eventually.

However, such efforts are mostly one-way: WordPress plugin code uses the APIs of other services to transfer data to or from WordPress. If WordPress is the only system in your personal or corporate data ecosystem, that may be fine. But for many many companies, there are legacy and complementary systems that need to interact with WordPress.

My background is in newspapers, where many newsrooms are struggling with how, and even if, to tie together legacy behemoth CMSs with ad-hoc Django/Rails projects and WordPress blog networks. There’s also third party special-purpose content services like CoverItLive and Storify, for which the primary means of “integration” seems to be dropping HTML/JS snippets in the CMS or WordPress posts.

What is really needed is a robust way for these heterogeneous networks of systems to communicate with each other. Most of the aforementioned services offer read+write APIs for just that purpose. But WordPress is the odd man out, and I do not understand why that has been allowed to continue for years.

The recommended “WordPress way” to handle this is to use the XML-RPC API. That would indeed be a good response if the API supported more than the basic blogging capabilities. But if you are using WordPress as a CMS and leveraging all the wonderful features added in and since WordPress 3.0, then you are S.O.L.

The WordPress XML-RPC API is essentially the descendant of the classic MetaWeblog, MovableType, and Blogger blog APIs. But while WordPress core has plowed ahead with new features every point release, the XML-RPC API is essentially frozen in time at 3.0, with minor bugfixes over the years.

One of the biggest consumers of the API today are the mobile apps, to which I started contributing this year. One of the biggest mobile app feature requests is support for custom post types, which were added to core in 3.0 but are still not supported by XML-RPC almost three releases later. Coming close behind is the need for better media management and support for custom taxonomies.

I’ve built python-wordpress-xmlrpc over the past year to facilitate integrations with Django (and other Python-based) projects. I know of a number of individuals and companies who have fallen back on custom WordPress plugins that add new XML-RPC methods to support their needs. That’s great for them, but results in the fragmentation of the ecosystem, and limits opportunities for shared knowledge and libraries. Everyone would be better off if we could all use the same API instead of each creating our own variants. And since you can’t run plugins on WordPress.com, that’s not even an option for commercial integrations wanting to support that market.

This summer, WordPress sponsored a Google Summer of Code project to rectify this situation, and we now have a series of patches to expose most of the core functionality through XML-RPC. These patches still require a fair amount of polish and testing before being commit-ready, but they are a good starting point.

Having all the raw features accessible via the XML-RPC API would be great, but it’s also only a start. There remain some critical usability flaws that limit the usefulness of the API as a means of cross-system integration.

There’s still no way to page through the posts archive, forcing very expensive database operations to pull the full archive for large sites, often blowing past memory limits. The media management methods don’t return IDs, making it difficult to change media after they are uploaded. There’s no way to differentiate published and modified dates on posts. None of the methods support ‘since <date>’ parameters to allow efficient discovery of changes for diff-based syncing. The use of actions and filters in the method implementations are incredibly inconsistent, if they exist at all, making it very difficult for plugins to modify the behavior without wholesale replacing the built-in methods. And the list goes on…

As you may notice, many of these issues already have tickets filed in the core trac, and most of them even have patches. Yet they sit there mostly untouched. I’ve offered to help on some of them, and am willing to do much more work on XML-RPC patches, but I won’t waste my time if they aren’t going to be acted upon.

Those of us building cross-system integrations and working on the official mobile apps need a better API. There’s really no downside, especially when there are people like me who are willing to put in serious work on patches and testing. I’ve watched the core devs agonize over internal (“core”) WordPress API design decisions, but they seem not to care nearly as much about the public-facing XML-RPC API.

So I’ll be petitioning for XML-RPC to be one of the focuses/priorities for the WordPress 3.4 release cycle. Hopefully someone will agree and we won’t see another release go by without improving WordPress’ interoperability.

Better and Consistent Headers

For some reason, almost every screen in the app uses a header that consists of a relatively large WordPress logo and then the typical page header text. That makes some sense for the blog selection screen (Fig. 1), but not for inner screens (Figs. 3 & 4) when the user is well-aware that they are using the WordPress app.

On the panorama screen, the blog’s name takes prime billing (Fig. 2), which makes total sense and should be replicated (with a smaller font size/height) for all other “interior” screens in the app. If the branding is still desired, we could maybe keep a small circular WP logo in the top-left or top-right corner of the screen.

For non-panorama pages, I’d also like to enable the system tray (clock, battery, wireless signal, etc.). This eats up some vertical screen space, but I think most users prefer to see it when using apps. It would also allow us to use the system progress indicator (see below).

Figure 1

Figure 2

Figure 3

Figure 4

Use Native Controls

When the WP7 app was originally made, the Windows Phone 7 SDK was pretty new and the ecosystem was relatively weak. As such, the original developer(s) was forced to implement custom controls for certain tasks, and they reverted to emulating the controls implemented/used by the WordPress for Android app. With the Silverlight Toolkit, that is no longer necessary, and it would be a great improvement to use native-like controls instead.

Multi-select lists

For comment moderation and category selection (Fig. 5), the app employs a multiple selection list control. The current design toggles a blue background for “selected” list items, where native WP7 apps like email use a checkbox paradigm. Figure 6 shows the multi-select list control from the Silverlight Toolkit.

Figure 5 - Category Selection

Figure 6 - Toolkit Multiselect Control

 Progress indicators

Another Android-ism used throughout the app is progress spinners (Fig. 7). For the 1.2 release, we added a WP7-style progress bar to the panorama screen, but most other screens use a modal spinner animation. With the WP7.5 SDK, Microsoft made it super-easy to use native progress indicators by building it into the system tray.

One advantage of the spinner was that we could load the underlying page controls and they would be impossible to interact with until the spinner disappeared. If we switch to using the system-tray ProgressIndicator, then we will likely want to hide or disable controls while data is loading or an action is being performed (or whatever else is causing a progress indicator).

Figure 7 - Spinner

Figure 8 - ProgressIndicator

Context menus

On the blog selection screen, you can currently delete a blog by choosing “delete blog” from the appbar menu (hidden under the ‘…’) and then choose the desired blog from a modal pop-up list. It would make a ton more sense to just support the tap-and-hold context menu paradigm as supplied by the Silverlight Toolkit. I would have the context menus contain “pin” and “delete” actions.

A similar pop-up list exists for pages and posts in the panorama, though it’s less clear that a context menu makes sense in that context. But that’s a non-issue, because…

Panorama Cleanup

Perhaps the biggest concession made to the WP7 app when porting it from Android was the use of the iconic panorama control. It opens to a grid of tiles for the most common actions (see Fig. 2 above), and has additional columns for comments, posts, pages, and statistics. While there is a separate “moderate comments” screen (see Fig. 4 above), the panorama is the only way to access the functionality for posts, pages, and stats.

The WP7/Metro design guidelines call for panoramas to be like “magazine covers”, with a very small amount of content that entices users to dig deeper into the content on additional screens. But the WordPress app’s panorama has heavy lists for comments (see scrollbar in Fig. 9), pages, and posts; this is not only in violation of the design guidelines, but also makes the panorama slow to load and heavy on memory usage.

Figure 9 - Panorama Long List

Recently, I’ve been attempting to discern usage patterns for users of the various WordPress mobile apps, in addition to mobile apps for other blogging systems/services (Tumblr, Blogger, SquareSpace, etc.). As far as I can tell, the primary scenarios for mobile blog wrangling are new post creation, comment moderation, and light tweaking of recent posts.

Curiously, all of the WordPress mobile apps give equal weight to the four top-level items (comments, posts, pages, stats), perhaps because that’s what the WordPress XML-RPC API supports. But I’m in favor of optimizing the UI for the most important use-cases, and relegating some of that functionality to deeper parts of the app.

My proposal is to keep the first actions screen, but with two groups of tiles:

1. “Actions”: Add a post, Moderate comments, View stats, and Blog Settings
2. “Manage”: Posts, Pages, Comments, Users (once supported by XML-RPC API), <one tile for each custom post type (once they are supported by XML-RPC API)>

I would have three columns in addition to the actions screen: unmoderated comments, drafts, and recent posts. These correspond with the scenarios I identified above, and would include a maximum of 3-5 items each, with “more…” links leading to the corresponding archive management pivot screens.

Content-type management pages

This light-weight panorama would require new screens for management of the archives for each content type (page, post, comments, etc.). I envision these using the pivot control, allowing pivoting by the ‘status’ field (all/draft/published for posts, unmoderated/approved/rejected for comments, etc.).

The appbar on these pages would contain “add”, “search”, and “refresh” buttons. Tapping a list item would open an edit/moderate page for that item, and tap-and-hold would open a context menu for actions like “delete”, “preview”, “view comments”, etc. (replacing the pop-up modal menu used in the panorama now for posts and pages).

Using the new Mango SQL CE database support and paging for XML-RPC in WordPress 3.4 (hopefully), we can load the entire archive of a blog (probably only the most important fields like title, author, status, format, date, tags, and categories; notably, not the full body contents) and support infinite-scroll on these pivot pages. This would also back the “search” operation, and we could add a feature to the post editing page akin to the internal linking feature in WP core admin.

About & Feedback Page

The app currently has a page that shows when unexpected exceptions occur, and has links to some sources of help. This is a decent start, but the app really needs a proper ‘about’ page, with links to it in the appbar of the blog selection page and the panorama page.

The page would contain the current app version information and links to common sources of help and feedback, such as the official forums, official FAQ, and marketplace review form. We’ve been getting a number of bad app reviews in the past months, but not much feedback on the forums so that we can actually identify and address users’ issues.

Live Tiles

In the last release, we added support for per-blog live tiles on the start screen. But these tiles are still static images, with nothing “live” about them. The tiles should probably update with the number of new comments, number of views today (if stats are enabled/allowed), and maybe the text of the most recent comment on the back. This will require some design work from Isaac to make it look pretty. I like the way Evernote has tackled this in their Mango release, maybe we can try something similar.

Animations

The WordPress app is painfully static, especially when compared with the first-party apps that are full of subtle animations that make the apps feel alive. The app would feel much more polished and enjoyable if we spent some time carefully designing proper animations and transitions. The Silverlight Toolkit makes transitions super easy, and animations aren’t particularly difficult either. This will require some serious design collaboration with Isaac, since it’s critical to get animations just right or else they will just annoy users and detract from the experience.

Conclusion

The end result of all of these proposed changes would be a noticeable change to the app’s feel and flow, which is why I’ve tagged them for version ’2.0′.

One of the Windows Phone marketing slogans was “designed to get you in and out and back to life”. I know the WP7 team had a strong scenario-focused design effort, and it really shows when you use the OS. I’d really like to see the WordPress for Windows Phone app streamline and orient itself such that it’s fast and easy to tackle the most common tasks.

The most complex page of the app is the post/page editor, which also requires some serious rethinking. This post was too long, so I’ve saved my thoughts on the editor another post soon.