What a squalid mess our system for reimbursing MPs’ expenses is. Whether it’s Mr Jacqui Smith’s much-publicised solo viewing habits, the inevitable confusion among highly-paid, highly-skilled representatives about first and second homes, or the shameless London MPs claiming for a second home despite being within easy commuting distance of Parliament, things have got to change.

Gordon Brown has ordered an inquiry into the whole system. While this may produce useful reforms, former Commons Clerk Sir Roger Sands fears the inquiry itself is vulnerable to political meddling and sabotage. Given the sovereignty of Parliament, this is inevitable.

Parliament is structurally proof against any kind of effective regulation. The final veto on an MP’s behaviour lies with citizens’ votes in the ballot box. But how can citizens be sufficiently well informed to be able to make good choices?

I propose an open database of MPs’ expenses operating in near-real time. The government seem to be very keen on databases for the rest of us so I’m sure they will be keen to commit resources to making this happen.

Every line item from every receipt submitted for reimbursement is keyed into the system. We will be able to see exactly what has been claimed and which claims are pending, approved and rejected.

Every line item is tagged. This will enable people to see not just the claims submitted by specific MPs but to easily make comparisons across the group. Want to see all MPs’ claims for their televisions? It should be as easy as a visit to http://expenses.parliament.uk/tags/tvs

Given that websites shouldn’t discriminate against machines, every piece of data in the system will be available through convenient feed formats like RSS and an open API, allowing programmers to build useful mashups and visualisations of the data.

If the only way to stop cabinet ministers on £135,000 a year claiming 88p for bathplugs in their family homes is to put every such claim online within a week, let us make it so. Given that the historical data is supposedly on sale for £300,000, can we find 300,000 people with a pound each to get it?

I’d like to emphasise a point that Emma Mulqueeny has alluded to in her seven principles for digital engagement and which I also made in passing in my previous article on building local news mashups.

The web is rife with discrimination of the most insidious and socially-destructive kind. It largely goes unnoticed as those that are well-served by the web care little for the plight of those that are not.

I’m talking about the web’s widespread discrimination against machines.

Conventional thinking about “websites” focuses almost wholly on human users. In the best-case scenario, people turn up to a website, find the information they want or do the thing they want to do, then go away again. If the website is useful and provides that information or service, and if it’s usable and accessible, people can do what they want to do with a minimum of fuss and effort and be satisfied.

Many websites are a long, long way from being able to provide a good experience for human users but I doubt many don’t have it as their goal, however ineptly they may deliver on the details.

By contrast, providing for machine users mostly happens either as an afterthought or not at all.

Machine users are other programs, websites or software systems that could interact with your website by extracting data, inputting data, or both.

The simplest example is an RSS feed fetching a list of news articles or job vacancies. By providing an RSS feed, a website makes it easy for other programs to capture that information and re-use it in any way possible. They might republish that information largely as-is, combine it with information from other sources in a mashup or even derive statistical information from it. Or all of these things.

More generally an API allows information to be read from or posted into another system. Can humans search your website? Then provide a search API for machines. Can people place orders and make payments online? Then provide an ordering and payment API. Every form on your website should have an API that makes provision for machine users to post in data and get a machine-readable response. Every sequential collection of information should be available as a feed.

Providing for machine users by building APIs and serving feeds is ultimately about serving human needs. Until machines achieve some kind of consciousness they will neither know nor care about accessing information on other systems. Every program is written by and provides information for people.

APIs make it possible for people to write programs to use your information and services in ways that suit them, in ways that you can’t anticipate, in ways that you don’t have the resources to provide, through systems that you won’t have to maintain. Many organisations won’t like this. It means the creation of infinite new layers of intermediaries, third-party services that provide new interfaces to your information and services or whole new applications that combine your services with others’. If you think you can provide the best interface to your services for every conceivable context you’re a fool. If you think you should be the sole gatekeeper of your services then prepare to lose your customers to other businesses or see your citizens disappear to third-party services implemented through any means necessary.

Let’s snap out of abstractions for a moment and look at an example.

As an activist with Living Streets I’m always on the lookout for maintenance issues and faults on the street, particularly where they affect pedestrians. I can report street faults directly to my local council or I can use FixMyStreet, a national service that ostensibly does the same thing.

I always use FixMyStreet. Why?

• It’s easier. The user interface is better.
• It’s geographically agnostic. I can report a fault anywhere in the country without having to know which authority is responsible for it.
• It’s public. There’s a lot of value for me in being able to see other people’s street reports as I’m interested in looking at the wider issue of urban design and maintenance, not just getting a specific fault that bothers me fixed. I can browse other fault reports and see statistics for each borough.
• It’s flexible. I can get reports sent to me by email or through an RSS feed. I can file a report on an iPhone and soon through many other mobile clients.

Overall it’s just better, and better in so many ways that my council and most other councils will not be able to emulate.

Smart councils would realise this and most probably abandon their local street fault reporting systems. They could put their resources into developing a clean API between their own faults database and FixMyStreet (or any other similar application). They could actually invest in FixMyStreet itself. It’s open source, so why not? It’s not going to disappear, and if it gets superseded by another, better, open-source system, no-one loses.

It’s APIs (and often, ersatz, hacky APIs) that make this kind of thing possible. It leads to better services, greater participation, and more flexibility, diversity. We need to put machine users on parity with human users so that people can be best served.

If you’re building any kind of website or online service, serve a feed for every stream and an API for every form. Let this be your mantra: DDAM, DDAM, DDAM. Don’t discriminate against machines.

Like this? Follow me on Twitter or subscribe to my RSS feed.

There are various lists of UK councils which use Twitter floating around. I assume that some are compiled manually and others by screen scrapers.

Given that the rel=”alternate” attribute of (X)HTML’s <link> element is already semantically broken by the widely-adopted RSS autodiscovery spec, is there any good reason why we can’t adopt something like the code below and allow the relationship between a site and a Twitter account (or indeed, any third party site controlled by the owner) to be expressed cleanly?

<link rel="alternate" type="text/html" href="http://twitter.com/adrianshort" title="Twitter: adrianshort" />

(For HTML newbies, this goes in the <head> section of your web page.)

View source on this page and you’ll see it “in the wild” already.

Like this? Follow me on Twitter: @adrianshort

Brian loves chess but finds it hard to get a decent match with an opponent at his level. He’d love to start a local chess club but doesn’t want to take the risk of setting something up and having too few people attend. He asks on a local chat forum whether anyone would like to start a club. He gets a couple of tentative offers and a suggestion that he posts a new pledge to Sutton Council’s own local version of PledgeBank.

PledgeBank is new to Brian but he soon creates a pledge, saying, “I will join a new local chess club and pay a membership fee of up to £20 a year but only if 15 other people from Sutton will too.” A month later, Brian has found 18 members for his new club. A few people found out about it through the chat forum (to which Brian posted a link to the pledge). Some more found it just by browsing Sutton’s PledgeBank. Items from the RSS feed of new pledges appear on the council’s website home page one week in every four, which brought in five new people. One person even had a filtered subscription to the new pledges feed for “chess”.

Brian easily finds a meeting room for his group using Sutton FreeSpace, which allows people to book halls and function rooms across the borough by finding free space from the various venues’ iCalendar feeds. But Brian doesn’t need to know anything about the technology, he just asks for a room for up to 25 people any weekday evening within his price range and he gets a few options nicely plotted on a map for him with availability and pricing. He books a room at a local community centre and as the event is a public meeting, an entry is automatically created in the centre’s public calendar feed which is then syndicated to the Sutton Guardian (where it appears in print as well as on their website), Upcoming, the council’s main borough calendar and a couple of local blogs. The first meeting of the new chess club is a great success. A year later, Brian is back on Sutton FreeSpace looking for a bigger venue for their club nights.

Early one Sunday morning, Mark walks to the newsagent for his paper. He almost literally trips over a metal street bollard that has been uprooted from somewhere and dumped a couple of doors down from his house. He scans the bollard with the RFID reader in his phone and learns that the problem has already been reported to the council. Mark rolls the bollard to the side of the pavement and with a tap adds this street fault’s progress RSS feed to his phone’s reader so he can make sure it gets fixed soon. The problem was reported by a neighbour of Mark’s who has waived his privacy and allowed his name to appear on the fault report. Later that day, Mark meets his neighbour and they discuss various incidents of vandalism which they think might be related to drinkers from a local pub. The neighbour invites Mark to a Neighbourhood Watch meeting and sends the event details to his phone, which Mark then forwards to another couple of neighbours. Two weeks later the police representative at the Neighbourhood Watch meeting agrees to increase pro-active patrols around the pub area at weekends. They’re also pleased to see (with a couple of taps on a phone) that the bollard has been cemented back in place. With another tap, Mark rates this transaction as “positive” on the council’s feedback system.

bbPress comes from Automattic, the people that do the insanely popular WordPress (WP) blogging system. It’s the newest of the four systems reviewed here and hasn’t quite reached version 1.0.

bbPress asks:

Have you ever been frustrated with forum or bulletin board software that was slow, bloated and always got your server hacked?

and then promises to solve all those problems. It does a creditable job, too.

If you’re familiar with WP, bbPress won’t bring many surprises. The admin interface is very similar to WP’s, with a simple dashboard and tabs for managing users, forums, settings, themes, etc. Like all the other systems reviewed here, the visual design of bbPress can be customised using themes and its functionality can be extended with plugins. bbPress manages these in a similar way to WP, though you can’t use WP’s themes and plugins, naturally.

Basic administration is simple and unfussy. Adding a new forum takes nothing more than typing its name and clicking Add. Rearranging the order of forums on the home page is done with a lovely Ajax interface that allows you to drag your forums into the order you require, then click Save. Why should it ever be harder than this?

bbPress is designed to integrate neatly with a WP installation and thus provide forum features to a blog site. You can link a bbPress and WP system together to share a common set of users and enable single sign-on across the whole site. For the millions of WP users out there this may well be bbPress’s killer feature. For everyone else, it can safely be ignored. bbPress runs fine as a stand-alone system.

Another big win for bbPress is its use of Gravatars, a remotely-hosted system for managing user avatars (profile pictures) that can be accessed by any website. Users sign up to Gravatar and upload a photo to associate with their email address. After that, their photo will be displayed on any website that uses Gravatars. As it’s a free, open and very simple system to implement it’s widely seen, including in WP (and on the comments on this very blog). Like bbPress, Gravatar is also part of the Automattic family and it’s good to see common, generic functionality like avatars implemented through a universally accessible remote service rather than embedded into individual packages.

bbPress provides a good, clean default theme that is streets ahead of all the other systems reviewed here. In part it has the advantage of doing less and therefore having less administrative debris to clutter the screen and distract the user from the real content. Also, it mostly eschews the heavy boxes and grids designs of other systems in favour of something lighter and more restrained. While every system’s themes can be replaced and modified, the default theme sets the tone and expectations for the rest of the system. bbPress looks clean and contemporary out of the box, which is more than can be said for many others.

The interface for members is generally clean and sound but the post editor seems to lack common functionality like being able to post hotlinked images and quoting of other members’ posts. Bare URLs are converted to active links and you can use a handful of HTML tags to style your post: a, blockquote, code, em, strong, ul, ol, li. No BBCode here, which is a relief. I never did quite see the value in rewriting a wonky version of HTML with square brackets when properly sanitised HTML can do the job just as well. There’s no rich text editor and sadly no support for simplified markup systems like Markdown and Textile. Perhaps these will come before version 1.0. It may be possible to implement these via plugins.

While bbPress adopts a common forum structure as seen on all apps like this, it also emphasises the use of tags to categorise topics. I really don’t like this. Members should be concentrating on writing content, not organising it with metadata. While the tagging system can safely be ignored, the fact that it’s there means it will probably be used by many, adding to the user’s cognitive load when writing a new post and no doubt quite confusing inexperienced users. In my experience of popular sites that use tags such as Flickr and Delicious, many people still don’t really understand how to write tags well. Though they add value to sites whose primary reason is to help aggregate content from large numbers of users, the structure of forums is well understood and adequate. It’d be good to see tagging taken out of bbPress’s core and available as a plugin for those that want it. I suspect most won’t.

Overall, bbPress benefits from being a relatively new system and therefore being able to avoid the technical and functionality legacy issues that plague the older packages in this review. For WP users it’s a no-brainer. For everyone else, it’s definitely worth close investigation before you settle on a system for your new forum.

Version: 0.9.0.2

Rating: 4 out of 5.

Let’s hope version 1.0 can add on that extra star and that in three years’ time it isn’t a bloated 50-table behemoth.