When you own a domain you’re a first class citizen of the web. A householder and landowner. What you can do on your own website is only very broadly constrained by law and convention. You can post the content you like. You can run the software you want, including software you’ve written or customised yourself. And you can design it to look the way you want. If you’re paying for a web hosting service and you don’t like it (or they don’t like you) you can pack up your site and move it to another host. Your URLs will stay the same and so your visitors won’t notice. You get a great deal of freedom in return for the cost of running your own site. Your site could still be there in a decade’s time, possibly even in a century.

If you use a paid-for web service at someone else’s domain you’re a tenant. A second class citizen. You don’t have much control. You’ll probably have to live with your landlord’s furniture and decoration and a restrictive set of rules. Your content will only exist at these URLs for as long as you keep paying the same people that monthly fee and for as long as your provider stays in business. Experience tells me that this isn’t very long. As a paying customer you’ll have a few rights under your contract but they probably won’t amount to very much. When you leave you’ll probably be able to get your data back in a useful format but when you put it back on the web somewhere else you’ll lose all your inbound links, search engine rankings and many of your visitors. This kind of service seems like a good deal until the day you need to move.

When you use a free web service you’re the underclass. At best you’re a guest. At worst you’re a beggar, couchsurfing the web and scavenging for crumbs. It’s a cliche but it’s worth repeating: if you’re not paying for it you’re the product not the customer. Your individual account is probably worth very little to the service provider, so they’ll have no qualms whatsoever with tinkering with the service or even making radical changes in their interests rather than yours. If you don’t like it you’re welcome to leave. You may well not be able to take your content and data with you and even if you can, all your URLs are broken.

The conclusion here should be obvious: if you really care about your site you need to run it on your own domain. You need to own your URLs. You’ll have total control and no-one can take it away from you. You don’t need anyone else. If you put the effort in up front it’ll pay off in the long run.

But it’s no longer that simple.

Anyone who’s ever run a website knows that building the site is one thing, getting people to use it is quite another. The smaller your real-world presence the harder it is. If you’re a national newspaper or a Hollywood star you probably won’t have much trouble getting people to visit your website. If you’re a self-employed plumber or an unknown blogger writing in your spare time it’s considerably harder.

Traffic used to come from three places: the real world (print advertising, business cards, word of mouth, etc.), search engines and inbound links. Whichever field you were in and at whichever level, you were competing against other similar sites on a fairly level playing field.

Social networks have changed all that. Facebook and Twitter now wield enormous power over the web by giving their members ways to find and share information using tools that work in a social context. There’s no obvious way to replicate this power out on the open web of independent websites tied together loosely by links and search engine results.

Not so long ago you had to be on MySpace if you were an up-and-coming band. Now it’s probably Facebook. Either way, your social network presence is more important than your own website.

If you’re an independent photographer looking to get established you probably need to get your pictures on photo sharing sites like Flickr where they can be easily found by millions.

Many of the most valuable conversations around technology and many other fields happen on Twitter. If you’re not there you don’t really exist, especially if you’re just getting started in your field.

You can turn your back on the social networks that matter in your field and be free and independent running your own site on your own domain. But increasingly that freedom is just the freedom to be ignored, the freedom to starve. We need to use social networks to get heard and this forces us into digital serfdom. We give more power to Big Web companies with every tweet and page we post to their networks while hoping to get a bit of traffic and attention back for ourselves. The open web of free and independent websites has never looked so weak.

Perhaps none of this would matter very much if the biggest player of them all — Facebook — wasn’t such a grotesque abuser of its position. Even before announcing Open Graph this week it was pretty clear that Facebook wanted to own everything everyone does online. Facebook currently has 750 million members. If it were a country it’d be the third most populous country in the world, bigger than everyone except China and India. The United States has a mere 312 million people — not even half the size of Facebook.

Facebook’s Open Graph technology allows third-party websites to tell Facebook what people are doing. It extends Facebook’s Like button to include any action that the site owners think might be interesting to Facebook. Play a song and your music streaming site tells Facebook what you’ve played. Read a newspaper article and Facebook knows what you’ve read. LOL at a lolcat and your LOL gets logged for all time on your indelible activity record. Facebook calls this “frictionless sharing”, which is their euphemism for silent total surveillance. Once you’ve signed up for this (and it is optional, at least for now) you don’t need to do anything else to “share” your activity with Facebook. It’s completely automatic.

Site owners and developers are lapping it up. Hosting company Heroku posted this incredible tweet the day after Open Graph was announced:

Huge Open Graph momentum with social devs, we’ve seen more than 33,800 new Facebook apps in last 24 hours #f8

Yes, that’s nearly 34,000 new Facebook apps created in one day by customers of just one hosting company. Astonishing numbers.

At least Facebook is up front about Social Graph. Facebook’s abuse of its Like button to invade people’s privacy is much less publicised. We all think we know how it works. We’re on a website reading an interesting page and we click the Like button. A link to the page gets posted to our wall for our friends to see and Facebook keeps this data and data about who clicks on it to help it to sell advertising. So much so predictable.

What most people don’t know is that the Like button tracks your browsing history. Every time you visit a web page that displays the Like button Facebook logs that data in your account. It doesn’t put anything on your wall but it knows where you’ve been. This even happens if you log out of Facebook. Like buttons are pretty much ubiquitous on mainstream websites so every time you visit one you’re doing some frictionless sharing. Did you opt in to this? Only by registering your Facebook account in the first place. Can you turn it off? Only by deleting your account.

This is where I draw the line. I’m well aware that everything we do online and many of the things we do in the real world creates a data shadow — a digital record of our actions. If you carry a mobile phone your location is continually recorded by your phone company. If you’re suspected of a crime or go missing then this data will be handed to the police. Most of us know this and choose to use mobile phones anyway. We know that when we buy things that transaction is recored by our bank and the shop unless we’re using cash. We know that our computers and our broadband providers record what we do online. But all these things are predictable and at least arguably necessary to provide the services we use. We might not like these intrusions into our privacy but we like the law enforcement, fraud protection and service quality that they buy us. It’s a compromise that most of us are willing to make.

What Facebook is doing is very different. When it records our activity away from the Facebook site it’s a third party to the deal. It doesn’t need this data to run its own services. Moreover, Facebook’s aggregation and centralisation of data across all our disparate fields of activity is a very different thing from our phone company having our phone data and our bank having our finances. Worst of all, the way Facebook collects and uses our data is both unpredictable and opaque. Its technology and policies move so quickly you’d need to be a technical and legal specialist and spend an inordinate amount of time researching Facebook’s activities on an ongoing basis to have any hope of understanding what they’re doing with your data.

As individuals we can opt out. It’s still possible to live a full life in the developed world and not use social networks. Some people may find it harder than others — missing out on event invitations that are only sent on Facebook, for example. Not being able to see your friends’ photos because they’re only posted to Facebook. Not being able to join conversations on Twitter. But for now there are sufficient alternatives for most of us. As with smoking, it’s easier to not start using the social web than to stop. Once you’ve signed up the cost of leaving increases with every “friend” you make, every photo you post, every tweet you send. That’s why I’m holding out against Google+ for now.

For organisations and business it’s very different. We’re already past the point where social networks can be ignored. If you don’t have a social networking presence your businesses is at a significant disadvantage compared with those that do. It’s where the attention, the traffic and the conversations are. Even public and government services are finding their social networking activities increasingly important. How long before they’re essential?

The promise of the open web looks increasingly uncertain. The technology will continue to exist and improve. It looks like you’ll be able to run your own web server on your own domain for the foreseeable future. But all the things that matter will be controlled and owned by a very small number of Big Web companies. Your identity will be your accounts at Facebook, Google and Twitter, not the domain name you own. You don’t pay Big Web a single penny so it can take away your identity and all your data at any time. The things you can say and do that are likely to be seen and used by any significant number of people will be the things that Facebook, Google and Twitter are happy for you to say and do. You can do what you like on your own website but you’ll probably be shouting into the void.

If I find any answers I’ll post them but right now things are looking bleak. It’s the end of the web as we know it and I feel pretty far from fine.

@adrianshort

Lee Jasper got more than a few backs up today with a tasteless comment on Twitter about the Norway shooter at the supposed expense of Boris Johnson:

Boris Johnson and Breivik anyone notice the striking similarities or is just me? He could be his younger brother

Jasper then proceeded to compound the offence caused by throwing around various racist comments to the effect that all white people were responsible for what happened in Norway.

All par for the course from Jasper. Or was it?

@andywasley suggested that Jasper’s Twitter account might have been hacked and that the offensive posts were written by an imposter.

Jasper certainly never claimed that his account was hacked so we can presume that the tweets were by him. He also went on to lock the account so that people couldn’t read what he’d written.

But if we needed to, how would we judge whether Jasper’s tweets today were in character or out of it?

Step forward the naive Bayes classifier — the kind of software that protects your inbox from email spam.

The Bayes classifier is an artificial intelligence tool that can be used to sort documents into various categories. For email the categories would be spam and not-spam (a.k.a. “ham”). The not-spam ends up in your inbox and the spam gets filed somewhere else.

The Bayes classifier uses mathematical probability to make its decisions. It won’t be right all the time but they tend to have a pretty good success rate.

We can use the same technique to see whether Jasper’s tweets today were likely to be his.

I set up the Ruby classifier gem for this job.

First we define two categories: Leejasper and Notleejasper.

Then we train the classifier using documents — tweets — that we know belong in each category.

I downloaded a batch of tweets from Jasper’s account. There were 97 from today and 442 before today.

So the 97 are “unknown” tweets that need to be classified. The 442 tweets from before today are used to train the Leejasper category.

We also need to train the Notleejasper category. I downloaded 8187 tweets from various people’s accounts, political and non-political, and used those.

Running the 97 “unknown” tweets supposedly from Lee Jasper today through the classifier indicates that 87 are Leejasper and 10 are Notleejasper — a result of 89% in favour of Leejasper.

And what of the other 10? They’re mostly very bland and could have come from anyone:

Notleejasper:  Actually I agree...
Notleejasper:  He wont be getting my vote either..
Notleejasper:  I think there are certain circumstances where its entirely appropriate to do so the AC Leukaemia Trust being one of them.
Notleejasper:  My point is the historical trend that provide the context for your example.
Notleejasper:  Of course what...?
Notleejasper:  What I actually said is that they 'look alike' so get of your high horse.
Notleejasper:  a rare thing indeed...
Notleejasper:  agism in the workplace don't make me laugh. The majority don't want to work longer thats why we have mass industrial action.
Notleejasper:  and the reason for that is?
Notleejasper:  what you dont think they look alike?

Overall a fairly persuasive result. I’ll reiterate that Lee Jasper hasn’t disowned his tweets from today but if he did he’d be on pretty thin ice.

Walsall Council tweeted their activity for 24 hours on 4-5 March using the #walsall24 hashtag. Here are my responses to points made in a discussion on a Guardian article about this project. The whole discussion thread from the Guardian was subsequently deleted for unknown reasons.

Many of the tweets are trivial and banal (Atomant77)

Taken out of context, just about everything is trivial and banal. The time of the next bus from here to the town centre is trivial and banal unless you’re here and you want to get to the town centre.

But that’s what happens when you release comprehensive information about something. Most of it isn’t of interest to most people. Conversely, there tends to be something for everyone. Just look at the Freedom of Information requests that people make.

I don’t live in Walsall but I was very interested to see that there was a clairvoyant appearing at a council library to teach Tarot. As a rationalist, I don’t think this is the kind of thing councils should be subsiding. Does it happen in my area, too? It turns out that it does. I’ll be following this one up.

When you’ve got information on a computer you can slice and dice it any way you like. Cut through the mass of information you don’t care about to find what you do.

Twitter isn’t a good medium for reaching Walsall’s residents. It’s just for the “chattering classes”. What about my 85-year-old gran? (liberalcynic)

As Chuffy and HenryHomer said, this is an experiment. It’s not a new council service and they won’t be doing this every day.

If councils are going to improve their services over the long term they need to experiment with new ideas. This doesn’t mean committing massive resources to untested ideas. It means doing exactly what Walsall is doing here: Short, one-off projects that are cheap and have no adverse impact on other services.

You don’t have to have a very long memory to remember when councils didn’t have websites. And if you remember that, you’ll probably also remember the people who were resistant to councils having websites. The internet was just for geeks and the chattering classes, they said. Well, look at it now. No, we still don’t have everyone online (nor equally good access for those that have it) but I hope no-one seriously still thinks that the web is a waste of time.

With half the country now on Facebook, councils learning how to use social media looks pretty important, not only because there’s already a huge audience there but because most of the other half will follow soon enough.

More generally, this project is about capturing and disseminating information. Just because it passes through a computer doesn’t mean that it’ll necessarily be consumed on one. Web pages can be printed out. So can RSS feeds. Data feeds can be displayed on public screens like the countdown boards at bus stops and train stations. Software can send out text messages that can reach just about everyone. I’m looking at #walsall24 and thinking, “How could we automate this? What else could this approach be used for?” I see nothing wrong with Walsall blazing the trail here for others as well as themselves. Everything has to start somewhere.

Walsall’s Twitter experiment is a drop in the ocean, but reminding people of all the shitty stuff that councils do is no bad thing. (Chuffy)

… and …

It’s just a shallow PR exercise to make the council look good (liberalcynic)

It may have “image” benefits in a PR-sense but I think this is more about engagement than self-promotion.

Many people missed the point of #gmp24, which as I remember it was to show people how much time Greater Manchester Police spent doing “social work” rather than fighting crime. It wasn’t so much “see how wonderful we are” as “see how our time gets wasted”. They wanted people to think about the role of the police and how it could best serve the community rather than affirm what a great organisation they were. What Walsall is doing with #walsall24 seems similar to that aim.

In my view, esteem has to be earned. If proper communication helps services to be accessible, efficient and popular, then esteem for the council will surely rise. (liberalcynic)

I take this point entirely. Councils should be engaging with residents and making themselves accountable to them rather than bigging themselves up. #walsall24 certainly couldn’t be rolled out as it is as a regular council service, but I’ll definitely be trying to think of ways in which some of the ideas could be applied to realise tangible benefits at a sustainable cost. Birmingham’s civic dashboard is taking steps in that direction and I expect to see far more realtime, fine-grained information being made available by councils and used across many media.

Twifakes is a spam website created by Cairo Noleto @caironoleto and Cleiton Francisco @cleitonfco. I’m sure they’ll be happy to answer any questions you may have about it.

You may have seen the website at http://twifakes.heroku.com/ which promises to tell you how many “fake” Twitter followers you have.

Do not authorise this website. It tweets without your permission and there’s no telling whether it may do other damage to your account.

Continue Reading »

A common social media dilemma solved:

Imagine you’re running social media for a public library service. You’ve got ten libraries in the service and you want to use Twitter, Facebook and Flickr.

How many accounts do you need?

Continue Reading »

Last week I launched @SutMobLib, a Twitter account that tweets the location of Sutton’s mobile library in real time. No, I’m not sitting here all day sending messages. A program does that automatically. Every time the library gets to a new stop it posts up its location.

Continue Reading »

There are various lists of UK councils which use Twitter floating around. I assume that some are compiled manually and others by screen scrapers.

Given that the rel=”alternate” attribute of (X)HTML’s <link> element is already semantically broken by the widely-adopted RSS autodiscovery spec, is there any good reason why we can’t adopt something like the code below and allow the relationship between a site and a Twitter account (or indeed, any third party site controlled by the owner) to be expressed cleanly?

<link rel="alternate" type="text/html" href="http://twitter.com/adrianshort" title="Twitter: adrianshort" />

(For HTML newbies, this goes in the <head> section of your web page.)

View source on this page and you’ll see it “in the wild” already.

Like this? Follow me on Twitter: @adrianshort