Dominic Sayers

Managing my reading lists

Steve Patrick, who clearly has my education as a personal mission, has recommended another couple of books (for which I thank him). I've added them to the lists. By the way, the idea of storing del.icio.us tags in del.icio.us itself is a fabulous one which I can see all sorts of uses for. I am very pleased with my list of lists. Now, Steve's comment raises a couple of points of order. Firstly, one of the books is a work of fiction. You (o erudite reader) might think that some of the other books on my lists are works of fiction too but I haven't read them so I can't say that yet. This has led me to open a new category, fiction I should read. Hopefully it won't get too long too soon. Secondly, Sean Park suggested that one of the books on my list wasn't very good so I gave it a "deprecated" category - not removing it from the list altogether but perhaps putting it to the end of the queue. Steve Patrick disagrees with this assessment and thinks it should be restored to its rightful place. How do I deal with this? I've decided to use the Charmed principle. Steve cannot remove a spell that somebody else has cast on him. Sean either has to remove it himself or another witch needs to cast a reverse spell. Them's the rules. If you want Traders, Guns and Money restored to its place on the list then let me know. Footnote: I don't think I could manage my reading lists this well using LibraryThing, Google Books or Facebook's Visual Bookshelf. Using del.icio.us is really quick and easy and does everything I need. The book-specific tools want me to do things in a particular way but del.icio.us is general and universal.

Listening to last.fm recommendations

If you have a last.fm account, one thing you can do is listen to tracks that it recommends for you based on your listening habits. Last time I tried this it gave me a farrago of lowest-common denominator stuff that meant nothing to me. Today I tried it again and I'm loving what it is piping into my ears. This is what it recommends for me. What has happened to change this into something useful? There are a number of possibilities: 1. last.fm has more information about me I've now got my favourite tracks in a playlist and I've been playing them a lot. That's a couple of hundred eclectic tunes that represent what I like hearing. last.fm learns from this. 2. last.fm has more information The number of users of last.fm is growing and it has information about a wider selection of tracks and artists, and more information about which tracks people play a lot. 3. The matching algorithm has improved Not necessarily, but certainly possible. The improvement in the service could be explained by the first two factors alone, but I'm sure they tweek the algorithm too. My listening tastes vary from the popular to the quite obscure (like everybody else's probably), but I've been impressed with the variety of tunes last.fm has chosen for me. Artists I would not have considered because I thought I knew I didn't like them (Moody Blues for example), artists I had never even heard of (Sun Electric anybody?), tracks I had never heard (a remix of Tubular Bells with an extended drunken 3am monologue from Vivian Stanshall). Some of the selections cause me to press the Ban button straight away but presumably this contributes information to the algorithm to improve the service for me and everybody else. Try it.

Value from tag triples (or: Dominic finally gets Web 2.0)

I had noticed that geo-tagging was quite a cool feature of Flickr these days - I found a photo of my back garden gate that somebody had taken when I searched my own location. The sat nav that my wife just bought for her car has a camera built in so I guess we get automatic geo-tagging of those pictures from the GPS device. Nice, although I'll probably carry on using a proper camera for important stuff. Until today I hadn't realised that Flickr was getting quite good at what it calls "machine tags" in general. For instance, if you tag a photo with "lastfm:event=35911" then Flickr realises this isn't a straightforward human mnemonic tag and doesn't clutter up the tag list with it. So why would you use such a tag? Well, look at last.fm: if you look up Glastonbury 2005 as an event you'll see a collection of photos. These are all the photos with the tag "lastfm:event=35911". That tag is a specific instruction to last.fm to include the photo against that event. This particular tag syntax is driven by last.fm - they decided on the namespace (lasftm) and the predicate (event). The geo-tagging is slightly different - Google provide an API to their maps and satellite imagery and Flickr have enabled the link themselves by programatically encouraging people to use a particular geo-tagging syntax. For last.fm I believe there is something important missing. last.fm chose Flickr. Flickr is a good choice for them because of the sophisticated way it deals with tags and the highly functional API. But not all last.fm users user Flickr to store their photos. How can last.fm discover other sources of material about Glastonbury 2005? The obvious answer is to look for stuff on other sites that is tagged "lastfm:event=35911". But that doesn't solve the whole problem because there's no indication of what sites to look on and how to display this content. Easy enough for del.icio.us links - last.fm could simply display them as a list like Google or del.icio.us themselves do. There were no links on del.icio.us tagged with "lastfm:event=35911" when I looked a moment ago, but there could have been if last.fm encouraged del.icio.us users to use this syntax. Other content is more difficult. Another photo site would have a different API to Flickr - how can last.fm keep track of all the photo sites? Clearly it can't do so at reasonable cost. The solution might be a degree of indirection. It sould be easy enough to set up a site that mapped namespace:predicate pairs to a URI with a placeholder for the value part. Something like this: http://www.tripleviewer.com/listviews.php?namespace=lastfm&predicate=event would return a list of URIs that knew how to display things on their site tagged with "lastfm:event=nnnnn". The list would consist of things like this: http://last.fm/events/$1 http://flickr.com/photos/tags/lastfm%3Aevent%3D$1/ http://del.icio.us/tag/lastfm%3Aevent%3D$1 http://myphotos.tv/tags/lastfm%3Aevent%3D$1 ...and last.fm would only need to enumerate this list, replace the placeholder with their event id and display the results on their page for that event. It replaces the need for last.fm to know how to manage every other site with a mechanism for any site to say 'I know how to include stuff on last.fm's page'. I know things could get more complicated than this.  It would help if these views were named so that you could identify individual views. Also, I've deliberately chosen an example with a single unique identifier for the item (the last.fm event id) and very often there will be a number of elements to the unique identifier (geo-tagging has a latitude and a longitude for example). But the idea remains the same - a site where you can register views of semantically tagged data. It's just a thought that sprung up when I read the post below and I haven't thought it through properly yet. Just blogging it as a memorandum to myself at this stage. Thanks to Clagnut for the germ of this post, and as always thanks to Phil Dawes for his ongoing primer on the semantic web.

Cleaning up the tags

As you probably know, the best way to get a bargain on eBay is to stumble on an item that has been poorly described. People searching for a pirate doll, for example, will not find an item that has been described as "priate doll" and hence its eventual sale price will probably be much lower than an item correctly described. Is the same true of information? In an article in D-Lib online magazine (a DARPA-sponsored publication about digital librarianship), two digital librarians discuss whether we should make any effort to "tidy up" folksonomies.

"On this scene enter – winged, horned, and spined – A longlegs, a moth, and a dumbledore

[Hardy] might have instead written "A crane fly, a moth and a bee", had he been willing to foresake the opportunity to instill a little local colour, but his choice to use dialect or common names was inspired, and the poem benefits from it. However, a search engine would not."*

It`s undeniably true that the way items are tagged on sites like Flickr and del.icio.us is very haphazard. This is often because words are mis-spelled due to carelessness or a particular idioglossary, and any such tags are unlikely to be useful to other people unless the mis-spelling is common enough to be statistically significant. Another reason for the variation is that there is no widespread convention on whether to use singular or plural words for a tag; looking for "goose" on Flickr will not find images tagged only with "geese". The third common reason for the variation in tags for a particular thing is the one referred to in the example of Thomas Hardy`s poem. Searching for "dumbledore" will give you a lot of hits about Harry Potter and some, but far fewer, about bees. Tags are applied in many languages, and even though most are in English, English has such an enormous vocabulary that most words have synonyms. The librarians who wrote the article recognise the difficulty of educating or coercing all users to use more useful tags, although they believe that regular users will naturally tend to use the same tags as each other because these are the tags they see most frequently in searches (a Power Law effect). My own opinion is that user behaviour is unlikely to get any better than it is today. In fact the more people who use tagging systems, the higher the proportion of naive tags there will be. However, Flickr has shown that value can be added to tags by using statistical methods to enhance searching. Search for "apple" and you will get results divided automatically into images of computers and images of fruit, simply because of the additional tags that are commonly applied to those two discreet sets of images. If you Google "geese" you will get some pages that only contain the word "goose". Google is sophisticated enough to know that the two words are closely related. We can learn to deliver useful search results based on tags of relatively poor quality. All that is needed is a critical mass of tags to begin with. Let`s start tagging stuff! * Interesting to note that even in this short extract there are two words, "foresake" and "instill", that my spell checker quibbles with. Even librarians have cultural differences.

Hierarchies

Everything that is really great and inspiring is created by the individual who can labor in freedom. (Albert Einstein)

---

My doh! moment for this came when I was thinking about getting rid of folder hierarchies for my documents, and this led me on to thinking about my feed aggregator and how I should get rid of the folder hierarchy in that and replace it with tags instead. We currently struggle with the structure of a large organisation because we try to force it into a single hierarchy, starting with the CEO at the top. When the organisation is big enough to be split into regions this is "enhanced" by a separate regional hierarchy and we end up with what we optimistically call a "matrix organisation". Everybody has a functional boss running a global service and a local boss reporting to the regional head.

Big fleas have little fleas On their backs to bite 'em Little fleas have littler fleas And so ad infinitum

This arrangement was determined the first time one man employed another on a permanent basis. It is effective in small organisations because there can only be one decision maker. It is the only model that has survived for any length of time in large organisations because it enables the control framework that the senior managers feel they need to direct the organisation. Is it still necessary today, or are we using a model that was only appropriate when real-time information was not available? What other options would be available today if you were creating a brand new multinational corporation from scratch? Links:

• GE/Durham: http://www.fastcompany.com/online/28/ge.html
• SchoolTool (Mark Shuttleworth): http://mscom.rabbithole.co.za/archives/4

Social OPML

Remember how you used to save your bookmarks in the browser (Favorites if you use Internet Explorer)? Then when you discovered del.icio.us you realised that you no longer had to maintain your bookmarks in two different browsers on three separate PCs. And you could use the fact that everybody else kept their bookmarks in the same place to make connections between your bookmarks and other people's You don't go back to using local bookmarks once you have seen the world of social bookmarking. Browsers have been around a lot longer than feed aggregators. Maybe that's why we are still happy to keep our feed collections locally. Maybe we simply haven't discovered social feed collections yet. But this would be the answer to something that is annoying me a lot more now than it did a few months ago. As more and more of the information I consume is available in a syndicated format, I add more and more channels to my aggregator. Some important messages now come through my aggregator. I am starting to depend on it for critical notifications. Wherever I go, my aggregator is open on my desktop and it is as necessary to me as my email client and my chat client. So the fact that I have to re-read the same posts that I have just checked when I move to a different PC is starting to bug me. The fact that I have to add a new channel manually to each aggregator on each PC is starting to bug me. I want to keep my feed collection centrally, outside my aggregator. And the Read/Not Read status of the posts too. del.icio.us for feeds. That's what I need. You read it here first. Stop and think about it for a minute before I start talking about implementation problems. 1. There is, of course, a chicken and egg issue. There's no point storing my feed collection centrally unless my aggregator can use the central store. The central store won't get any kind of critical mass until most common aggregators are using it. None does at the moment. 2. The second point is that once you have decided to store your feed collection centrally and also the Read/Not Read status for each post then you are very close to deciding to store the posts themselves centrally. I'm not sure about this although it seems to be a logical conclusion. 3. I don't understand the business model of del.icio.us sufficiently to understand how you could finance a feed.licio.us equivalent. I hope these are mere details :-)  If we build it they will come, as somebody once said. p.s. This post arose from a conversation with Adam Tennet

Corporate feed aggregator revisited

I should clarify something I said in a previous post about the ideal corporate aggregator. I said it could not be browser-based and when I said that I had in mind a very specific architecture that would not work in a corporate environment. Thinking about this some more there are of course ways of implementing a personal aggregator with a browser interface. My objection was to feed caches that simply subscribe on behalf of their users to a set of feeds then serve them using HTML. This cannot work in a corporate environment because the server is either subscribing only to public (non-secured) feeds or supplying a general set of credentials on behalf of the entire user population. You could implement a corporate aggregator with a browser interface if you designed it so that it was in effect a virtual personal aggregator application sitting on a server. Each user would have his or her own credentials held on the server, and for secured feeds they would need their own personal storage of that content. Public content could be shared among all the users and this might cut down on the overall storage requirement across the corporation. How many times do you need to store the same Boing Boing posts? So, I will reiterate my seven requirements for an ideal corporate aggregator: 1. It must be able to supply to a secured feed the personal credentials of an individual user. 2. It must support all the authentication mechanisms used in the corporate environment: digital certificate, Kerberos, NTLM. Any more? 3. It must be able to be managed centrally by the support department. This means managing a user's portfolio of feeds for him. There are three reasons for this: (a) there are users who will not want to manage their own feeds at the outset as the technology is new to them, (b) there will likely be a standard set of feeds that all users must consume (corporate bulletins and the like) and (c) when the aggregator is upgraded or replaced we will need to preserve each user's feed collection during the process. These actions need to be scriptable in some way. 4. The Read/Not Read status of any post in any channel must follow the user around. Corporate users change locations frequently, they work from home, they make VPN connections from internet cafes. Whatever the situation they need to see the same feed collection and the same unread posts. 5. The user interface must be sufficiently rich to make consuming a large number of feeds efficient. This is another challenge to browser-based aggregators which currently do not support single-key reading. However carefully you select your channels there will always be a certain amount of noise in amongst the signal. You need to be able to move rapidly through irrelevant posts to give you more time to read the relevant ones. 6. On the same subject of signal-to-noise ratio, there are channels that frequently have useful content so you need to subscribe to them but it is mixed in with the personal bugbears of the author. Some feed publishers will provide a feed of certain categories of post so you can filter out the noise this way, but this means you miss any interesting new categories the author introduces. What I need is a way of subscribing to all posts except those which mention Unicorns for instance. The aggregator must support filtering. 7. We need to get away from the arrangement of feeds into a folder hierarchy. I want to tag my feeds with arbitrary categories. I want to be able to Mark All As Read in any given category if I am pushed for time. Der. I can't believe nobody has done this yet. Tags not hierarchy! Somebody tell me they are working on this, please.

Features of a corporate feed aggregator

Here's what I think is essential for a corporate RSS reader: 1. It must support individual authentication in order to access secured feeds. Preferably using a digital certificate to identify the user. This rules out all browser-based aggregators 2. Most users will want to get started straight away and will not want to spend a long time managing their list of feeds. We need a way of managing centrally a user's OPML file and adding a feed for them if they request it. 3. We need the Read/Not Read status of the content to follow the user around, whether he is using his regular desktop machine, working from home on his laptop, or in an internet cafe using a server-based computing environment or a VPN. 4. The consumption of new content needs to be efficient. My aggregator supports Space Bar single-key reading. I just hammer away at the Space Bar until I find a post I'm interested in. This enables me to consume a large number of feeds in a short amount of time. We will be accused of creating an UNproductivity tool unless we can demonstrate this! These things are pretty important for this technology to get accepted in the corporate environment. For my own benefits I would also like to see a feed aggregator that has the following feature: 5. Feeds are organised by tagging them, not by creating a hierarchical folder structure. Individual posts could also be tagged.

Categorising posts

I have moved my blog from Blogger to Wordpress because I think it's important to categorise my posts and Blogger doesn't allow me to do that. When I say important I mean, like Humpty Dumpty, something quite specific and possibly not all the senses of the word important are covered by my move to Wordpress. Coverage of my move in The Times has been limited for example. Important because I have said in previous posts that tagging has value. If I believe this then I should tag my posts. And here I can and will do so.

Books I should have read

For no other reason than self-flagellation I'm going to start a list of books I should have read. Hopefully this will provoke me to get on with it. Worse, I have copies of some of these and have still not got round to reading them. I will also note the must-read books I started then gave up on.

Some of these may date quickly. If I'd been compiling this list in the 70s I would probably feel I had to read The One-Minute Manager and stuff like that. Hey, there's another list! Books that are no longer required reading! The Mythical Man-Month, Men Are From Mars Women Are From Venus, etc. etc. If I procrastinate long enough many of the books on my list will drop off due to no longer being trendy (No Logo springs to mind as a candidate).

How can I manage this list? I would like other people to prioritise them for me since I am unlikely to be able to read them all anytime soon. Which do you recommend?

Where is the last.fm for books?

1. The Tipping Point: How Little Things Can Make a Big Difference, Malcolm Gladwell
2. Blink: The Power of Thinking Without Thinking, Malcolm Gladwell
3. The Wisdom of Crowds, James Surowiecki

Many, many more to come.