Dominic Sayers

Item status

Richard Rutter has come up with a better phrase to describe the Read/Not Read status of a feed item. He just calls it "item status". That will do for me from now on. He chose Shrook as his aggregator because it supported synchronisation of item status between environments back in 2004. Unfortunately it's just a Mac client. However, it's good to have company in recognising this is a valuable feature.

Social OPML

Remember how you used to save your bookmarks in the browser (Favorites if you use Internet Explorer)? Then when you discovered del.icio.us you realised that you no longer had to maintain your bookmarks in two different browsers on three separate PCs. And you could use the fact that everybody else kept their bookmarks in the same place to make connections between your bookmarks and other people's You don't go back to using local bookmarks once you have seen the world of social bookmarking. Browsers have been around a lot longer than feed aggregators. Maybe that's why we are still happy to keep our feed collections locally. Maybe we simply haven't discovered social feed collections yet. But this would be the answer to something that is annoying me a lot more now than it did a few months ago. As more and more of the information I consume is available in a syndicated format, I add more and more channels to my aggregator. Some important messages now come through my aggregator. I am starting to depend on it for critical notifications. Wherever I go, my aggregator is open on my desktop and it is as necessary to me as my email client and my chat client. So the fact that I have to re-read the same posts that I have just checked when I move to a different PC is starting to bug me. The fact that I have to add a new channel manually to each aggregator on each PC is starting to bug me. I want to keep my feed collection centrally, outside my aggregator. And the Read/Not Read status of the posts too. del.icio.us for feeds. That's what I need. You read it here first. Stop and think about it for a minute before I start talking about implementation problems. 1. There is, of course, a chicken and egg issue. There's no point storing my feed collection centrally unless my aggregator can use the central store. The central store won't get any kind of critical mass until most common aggregators are using it. None does at the moment. 2. The second point is that once you have decided to store your feed collection centrally and also the Read/Not Read status for each post then you are very close to deciding to store the posts themselves centrally. I'm not sure about this although it seems to be a logical conclusion. 3. I don't understand the business model of del.icio.us sufficiently to understand how you could finance a feed.licio.us equivalent. I hope these are mere details :-)  If we build it they will come, as somebody once said. p.s. This post arose from a conversation with Adam Tennet

Corporate feed aggregator revisited

I should clarify something I said in a previous post about the ideal corporate aggregator. I said it could not be browser-based and when I said that I had in mind a very specific architecture that would not work in a corporate environment. Thinking about this some more there are of course ways of implementing a personal aggregator with a browser interface. My objection was to feed caches that simply subscribe on behalf of their users to a set of feeds then serve them using HTML. This cannot work in a corporate environment because the server is either subscribing only to public (non-secured) feeds or supplying a general set of credentials on behalf of the entire user population. You could implement a corporate aggregator with a browser interface if you designed it so that it was in effect a virtual personal aggregator application sitting on a server. Each user would have his or her own credentials held on the server, and for secured feeds they would need their own personal storage of that content. Public content could be shared among all the users and this might cut down on the overall storage requirement across the corporation. How many times do you need to store the same Boing Boing posts? So, I will reiterate my seven requirements for an ideal corporate aggregator: 1. It must be able to supply to a secured feed the personal credentials of an individual user. 2. It must support all the authentication mechanisms used in the corporate environment: digital certificate, Kerberos, NTLM. Any more? 3. It must be able to be managed centrally by the support department. This means managing a user's portfolio of feeds for him. There are three reasons for this: (a) there are users who will not want to manage their own feeds at the outset as the technology is new to them, (b) there will likely be a standard set of feeds that all users must consume (corporate bulletins and the like) and (c) when the aggregator is upgraded or replaced we will need to preserve each user's feed collection during the process. These actions need to be scriptable in some way. 4. The Read/Not Read status of any post in any channel must follow the user around. Corporate users change locations frequently, they work from home, they make VPN connections from internet cafes. Whatever the situation they need to see the same feed collection and the same unread posts. 5. The user interface must be sufficiently rich to make consuming a large number of feeds efficient. This is another challenge to browser-based aggregators which currently do not support single-key reading. However carefully you select your channels there will always be a certain amount of noise in amongst the signal. You need to be able to move rapidly through irrelevant posts to give you more time to read the relevant ones. 6. On the same subject of signal-to-noise ratio, there are channels that frequently have useful content so you need to subscribe to them but it is mixed in with the personal bugbears of the author. Some feed publishers will provide a feed of certain categories of post so you can filter out the noise this way, but this means you miss any interesting new categories the author introduces. What I need is a way of subscribing to all posts except those which mention Unicorns for instance. The aggregator must support filtering. 7. We need to get away from the arrangement of feeds into a folder hierarchy. I want to tag my feeds with arbitrary categories. I want to be able to Mark All As Read in any given category if I am pushed for time. Der. I can't believe nobody has done this yet. Tags not hierarchy! Somebody tell me they are working on this, please.

Blog post filtering

I agree with Cory Doctorow for once. Feed Rinse is great. Feed filtering should be built into your aggregator. My €0.02, which is why I'm posting rather than just adding the link to del.icio.us, is from my standpoint as an evangelist for a corporate aggregator. If you're consuming internal feeds (i.e. feeds from your intranet) then you can't use a separate, external third-party filter. If you're consuming secured feeds (feeds that start with with "https://") then you can't add an external filter into the data flow without moving all your authentication credentials into the filter service.  It has to be built into the aggregator.

Features of a corporate feed aggregator

Here's what I think is essential for a corporate RSS reader: 1. It must support individual authentication in order to access secured feeds. Preferably using a digital certificate to identify the user. This rules out all browser-based aggregators 2. Most users will want to get started straight away and will not want to spend a long time managing their list of feeds. We need a way of managing centrally a user's OPML file and adding a feed for them if they request it. 3. We need the Read/Not Read status of the content to follow the user around, whether he is using his regular desktop machine, working from home on his laptop, or in an internet cafe using a server-based computing environment or a VPN. 4. The consumption of new content needs to be efficient. My aggregator supports Space Bar single-key reading. I just hammer away at the Space Bar until I find a post I'm interested in. This enables me to consume a large number of feeds in a short amount of time. We will be accused of creating an UNproductivity tool unless we can demonstrate this! These things are pretty important for this technology to get accepted in the corporate environment. For my own benefits I would also like to see a feed aggregator that has the following feature: 5. Feeds are organised by tagging them, not by creating a hierarchical folder structure. Individual posts could also be tagged.

Feed tagging

I've just had a doh! moment. I consume a lot of syndicated feeds in my aggregator, maybe 150 or so. I haven't counted them, I'm not a mental. Being ever so slightly towards the Aspergers end of the autism spectrum, I have carefully organised these feeds into a set of folders. My aggregator encourages me to do this and it successfully allows me to pick and choose what I read when time is tight (ain't it always). For example, I divide the incoming feeds into Internal and External: the Internal ones are coming from intranet pages at my place of work. The External ones are divided into categories such as Technology, Humour, Cat Photos etc. As I add more feeds (and I seem to add more than I can remove) this folder structure gets more labyrinthine. And it just occurred to me I wrote a blog post two days ago about what a stupid way this is to organise information. So, aggregator developers, where are your Web 2.0 credentials? I want an aggregator that lets me add tags to each feed instead of organising them into folders. Then I can slice and dice my feeds any way I want. No more forcing them into a single taxonomy. The Technology feeds can be both Internal and External. Imagine an aggregator with a tag cloud as part of its main user interface. The tags show which feed categories have unread posts. The size of the tag indicates how many unread posts there are in that category. Clicking on the tag takes me to all the unread posts from feeds with that tag. Simple.

Aggregator authentication part 94

Good news this time. I installed intraVnews and it actually authenticated with one of our corporate secured news feeds (from a Microsoft Sharepoint Server site that has Bluedog's Syndication Generator installed).

I tried the same feed with both NewsGator and NewzCrawler and neither could be configured to authenticate with the feed :-(

intraVnews is a Microsoft Outlook-based aggregator and I would prefer a stand-alone client. But beggars can't be choosers in this situation.

Feed aggregator authentication (again)

In my eternal quest to find a feed aggregator that supports a wide range of secure authentication methods I have so far drawn a blank. Some aggregators support SSL and some support HTTP Authentication. None will supply a digital certificate to the server. In a forum on the Newz Crawler site a helpful but largely anonymous person calling herself Alice suggested trying stunnel (http://www.stunnel.org/). Great! It claims to be able to wrap any TCP connection in an SSL layer and can supply a client-side digital certificate! A pity for me that it is only configurable through the shibboleth of a unix-style config file. Oh, life is too short for me to get to grips with this. Help! Anybody used stunnel before and would like to help me out?

Applications for secure syndication

I have been asked for more specific information about our requirements for secure syndication. There are a couple of things we would like to do straight away. I work for an investment bank, in the corporate finance division. We are not allowed to divulge information about the transactions we are working on to other employees of the bank, on pain of severe fines, damage to reputation, loss of license etc. We have to demonstrate that we are taking precautions to enforce these "chinese walls". Our customer relationship management system is updated daily with new information about the transations we are working on. I would like these regular updates to flow into people's news aggregators. BUT ONLY THOSE PEOPLE WHO ARE ENTITLED TO SEE THAT INFORMATION SHOULD RECEIVE IT. We not only need to control the information flow, but been seen to be doing it in an auditable way. The second application for this technology would be in publishing research reports for the bank's customers. Our highly-paid research analysts need to be sure that only the people who have paid for their reports actually receive them. This means we have to absolutely identify the person who is requesting the report. If we had a news aggregator that could supply a digital certificate then we could pump out the research reports using RSS or Atom over SSL to these customers. That would be cool.

Secure syndication

Two contradictory thoughts. If companies wish to disseminate commercial information using a syndication protocol then they need a news aggregator that can supply authentication credentials to the server. Some already can, but, depending on their compatibility with the authentication methods available on the server, it is possible they will be transmitting usernames and passwords unencrypted over the internet. This is clearly a Bad Thing. If Digest Authentication were reliably available then this would mitigate the problem. This would be an appropriate solution for a company wishing to set up a public, but paid-for, service. For the type of information my company wants to disseminate, we would prefer to issue our customers with a digital certificate. I am not enough of a security guru to comment on the relative merits of this technology compared to username/password, but I know it is the norm in my industry (financial services) to operate like this. Here (at last) is my point: the first news aggregator to offer digital certificate authentication is in a unique position. Not only is it likely to be widely adopted in the financial services industry, to the complete exclusion of its competitors, but it will also generate a good deal of publicity among some very influential consumers, for syndication technologies in general. Blogging will have grown up all of a sudden. What is the downside? Well although financial services is a large and influential industry, it isn't mass market. If, as a news aggregator product owner, your business model is to make $0.02 from everybody on the planet then it's probably not worth building this. On the other hand you would like to sell to an industry that doesn't mind paying top dollar for the right technology then go right ahead and add this feature.