Dominic Sayers

HOWTO: Fix problems with Firefox extensions

I have been having problems with a Firefox extension that disappeared whenever I upgraded either Firefox or the extension itself. My specific problem was with an extension called Firebug, but I'm betting this affects other add-ons too. After extensive searching I eventually came up with an answer that worked for me. I'm posting it here so other people might avoid the hot tears of frustration that I cried trying to fix this.

The symptom

You choose a Firefox extension by clicking on the install link, the extension downloads and installs. You click the button to restart Firefox and the extension is simply not there. No error message, no indication why it failed. Just stubborn, passive-aggressive refusal to install. Arms folded, lips pursed, the extension just won't do what you want. Gah.

The solution

Yes, it's the old favourite - your virus checker. In my case all I had to do was open the AVG Anti-Virus control panel, double-click Resident Shield and unclick the "Resident Shield active" checkbox. Click on Save Changes. Then if you repeat the Firefox estension install you should find it installs OK. Don't forget to turn your virus checker back on afterwards.

Comments in email addresses

Quick links: Source code | Email address validators head-to-head

I was turning a blind eye to the part of RFC5322 that allows you to put comments within an email address. But Cal H brought it up in an email so I had to bite the bullet. On reflection I think this was worthwhile. The most common error in email address validators is that they reject valid addresses. This really annoys people who like to put a '+' in their address and find they can't because registration form won't allow it. Why do they like putting a '+' in their address? Well it effectively tags the incoming email for you automatically. Mail sent to first.last+hello@example.com will go to the first.last mailbox, tagged with 'hello'. GMail will do this for you - try it. So that's why I think it's worth allowing comments. The next GMail might be able to do the same thing or something even more useful with comments: first.last(notify IM)@example.com Version 1.6 of my validator now passes all 222 unit tests. So does Cal's. I see no reason why you wouldn't use one of these in your project: they are free and they work. Why reinvent the wheel?

RFC nerd notes

Comments can contain folding white space and can be nested. This is the final nail in the coffin for regular expressions that claim to validate email address. Show me a regex that says this is a valid address: first(Welcome to  the ("wonderful" (!)) world  of email)@example.com A thank-you also to Paul Gregg who allowed me to add his validator to the head-to-head (and added mine to his page). He also provided some more unit tests.

Quick links: Source code | Email address validators head-to-head

light@the.end.of.the.tunnel

Quick links: Source code | Email address validators head-to-head

The amazingly helpful Cal Henderson has nearly caught up in the arms race that is email address validation. After our latest round of discussions we disagree about only one of the 161 test cases in the test suite. Cal's validator successfully validates all the test cases except that one (and he may be right about it -we'll see). I've released version 1.3 of my validator and you can download it with the test cases from Google Code.

RFC nerd notes

I had a lesson from Cal on Folding White Space. Who knew that you could have an email address that was split over several lines? Please don't run out and try this - it's strictly for completeness. Secondly, all those validators out there that use RFC2822 as their authority: those are yesterday's validators. All the cool kids are validating using RFC5322. It's the latest thing.

Quick links: Source code | Email address validators head-to-head

Email address validation 1.2

Quick links: Source code | Email address validators head-to-head

I had some very helpful input from Cal Henderson on interpreting RFC 2822. I've included his latest validator in my head-to-head comparison and updated the test suite to reflect the consensus from our discussion. I've also updated my own routine so that it still passes all the tests, of which there are now 158. As always, the full analysis is on my website and the latest version of the source code will always be in Google Code. I would really appreciate it if you find a valid address that fails my validator, or an invalid one that passes, if you would get in touch with me and let me know. Here are the latest scores: Dominic Sayers: 100% validated correctly Simon Slick: 91% Cal Henderson: 84% Phil Haack: 80% Dave Child: 79% I know Cal is working on IPv6 validation so this may change soon.

RFC nerd notes

There are two principle changes in this version. Firstly, I now accept Dave Child and Cal Henderson's opinion that a backslash can escape anything in a quoted string local element. However, it must escape something - it can't be at the end of the string. Secondly, if the string is not quoted then the local element must be a vanilla atom. You can't escape individual characters unless the element is surrounded by double quotes. By "local element" I mean a single component of a dot-delimited local part. I've extracted the raw specification of a valid email address in BNF (Backus-Naur form) from RFC 5322 and put it here: The BNF from RFC 5322 defining parts of a valid email address. If you try to follow this in the RFC you end up going backwards and forwards and getting confused - it's much easier to understand in the way I've laid it out.

Quick links: Source code | Email address validators head-to-head

Email address validation 1.0

Quick links: Source code | Email address validators head-to-head

OK, after some correspondence with Dave Child and Cal Henderson I have now released version 1.0 of my PHP email address validator. This version passes all 139 unit tests in the suite - tests I have collated from RFC 3696 and from articles by Dave Child, Doug Lovell and Phil Haack plus some of my own to test the IPv6 address format. The full analysis is on my website and the latest version of the source code will always be in Google Code. I would really appreciate it if you find a valid address that fails my validator, or an invalid one that passes, if you would get in touch with me and let me know. Here are the scores on the doors: Dominic Sayers: 100% validated correctly Simon Slick: 86% Dave Child: 81% Phil Haack: 79% Cal Henderson: 71% For the RFC nerds, this version now correctly validates the obsolete local form of word *("." word). In other words you can regard the local part of the address as a dot-delimited series of elements each of which can be separately quoted. This means that first."last"@example.com is now recognised as a valid address.

Quick links: Source code | Email address validators head-to-head

Social data flow in action

OK, I've seen a few things online recently that amused me and I'd like to share them. I don't want to add them to delicious.com because that's what I use for links I need to find again someday. These are things I've seen and enjoyed but they are gone. For me they are yesterday's newspaper. But you may not have seen them and I think you should. So I've decided to add them to Tumblr. Easy. Tumblr is a great tool for doing this, it makes it easy to add videos, links, pictures, anything. I am very happy with this choice. But this has now repurposed Tumblr in my social data flow from an output device to a content provider. Before, it just used to receive my status updates and micro blogs. Now it is the place where I link to things that have amused me. Here's how I needed to reconfigure my social data flow: 1. Add Tumblr to my aggregators. Now my Tumblr posts will appear in my All Activity feed. 2. Remove Tumblr from my syndication list. Now Tumblr will not receive my status updates or micro blogs. This is important because otherwise these updates would appear twice in my All Activity feed. 3. Er... 4. That's it. Simple.

Revised social data flow

In an attempt to remove the latency problem from my social data flow, I've rearchitected it a bit. Unfortunately the reworked flow is not quite as elegant as the original but then it had more sophisticated design goals so that's not terribly surprising. Here's the revised flow with notes below:

(Click on the image for a bigger version)

Notes:

1. Now I can allow ping.fm to update Twitter and identi.ca immediately so I have removed the latency problem from the previous configuration. Once again I can use Twitter in a conversational way instead of it being a mere receptacle for RSS updates every half hour.
2. You'll note also that Twitterfeed can now update identi.ca and other Laconica-based micro-blogging sites. Thanks, Mario. This means I can promote identi.ca to be a peer with Twitter and receive all my updates including real-time micro-blogs (tweets).
3. The three subscription points are the same: Writing, Status Updates and All Activity. The URIs for those resources have not changed.
4. I have distinguished between blog posts such as this and the notification of those posts. To subscribe to the full post stream use the Writing feed. If you subscribe to the All Activity feed you will be notified about posts but only see the title not the whole article. Unfortunately if you subscribe to both you will see both the blog post and a separate notification of its existence - I can't do anything about this yet.

So there's an emerging gap between those select sites that I use to do stuff and those sites that merely receive notification of that activity. My strategy will be to keep to a minimum the sites I use for doing things, and maximise the number of sites that are notified about it. This way I can keep all my pictures, links, writing etc. in one place but all my disparate multiple friend communities will get equal notification of new activity in one of those upstream sites. I think the new slightly less elegant social data flow is functionally better and I'll try to stick with it for a while to see how it performs in practice.

Initial thoughts on my new social data flow

Firstly, apologies to anybody who couldn't see the social data flow diagram in my earlier post - I've now hosted it on my own server so it shouldn't disappear quite so often.

(Click on the image for a bigger version)

The new data flow, using ping.fm to syndicate my status updates and micro-blogs and FriendFeed to aggregate my other activity and consolidate everything into a coherent set of feeds, is working quite well. I would like to be able to have more control over the format of what both ping.fm and FriendFeed do for each target system, but the defaults are adequate for now. Here are the issues I'm experiencing: 1. Using Twitterfeed to take my FriendFeed RSS and pipe it into Twitter introduces a degree of latency that's a bit inappropriate for Twitter. Twitter should be near real-time and conversational (whilst not being an instant messaging system), but Twitterfeed only reads my FriendFeed every half an hour. If I get ping.fm to update Twitter directly then I'm going to get some updates twice, which is exactly what I'm trying to avoid. 2. Some of the connected systems (identi.ca for example) are Twitter-like, and I'd like to treat them the same way as Twitter: as an output device for all my activity. But Twitterfeed is Twitter-specific and I haven't found an equivalent for identi.ca etc. yet. In any case I don't want a different Social ETL tool for each output system; I want Twitterfeed (or something) to do this for a number of target systems. 3. There's no class solution for location-based systems yet. Obviously I'd like to be able to update my location and travel plans in one place and have it propagate to TripIt, Dopplr, BrightKite etc. I think FireEagle may be trying to do this but it's short of two things at the moment: co-operation from the other systems and an invitation for me. If anybody has access to FireEagle I would appreciate a way in. 4. Now I need to do the other thing - deduplicate my profligate friends' updates. When Myrto uploads a picture to Flickr, for instance, I get notified about it four times. I can rationalise this a bit but I can't ignore her FriendFeed or Twitter because I would miss some of her updates, so I'm condemned to hearing about her new pictures multiple times unless (a) she adopts a version of my social data flow or (b) I find a solution to deduplicating the same update coming through multiple channels. This duplicate filter is necessary until everybody adopts my architecture, i.e. for ever. Does anybody know of a solution out there? This all seems quite difficult to manage and well beyond anybody who is either busy or technically challenged. There's no real way of developing a packaged solution until all the social systems adopt a single sign-on technology like OpenID, and many of them have sound technical reasons for not doing so (OpenIDs can be created by anybody anywhere so it's a bit like expecting system owners to trust a digital certificate that has no trusted root certification authority). Anyway, that's the state of play at the moment. More to come I'm sure.

Managing my reading lists

Steve Patrick, who clearly has my education as a personal mission, has recommended another couple of books (for which I thank him). I've added them to the lists. By the way, the idea of storing del.icio.us tags in del.icio.us itself is a fabulous one which I can see all sorts of uses for. I am very pleased with my list of lists. Now, Steve's comment raises a couple of points of order. Firstly, one of the books is a work of fiction. You (o erudite reader) might think that some of the other books on my lists are works of fiction too but I haven't read them so I can't say that yet. This has led me to open a new category, fiction I should read. Hopefully it won't get too long too soon. Secondly, Sean Park suggested that one of the books on my list wasn't very good so I gave it a "deprecated" category - not removing it from the list altogether but perhaps putting it to the end of the queue. Steve Patrick disagrees with this assessment and thinks it should be restored to its rightful place. How do I deal with this? I've decided to use the Charmed principle. Steve cannot remove a spell that somebody else has cast on him. Sean either has to remove it himself or another witch needs to cast a reverse spell. Them's the rules. If you want Traders, Guns and Money restored to its place on the list then let me know. Footnote: I don't think I could manage my reading lists this well using LibraryThing, Google Books or Facebook's Visual Bookshelf. Using del.icio.us is really quick and easy and does everything I need. The book-specific tools want me to do things in a particular way but del.icio.us is general and universal.