Dominic Sayers

Mutual societies and apocalyptic events

Back at the dawn of time (2006), I wrote a series of posts to record some things I believed in. These were things I believed without being able to prove they were true - naked belief. I'm happy to say I still hold all these beliefs. Not that I cling to opinions when the facts change - I'm with Keynes on that one. But history has not demonstrated that any of these beliefs are wrong. In fact, history has started to provide some evidence in favour of some of these beliefs.

Mutual societies

Today news emerged of the Dunfermline Building Society's failure as a going concern. I claim this actually reinforces my belief that mutual societies should be some sort of protected species. What went wrong seems to be the the Dunfermline stopped acting in the long-term interests of its members and started behaving as if it had shareholders. Making aggressive commercial loans and approving risky mortgages is not what a mutual society should be doing (unless it's a mutual society of capital markets professionals). So my belief is refined somewhat: mutual societies should be encouraged by (for example) favourable tax legislation but their activities should be constrained by some sort of covenant that members are made aware of when they join.

Apocalyptic internet event

You're mad! was the usual reaction when I tried to persuade the pezzonovanti at my last company that there was a seismic event waiting to happen in the financial services industry. Turns out I was right, but I'm not claiming to have predicted the sub-prime crisis. My event was to do with data security and it hasn't happened yet. But the bots are gathering, and they are being expoited commercially in an ever more sophisticated way. The datapocalypse is coming. Don't say I didn't warn you. There's more than one black swan.

HOWTO: Fix problems with Firefox extensions

I have been having problems with a Firefox extension that disappeared whenever I upgraded either Firefox or the extension itself. My specific problem was with an extension called Firebug, but I'm betting this affects other add-ons too. After extensive searching I eventually came up with an answer that worked for me. I'm posting it here so other people might avoid the hot tears of frustration that I cried trying to fix this.

The symptom

You choose a Firefox extension by clicking on the install link, the extension downloads and installs. You click the button to restart Firefox and the extension is simply not there. No error message, no indication why it failed. Just stubborn, passive-aggressive refusal to install. Arms folded, lips pursed, the extension just won't do what you want. Gah.

The solution

Yes, it's the old favourite - your virus checker. In my case all I had to do was open the AVG Anti-Virus control panel, double-click Resident Shield and unclick the "Resident Shield active" checkbox. Click on Save Changes. Then if you repeat the Firefox estension install you should find it installs OK. Don't forget to turn your virus checker back on afterwards.

South Africa bids to join the big league

Cricket South Africa (CSA) has made a strong move in the war to be the most ridiculous cricket governing body in the world. Don't forget this is big-league stuff - cricket's governing bodies are strong contenders for The World's Worst Sports Governing Body. Following on from brilliant efforts by the ECB[1], the WICB[2] and the ICC[3], CSA have appointed the likeable Ashwell Prince as the stand-in captain for the next Test Match then relieved him of the captaincy 24 hours later, appointing one of the world's least popular cricketers in his place - Jacques Kallis. This is on top of moving him from his favoured middle-order position to open the batting in place of the injured Graeme Smith. And dropping Neil Mackenzie for the crime of scoring 1073 runs in 2008 at an average of 50+. Moves like this always reveal underlying tensions in the governing body and team management. In this case the arrival of Mike Procter as the head of selectors seems to have been the catalyst. But it's unfair to blame Procter for this collapse into incompetence. Somebody appointed him (Gerald Majola presumably). CSA needs to get its act together quickly, or the relocated ICC Champions Trophy will join the long list of international tournaments that have made cricket a laughing stock in world sport. [1] The England & Wales Cricket Board (ECB) quite appropriately conceals the Welsh element from its abbreviation since cricket is a long tail sport in the principality. Notwithstanding this, the ECB have chosen the hotly-anticipated 2009 Ashes series to introduce a new venue to the Test Match roster. One that is smaller than all the other venues on the circuit. One that is not even the largest stadium in its town, or even the second largest. One that is in Wales. [2] Where to start? The West Indies Cricket Board (WICB) is responsible for squandering the revenue from the 2007 World Cup and Sir Allen Stanford's brief flirtation with cricket in the region. The result has been white-elephant stadiums in places that ordinary West Indians can't get to and which aren't even fit to play the game. CLR James would weep, as should everybody else. [3] Venal and incompetent, the International Cricket Council is dominated by Indian television revenue making it difficult for it to take a neutral view of world cricket. Incapable of running a decent tournament itself, it tries to avoid blame for other international cricket disasters by disassociating itself from, well, international cricket.

Comments in email addresses

Quick links: Source code | Email address validators head-to-head

I was turning a blind eye to the part of RFC5322 that allows you to put comments within an email address. But Cal H brought it up in an email so I had to bite the bullet. On reflection I think this was worthwhile. The most common error in email address validators is that they reject valid addresses. This really annoys people who like to put a '+' in their address and find they can't because registration form won't allow it. Why do they like putting a '+' in their address? Well it effectively tags the incoming email for you automatically. Mail sent to first.last+hello@example.com will go to the first.last mailbox, tagged with 'hello'. GMail will do this for you - try it. So that's why I think it's worth allowing comments. The next GMail might be able to do the same thing or something even more useful with comments: first.last(notify IM)@example.com Version 1.6 of my validator now passes all 222 unit tests. So does Cal's. I see no reason why you wouldn't use one of these in your project: they are free and they work. Why reinvent the wheel?

RFC nerd notes

Comments can contain folding white space and can be nested. This is the final nail in the coffin for regular expressions that claim to validate email address. Show me a regex that says this is a valid address: first(Welcome to  the ("wonderful" (!)) world  of email)@example.com A thank-you also to Paul Gregg who allowed me to add his validator to the head-to-head (and added mine to his page). He also provided some more unit tests.

Quick links: Source code | Email address validators head-to-head

light@the.end.of.the.tunnel

Quick links: Source code | Email address validators head-to-head

The amazingly helpful Cal Henderson has nearly caught up in the arms race that is email address validation. After our latest round of discussions we disagree about only one of the 161 test cases in the test suite. Cal's validator successfully validates all the test cases except that one (and he may be right about it -we'll see). I've released version 1.3 of my validator and you can download it with the test cases from Google Code.

RFC nerd notes

I had a lesson from Cal on Folding White Space. Who knew that you could have an email address that was split over several lines? Please don't run out and try this - it's strictly for completeness. Secondly, all those validators out there that use RFC2822 as their authority: those are yesterday's validators. All the cool kids are validating using RFC5322. It's the latest thing.

Quick links: Source code | Email address validators head-to-head

confused@rfc5322.com

Do you think this is a valid email address?

""@example.com

No, me neither. In one sense it clearly isn't since there is no mailbox of that name at example.com's mail host. Send mail to that address and you won't find anybody reading it. In another sense, however, it is a perfectly good email address. That is, if you believe the RFCs that define how email addresses should be constructed. Here's the BNF from RFC5322 for example:

addr-spec       =       local-part "@" domain

local-part      =       dot-atom / quoted-string / obs-local-part

quoted-string   =       [CFWS]
                        DQUOTE *([FWS] qcontent) [FWS] DQUOTE
                        [CFWS]

Don't worry if you can't follow the BNF syntax - I couldn't either until this week. Focus on the asterisk before ([FWS] qcontent). That asterisk means the contents of that bracket can occur zero or more times. So tracing it through the specification:

1. An email address is a local-part followed by an @ sign followed by a domain
2. A local-part can be a quoted-string
3. A quoted-string is a pair of double quotes surrounding zero or more characters.

""@example.com follows these rules perfectly. And yet common sense suggests this is a preposterous address. Where should the receiving host deliver it? I have had some discussions about this with Cal Henderson and we are somewhat at a loss. Both of us have functions that validate email addresses and we cannot decide whether to follow the RFCs or common sense. I could suggest the IETF add an erratum to their RFC but this format is also documented in RFC 5321 as well. No amount of published errata will correct the impression given by two published RFCs. Perhaps we are wrong - can you think of a valid purpose for an email address of this format?

Quick links: Source code | Email address validators head-to-head

Email address validation 1.2

Quick links: Source code | Email address validators head-to-head

I had some very helpful input from Cal Henderson on interpreting RFC 2822. I've included his latest validator in my head-to-head comparison and updated the test suite to reflect the consensus from our discussion. I've also updated my own routine so that it still passes all the tests, of which there are now 158. As always, the full analysis is on my website and the latest version of the source code will always be in Google Code. I would really appreciate it if you find a valid address that fails my validator, or an invalid one that passes, if you would get in touch with me and let me know. Here are the latest scores: Dominic Sayers: 100% validated correctly Simon Slick: 91% Cal Henderson: 84% Phil Haack: 80% Dave Child: 79% I know Cal is working on IPv6 validation so this may change soon.

RFC nerd notes

There are two principle changes in this version. Firstly, I now accept Dave Child and Cal Henderson's opinion that a backslash can escape anything in a quoted string local element. However, it must escape something - it can't be at the end of the string. Secondly, if the string is not quoted then the local element must be a vanilla atom. You can't escape individual characters unless the element is surrounded by double quotes. By "local element" I mean a single component of a dot-delimited local part. I've extracted the raw specification of a valid email address in BNF (Backus-Naur form) from RFC 5322 and put it here: The BNF from RFC 5322 defining parts of a valid email address. If you try to follow this in the RFC you end up going backwards and forwards and getting confused - it's much easier to understand in the way I've laid it out.

Quick links: Source code | Email address validators head-to-head

Email address validation 1.0

Quick links: Source code | Email address validators head-to-head

OK, after some correspondence with Dave Child and Cal Henderson I have now released version 1.0 of my PHP email address validator. This version passes all 139 unit tests in the suite - tests I have collated from RFC 3696 and from articles by Dave Child, Doug Lovell and Phil Haack plus some of my own to test the IPv6 address format. The full analysis is on my website and the latest version of the source code will always be in Google Code. I would really appreciate it if you find a valid address that fails my validator, or an invalid one that passes, if you would get in touch with me and let me know. Here are the scores on the doors: Dominic Sayers: 100% validated correctly Simon Slick: 86% Dave Child: 81% Phil Haack: 79% Cal Henderson: 71% For the RFC nerds, this version now correctly validates the obsolete local form of word *("." word). In other words you can regard the local part of the address as a dot-delimited series of elements each of which can be separately quoted. This means that first."last"@example.com is now recognised as a valid address.

Quick links: Source code | Email address validators head-to-head

Email validation head-to-head

Quick links: Source code | Email address validators head-to-head

I've discovered a number of other public-domain functions that attempt to validate the format of an email address. I tried to validate my 124 test cases against these functions with the results below: Dominic Sayers: 100% correctly validated Simon Slick: 88%  Dave Child: 80% Cal Henderson: 72% Interestingly, all the other functions failed correctly to validate the example addresses given in RFC3696 (Section 3: Restrictions on email addresses). I couldn't include Doug Lovell's function in my tests because the code is copyright All Rights Reserved by Linux Journal (ironically). Some of the other functions also supply test cases; I've added these to my test suite. More analysis and commentary here: RFC-compliant email address validator. In summary, I recommend you use my function if you want totally RFC-compliant address validation.

Quick links: Source code | Email address validators head-to-head

Alan Bennett's mobile phone number

I have Alan Bennett's mobile phone number and I'm happy to share it with you. Please feel free to call him and congratulate him on his wonderful work with Thora Hird and Dr Jonathan Miller.

Dear Friend,   I am contacting you in good faith with a proposal of mutual benefit.I am Mr. Alan Bennett, AUDITOR of GUARANTY PRIVATE CREDIT BANK(GPCB),London,United Kingdom.I discovered the sum of (£16,000,000.00) SIXTEEN MILLION POUNDS in a dormant account at our bank. We intend to credit this fund into any of your nominated bank account for investments.I will give you directives and all needed information as soon as I receive your positive response.Please get in touch with me by my alternative email: alanbennett101@live.com for better clarification.

 

Sincerely, Mr. Alan Bennett, Phone Numbers: +447024018524 or +447896149937