FUD

I usually provide a link to this page in response to a certain kind of a FUD attack, or when I strongly suspect that someone's merely repeating the same FUD without being aware of its historical background. Rather than having to rebut the same, worn-out FUD every time, I now refer people to this web page. My spirited detractors always seem to forget to mention a few details, before condemning me to the tenth level of Hades..

I think that people need to be aware of the following history before forming any conclusions. All of the following historical timeline was pulled straight out of Google archives. Google files from that time period are somewhat sketchy, but still contain enough detail to piece together most of the key events.

Chapter I

In May 1992 Dan Bernstein suggested (original link) using RFC 931 to defeat certain classes of forged mail headers. Mark Crispin objected on several technical grounds. In general, he argued that RFC 931 was fundamentally flawed; it's unfixable; and that it should simply be scrapped. Bernstein's position was that the RFC 931-based solution may not be 100% ideal, but it is still useful in the majority of situations (original link). It's “good enough”, in other words.

Crispin's subsequent posts became increasingly heated (original link); finally, he proclaimed that “Bernstein is still clueless” (original link), because “RFC-931 is worse that security by obscurity” (original link).

Well, as it turned out, Bernstein eventually won this argument, even though working in Crispin's favor (and supporting his position) were certain other technical problems with the RFC 931 document. But, as history shows, eventually RFC 931 was revised and updated to become RFC 1413. The revised document gives an acknowledgement to Bernstein. Mark Crispin's objections were not mentioned anywhere.

Chapter II

Crispin went on to work on the UW-IMAP server, and the IMAP protocol. Bernstein went on to write the Qmail server. Qmail introduced a new filing method for storing E-mail, maildirs, on which the Courier server is based. Maildirs addressed several long-standing shortcomings of the traditional Bezerkeley mbox mail format (the default mail format used by the UW-IMAP server) that have been known for some time.

Between 1995 and 1999 Qmail gained popularity until it became the second most popular mail server on the Internet. With Qmail's popularity growing, people began asking Crispin about adding support for Qmail's maildirs to the UW-IMAP server. Crispin, still simmering over losing the flame war over RFC 931, flogged this opportunity for all it was worth. He seemed to relish refusing every such request, explaining that “Maildir is a support and performance nightmare” (original link). He repeatedly refused to add maildir support to the UW-IMAP server “because Maildir has dreadful performance (worse than any supported mailbox format) and doesn't scale, that's why” (original link).

Although he was always quick in gratuitously declining all requests to add maildir support to the UW-IMAP server, he wasn't too forthcoming in explaining his reasoning why maildirs were so supposedly inefficient in an IMAP context. Sometimes he went as far as mentioning some filesystem-specific factors, but that was far from a complete explanation. Furthermore, he never told anyone about his old flame war with Dan Bernstein.

Chapter III

This remained the status quo until early 2000.

I finished writing the initial implementation of the Courier server in 1999. After the first few public releases, many people quickly discovered this fast, small, IMAP server, which simply defied the notion that it was not possible to have an efficient IMAP server based on maildirs.

People stopped pestering Crispin about adding maildir support to the UW-IMAP server, because a viable, working alternative was now available. Suddenly, Crispin was no longer able to get his jollies by badmouthing maildirs (and taking indirect swipes at Qmail and Bernstein). That, obviously, didn't sit well.

Publicly, he went on the offensive, claiming that there was something improper, or non-compliant, about the IMAP implementation in Courier (but, like always, short on details). An ominous warning came out: “beware of Courier-IMAP” (original link). That public service announcement was justified because he, Mark Crispin, had a higher calling in life: “I have an obligation to report non-compliant servers and defiant vendors who refuse to implement the specification.” (original link)

Epilogue

And that's how this FUD started. My response to claims that Courier-IMAP is not a compliant IMAP server is very simple: there was no such thing as a “compliant” IMAP server, because Crispin's IMAP specification makes that an impossible goal. I reached that conclusion while developing Courier-IMAP based on RFC 2060. I found that RFC 2060 was, in many instances, confusing, contradictory, and unclear. I said so publicly, which obviously ruffled some feathers.

But he really had no choice. In 2003, Appendix B of RFC 3501 identified over a hundred corrections to RFC 2060. Observe that RFC 3501 does not define a new version of the IMAP protocol. It's the same version. This is a “revised” document, which attempts to clarify and address numerous issues and inaccuracies in the original specification (some of which I noted publicly).

And, to make things even more interesting, as of this date there are already several reported errata to 3501. Furthermore, RFC 3501 actually changed the IMAP protocol, adding several new requirements, but it kept the IMAP4rev1 version. In other words: the protocol has changed, but it's still the same protocol, officially. Figure that one out.

So, how is it possible to define what exactly is a “compliant” IMAP server, when the IMAP specification is such a moving target? I am unable to come up with any other Internet protocol:

Some IMAP clients certainly do have problems with the Courier-IMAP server, from time to time. Crispin always blames Courier-IMAP. But, it's just as true that some clients will also have a problem with the UW-IMAP server. It won't take a long time to find published reports of interoperability issues between the UW-IMAP server and some mail clients. My position is that the fault for this lies squarely on the specification. You can take SMTP, or POP3, or IRC, or most any other wire protocol; write code according to the specification; and have a fairly good chance of having your end result work, more or less, with other software. The same cannot be said for IMAP. To have any reasonable chance of coming up with a working client or server, it is always necessary to use a working counterpart, and examine its input or output in order to figure out how IMAP's supposed to work.

Full speed ahead!

The party line these days is that there are only two “real” IMAP servers out there: UW-IMAP, and Cyrus (original link). Sometimes the trolling is so obvious (original link), that it's sad, more than anything else.

An unfortunate announcement went out in 2002: another Internet provider now switched from UW-IMAP to Courier (original link). This one was certainly a mortal insult. It's quite obvious to see how some cookies got frosted over this. I also learned (via other channels) of a few lame attempts lobby this rogue Internet provider into changing their mind. Alas, that didn't pan out (AFAIK as of this date).

I note that the original party line was that the UW-IMAP server is the only well-written IMAP server in world (and the only well-written IMAP mail reader is pine). Recently (and probably because of the recent developments with this ISP), both UW-IMAP and Cyrus are now officially blessed; the apparent goal is a tactical decision to steer people away to these two servers, and away from the uppity Courier.

Other things to keep in mind

A few more random issues sometimes come up:


Sam Varshavchik
March 16, 2008