Hype is not unknown in the software field. The advocates of every new software idea exaggerate the benefits of using that idea. Those exaggerated claims generally have no basis in reality. In this week's column, Robert Glass explains his theory about Open Source Software.
Most of what I dislike about the open source movement can be summed up in one word: Hype. Unfortunately, and perhaps surprisingly, the advocates of open source are no better in this regard than their proprietary colleagues.
The claim is frequently made that open source programmers are the best programmers around. One author, apparently acting on input from open source zealots, said things like "Linux is the darling of talented programmers," and opined that the open source movement was "a fast-forward environment in which programming's best and brightest ... contribute the most innovative solutions" (Sanders 1998). Is there any truth to those claims? My answer is "No", for several reasons.
Attempts to define Programmer Aptitude Tests, which evaluate the capabilities of subjects to become good programmers, have historically been failures. In an early study, the correlation between computer science grades and practitioner achievement, was found to be negative. Although some programmers are better than others, nothing in the fields' research suggests that we have found an objective way of determining who those best people are.
Since we can't identify who the best people are, there is no way to study the likelihood of them being open source programmers. Thus, those who claim that open source people are software's "best and brightest" cannot possibly support those claims with any factual evidence. It is an interesting characteristic of programmers that most of them tend to believe that they are the best in the field. Certainly, I know that few programmers are better than me! It used to be a standard joke in the software field that, if a roomful of programmers were asked to self-rate themselves, none of them would end up in the second tier. Therefore, I suspect that if you took any group of programmers, including open source programmers, and asked them if they were the best and brightest, they would answer in the affirmative. To focus on open source quality claims, let's take a look at reliability and security.
The claim is also frequently made that open source software is the most reliable software available. In this case, there are some studies containing interesting data. The first thing that should be said about open source reliability is that its advocates claim that a study identified as the "Fuzz Papers" (The Fuzz Papers) produced results that showed that their software was more reliable than proprietary alternatives.
I obtained the papers, read and analyzed them, and contacted their author to investigate the matter even further. The bottom line is, the Fuzz Papers have virtually nothing to say about open source software, one way or the other, and their author agrees with that assessment. He does say, however, that he personally believes that open source may well be more reliable. It is truly bizarre that anyone would claim that these peculiar studies of software reliability actually support the notion that open source code is reliable. To understand why I say "peculiar" you should read them yourself!
Analogous to the reliability claims, there are many claims that open source is more secure. The more that public and industry concern for security increases, the louder those claims become.
There is very little evidence on either side of the ledger regarding open source software and security. Certainly security holes have been found in proprietary software. Certainly also, holes have been found in open source code (for example see SP 2002). And both sides have made strong claims that their software is either the most secure, or that they are making it so.
Probably the most accurate statement one