Open Source, Proprietary Code Quality Comparable

Commercial vendors sometimes take pot-shots at open-source software projects because they think the code is weaker, or hasn’t been validated by professional testers. But a new study by an inspection firm has found that the latest version of Apache Web server is comparable in quality to its commercial brethren.

Mountain View, Calif.’s Reasoning, which charts the degree of flaws in such programming languages as Java, C++ and C, found 31 software defects in 58,944 lines of Apache Web server V 2.1 code.

Using the industry-accepted reliability indicator called defect density, which is the number of defects found per thousand lines of source code, the group found a defect density of the Apache code of 0.53 per thousand lines of source code. Meanwhile, the average defect density of commercial code was 0.51 per thousand lines of source code.

Measuring those findings against the findings in a similar test from February, which found that the TCP/IP protocol stack implementation in version 2.4.19 of the open source Linux kernel has fewer defects than the TCP/IP protocol stacks of several commercial equivalents, the testing company concluded that maturing open-source software can be as high in quality as commercial vendors’ software products.

The findings could leave folks to draw their own conclusions, but for those advocates of commercial products, it could poke holes into their oft-floated assertions that software that is sold is of higher quality — an argument leading vendors have made when trying to discredit open-source operating systems like Linux, or open-source databases such as MySQL.

Bill Payne, President & CEO of Reasoning, said that February study, which concluded that open source had a significantly lower defect density compared to commercial equivalents, led for developers to call for another similar test, albeit one in which the open-source application was less mature.

“We received numerous inquiries about that study and took seriously requests for us to examine defect density rates in a less mature Open Source application and compare it with the commercial equivalent,” Payne said. “Taking advantage of our database of automated software code inspection projects, we were able to do exactly that, and found the difference in defect density between the two was not significant.”

Reasoning’s inspection service is based on a combination of proprietary technology and repeatable process. The company pledges objective results that are comparable across software applications, development methodologies, and coding styles.