Controversial Report Finds Windows More Secure Than Linux

Contrary to popular wisdom, Windows appears to be more secure than a popular version of Linux, according to an upcoming report from two security researchers.

The researchers found that Windows Server 2003 actually had fewer security vulnerabilities identified last year than Linux and that the holes in Windows took less time to patch.

But the study is already attracting controversy for its methodology. Linux proponents note that the two systems have different configurations and are not easily comparable since they contain different functionality out of the box.

"A lot of people are under the impression that one platform has more advantages," said Max Clark, a network consultant with Intercore, a Los Angeles-based consulting firm that provides support for both Windows and Linux systems. "The expertise of the person deploying it is what matters. The default configurations are important, but once you start consolidating software on top of the system, the system is only as secure as what's running on it."

The study, which compared Windows Server 2003 to Red Hat Enterprise Linux ES3, was conducted by Richard Ford, a research professor in the computer sciences department at the Florida Institute of Technology's College of Engineering, and Herbert Thompson, director of research and training at Security Innovation, a security technology provider.Linux advocates criticized the study over allegations that the researchers accepted funding from Microsoft, a criticism also leveled at earlier studies finding Windows security superior to Linux.

The researchers declined to comment on whether Microsoft is funding the current study, saying they will disclose funding sources when the study is published finally. They defended the study, saying they are interested in hearing feedback from others willing to test their research findings to see if they are sound.

They Surprised Themselves

When researchers previewed the study at the RSA Conference in February, Ford told attendees he was a "Linux fan," according to accounts in the Seattle Times and VNUnet. He runs Linux and other open source software in his home.

Ford and Thompson said they were surprised by some of their results.They examined typical Web server configurations, comparing a Windows Server 2003 system running Internet Information Server 6.0, SQL Server 2000 SP3 for Windows, and ASP.NET scripting against an open source system running Red Hat Enterprise Linux ES3, Apache web server with OpenSSL and OpenSSH, MySQL database, and PHP scripting.

For Red Hat, Thompson and Ford looked at both a default configuration as well as a minimal configuration with only the components essential to act as a Web server.

For Windows Server installed with all of its components, the researchers found 52 vulnerabilities that were fixed in calendar year 2004.

For Red Hat, in the minimal case, they identified 132 vulnerabilities fixed in 2004, and in the default configuration, they found 174.

They also looked at the time between when a vulnerability was publicly disclosed and when a patch was issued, which they referred to as the "days of risk." With Windows Server, they found there were 30 days of risk, but with Red Hat Linux there were 71."In the minimal stripped down case, the gap between the two was surprising," Thompson said. "With Microsoft's adoption of their secure development lifecycle, I believed that Windows would probably beat the default installation, but I did not believe it would beat the minimal installation."

Earlier Studies Agree

This is just the latest in a series of controversial studies that found Microsoft software more secure than Linux and other open source software. Last year, Forrester Research conducted a study where it also looked at days of risk and number of vulnerabilities. Forrester concluded that both Windows and four of the most popular Linux distributions could be deployed securely and that Microsoft had the lowest average total days of risk.

However, several Linux vendors took exception to the report's methodology, and recalled that Microsoft had commissioned an earlier report in 2003 from Forrester on the total cost of developing and deploying Web-based portal applications on Microsoft vs. Linux platforms. Although Microsoft did not fund the 2004 Forrester security report, critics claimed the earlier funding was evidence of bias.

The new study is receiving similar accusations. Messages on sites such as Slashdot pointed to Microsoft funding for other Florida Institute of Technology research projects.Security Innovations does not disclose details regarding relationships with clients, but Microsoft, HP, IBM, SAP, and Cisco have been previous clients, Thompson said. He declined to state whether Microsoft or Red Hat provided any funding for the new study, but said this information will be disclosed with the release of the final report. A full description of the methodology will be released so other researchers can scrutinize and try to repeat it, followed by the full report with the disclosure of the funding.Open source advocates say the pro-Microsoft studies are suspect because they don't take into account the severity of the vulnerabilities or the different ways that vulnerabilities are reported and dealt with in the Linux community.

Mark Cox, head of the security response team at Red Hat, pointed to flaws with a recent comparison by Microsoft chief security executive Mike Nash in an online chat session last month. Nash claimed that 34 vulnerabilities had been found in Red Hat Enterprise Linux so far in 2005, but only 15 in Windows Server 2003.

"He was implying there were twice as many vulnerabilities in Enterprise Linux," says Cox. "Three of the Microsoft vulnerabilities were critical flaws, something that can be exploited without user interaction, like worms. Of the 34 vulnerabilities in Enterprise Linux, none of them were critical. The metrics are quite useless unless you take into account the relative severity of the issues."

Microsoft did not cooperate in requests for comment on this story..

Red Hat said Linux programmers address flaws relatively quickly. "The problem is these metrics only look at days of risk when the vulnerability is published and then when it's fixed, not the date when the bad guys found it and not the date when it was exploited," Cox said.Thompson responded that the Linux community's quickness to disclose vulnerabilities to the public makes the operating system less secure. "Fewer people have historically followed responsible disclosure on the open source side," Thompson said. "With responsible disclosure, if someone finds a vulnerability in an application or the operating system, they report it directly to the vendor or the package owner. In the open source case, there are more people who disclose the flaw publicly in bug lists. Whereas in the Microsoft case, historically more people have followed responsible disclosure, and then Microsoft discloses the vulnerability and releases a patch for it."

Thompson said he hopes to continue the study on an ongoing basis, and to receive feedback from other security researchers as well as encourage independent testing by others.

"We want people to bake the cake themselves," he says. "We want to give people the recipe so they can see the numbers for themselves and get back to us."

Size Matters

Linux can appear to be less secure because the distributions are larger than Windows, said Novell. The company's SUSE Linux has 2,600 packages, far greater than Windows; the number of packages would have to be reduced significantly to make the software comparable, said Novell security architect Roman Drahtmueller.In addition, the Linux distributors also supply the source code for every fix, so various users can check the code themselves, recompile it, and make sure there are no hidden back doors. "Customers can make sure that we have supported nothing but the fixes," Drahtmueller said. "The only way to prove we haven't put in a back door is because of the transparency we have on the package, and the way we publish the work to our customers."

The Open Source Development Labs criticized the study because it only looks at one flavor of Linux. "This is comparing Microsoft as shipped to a single vendor of Linux solutions," says Bill Weinberg, open source architecture specialist with the organization, which oversees Linux development. "It's not representative of the entire market."

Linux has several add-ons that can substantially increase the security of the system, such as Security-Enhanced Linux (SELinux), which was developed by the National Security Agency, and grsecurity, a set of kernel patches that prevent many buffer overflow issues, said Johannes Ullrich, chief technology officer for the SANS Internet Storm Center, a service that issues advisories of security vulnerabilities and hacking exploits. Skilled administrators who know how to apply these add-ons can effectively protect their systems and are better off than if they switch horses in midstream.

"Whenever a study comes out that says operating system A is better than operating system B," he said, "you have people switching from one operating system to another, even if they know the first operating system better, and you end up with a much less secure operating system."

Michael Cohn is a freelance journalist.)0