The Trouble with Rootkits

Similarly, IE7 in Windows Vista operates by default in protected mode. In this mode, IE7 cannot modify user or system files and settings without user consent, helping shield users from compromised websites.

For workstations running the 64-bit version of Vista, Kernel Patch Protection (KPP) is included. “KPP is designed to prevent attackers from modifying or extending the kernel, through techniques such as rootkits,” Toulouse said.

It also prevents changes through undocumented, non-supported methods, further hardening the kernel. Future versions of Microsoft’s Longhorn server software will include similar features, and both versions will emphasize driver signing as well.

Legitimate Rootkits

Rootkits are nothing new, but since malware detection has become standard, given away for free on new PCs or with broadband subscriptions, attackers have placed a greater emphasis on being stealthy.

Part of the trouble with rootkits is similar techniques are used for legitimate purposes. Antivirus vendors, such as Symantec and Kaspersky, have used rootkit techniques and both have been criticized for doing so, although their intentions were legitimate. The intended to hide important security files, so users wouldn’t accidentally delete them.

Then there is the case of the Sony BMG rootkit that was distributed on music CDs. According to Gartner’s MacDonald, there’s a big difference between what Sony and the security vendors did.

“Sony used rootkit technology to hide their DRM software from tampering, whereas security vendors hid processes for legitimate reasons,” he said. Sony’s rootkit also contained a spyware component that tracked user behavior.

Users never opted in to having rigid DRM protection, nor were they aware the Sony rootkit created a gaping hole in their security profile. The US Federal Trade Commission (FTC) sued Sony over its rootkit, and Sony ended up settling; agreeing to pay customers who bought rootkit-invested CDs up to $150.

The settlement may not be the end of the story, though. It didn’t take long for hackers to develop worms targeting the vulnerability created by the Sony rootkit, and it shows how even legitimate vendors that end users have no reason not to trust can compromise security.

“Technology is highly integrated. From operating system integrations to smartphones to networked ATMs, devices are interacting with each other on a larger scale than ever before,” said Philippe Honigman, COO of SkyRecon Systems, a security company that is currently developing anti-rootkit technologies. “SASSER, for instance, thrived because of interoperability. The treat for major havoc is alive and well because of our complex and integrated networks.”

Another troubling thing about rootkits is how difficult they are to remove. In the past, viruses and worms caused great trouble, but most were easily removed once antivirus vendors caught up with them. With rootkits being so tightly tied to low-level OS processes, many security experts argue that removal means starting from scratch. You have to reformat your drive to ensure that sophisticated rootkits is gone.

Some vendors advocate simply renaming and quarantining files to effectively disable them, but it’s debatable as to whether that approach is viable. Users could accidentally rename the wrong file or miss other files that allow the rootkit to persist.

Obviously, this places a premium on not being infected in the first place.

“In an enterprise setting, avoiding infection means that the enterprise must be in control of what’s being installed on company PCs,” Honigman said. “Users should never be given administrator status, which is easier said than done since most organizations give employees control over their own settings and downloads, exposing them to many threats.”

Vendor’s Options

Rootkit detectors have hit the market, though they are in the early stages of development. SkyRecon Systems will launch a rootkit detector specifically designed for Vista later in the year, while detectors are also offered by F-Secure, Sophos, and Microsoft, among others.

The rootkit threat again points to the need for layered security.

“Put in good end-point security beyond signature-based security solutions and firewalls, and stay current with patches. A personal firewall is not enough anymore since it cannot control what users do. Network access control is critical, as well, but its best left to the pros,” Honigman said.