E-mail Authentication Techniques

The latest figures from Symantec's Security Threat Report and Forrester Research peg spam at 67 percent of all Internet e-mail traffic. In other words, of every 10 messages that come into your network, only about three deserve your attention.

The most current widespread approach to combating spam is to use software that attempts to determine, for each user, what is spam and what is not. This filtering software certainly helps, but we have yet to see a product work with 100 percent accuracy; mistaking legitimate e-mail as spam is an unacceptable byproduct. Can e-mail sender-authentication technologies join with filtering software, adding another layer of security to thwart spammers? We decided to find out.

Inside the Spammer's Mind

E-mail and SMTP weren't designed with security in mind. Consider the resemblance between e-mail and snail mail: When sending a handwritten letter to a friend in California, there are no restrictions that force you to place the correct mailing or return address on the envelope, drop it off at a specific post office, or even place the right letter inside the envelope. Likewise, e-mail systems don't force senders to address their e-mail accurately, place the correct return e-mail address, use a specific e-mail server when sending messages, or type the correct text in the body of the message. Spammers capitalize on these limita- tions by, for example, never using a correct return address. They maintain lists with hundreds of thousands of e-mail addresses, take a scattershot approach and hope a fair percentage get delivered. Obviously, they're all too successful.

Spammers love open relays--mail servers that deliver mail for nonlocal users--because an open relay helps mask the spammer's identity. More and more, responsible mail admins are shutting off open relays on their systems. Services to track open relays, called real-time blacklists, combined with e-mail server software that blocks e-mail from these sources, have made a dent in the number of open relays, forcing spammers to find other methods.

Happily for them, the rise in broadband adoption and lax endpoint security have given spammers access to hundreds of thousands of always-on, high-speed-connected computers. Once a vulnerability is exploited, usually through home PCs infected with a Trojan or virus, spammers install e-mail-relaying software, effectively turning innocent home computers into open-relay e-mail servers. To combat this, ISPs began blacklisting IP addresses from other ISP DHCP pools, because ISPs should relay mail using only their outbound SMTP servers. But then spammers enhanced their e-mail-relaying software to submit the e-mail to outbound SMTP servers instead, knowing that other ISPs can't block those servers without breaking e-mail altogether.

Spammers also randomize their messages, changing words and phrases to get past content filters. They commonly add content from well-known published works, for instance, to fool filters. Phishing attacks are a huge, spam-borne security threat and contribute to ID theft.Authenticate Yourself

E-mail sender authentication technologies aim to fix some of these exposures. A variety of draft implementations have been submitted to the Internet Engineering Task Force for consideration. SPF (Sender Policy Framework), Sender ID and DKIM (Domain Keys Identified Mail) are the leading contenders. E-mail appliance vendor IronPort released information on another e-mail authentication technique, SMTPi, in 2003; the only place you can find any information on it now, though, is through Web archive search engines.

Sender authentication won't eliminate spam--that isn't what it's designed to do. If the entire Internet e-mail community adopted e-mail sender authentication, we would see a drastic reduction in spam, but there's nothing to prevent spammers from joining the sender-authentication camps. Still, the success of simple techniques, like faking "from" addresses and using infected broadband computers, would decline drastically.

In addition, reputation services, including Return Path's Bonded Sender Program and Habeas' eponymous service, profile each e-mail domain's likelihood of distributing spam. Once sender authentication becomes more widespread, these services will determine a domain's reputation more accurately, letting your filtering software take more educated guesses on messages' "spamminess."

What's SPF? (No, Not Sunscreen)SPF records, specially crafted DNS TXT records, tell other e-mail servers which mail servers you allow outbound e-mail from, somewhat like "reverse MX" records. SPF records also contain a confidence level that helps the receiving e-mail system determine authenticity.

It won't cost you anything to let receiving e-mail servers act on your SPF records. However, enabling your e-mail servers to read and take action on others' SPF records requires modifications to your e-mail software. SPF support exists for many e-mail security front ends, antispam devices and common MTA (message transfer agent) software, all of which will require some tweaks to get SPF working. If your SMTP gateway doesn't support SPF, it's time to go shopping: Twenty-five percent to 30 percent of all e-mail domains publish SPF records, according to Forrester, making it the most widely adopted sender-authentication technique.

SPF validates that the envelope sender, the "from" address in the SMTP protocol, is indeed allowed to send e-mail from the sending server. Outbound mail servers for AOL, for example, should never be sending messages with Hotmail envelope sender addresses. When an SPF-enabled receiving e-mail server gets such a message, it looks up the SPF record for Hotmail and determines that the sending AOL server isn't allowed to send e-mail on behalf of Hotmail. Using the confidence level Hotmail DNS servers return with the SPF query, the receiving e-mail system can take appropriate action.

All SPF records start with v=spf1, indicating an SPF record. Immediately following the initialization string is a list of authorized sending e-mail servers. SPF supports keywords like ptr, which define all hosts that end in the domain name being checked, and mx, which defines all MX hosts (a list of valid notations is at www.open spf.org/mechanisms.html). These keywords assist admins by adding hosts that match those keywords automatically.

The last piece of the SPF entry defines a confidence level--fail, softfail, pass or neutral--giving the receiving e-mail server valuable information to make a decision.SPF records are being published by e-mail providers including AOL, Hotmail and RoadRunner, as well as large companies like Bank of America, eBay and Ticketmaster. Ticketmaster returns a fail confidence level, AOL returns a neutral rating, and all others return softfail confidence level to anyone checking their respective SPF records.

Anyone can look up SPF records using DNS querying tools and specifying the resource record type as TXT. Bank of America's SPF record looks like this:

The advantages of SPF are clear: Simply add DNS entries so others can validate SPF, possibly enhance your MTA with additional SPF-aware software, and you can use others' SPF records to defeat the common spamming techniques of sending e-mail with incorrect envelope return addresses from incorrect SMTP servers.

Although we encourage everyone to configure SPF entries for their domains--there's nothing to lose, after all--the system does have its limitations. Most important, it doesn't stop enough phishing e-mail because it's checking the envelope return address and not the "from" address that users see in their e-mail clients. This is by design; spammers could bypass SPF with phishing attacks using the correct envelope "from" address and outbound e-mail servers but trick users with faulty content and "from" addresses seen in their mail clients.

SPF also poses problems for mobile users who change their outbound SMTP servers based on the ISP they're using. The envelope sender won't match their ISP's SPF records, and their e-mail will be blocked. You can avoid this by implementing SMTP AUTH--a good idea anyway--so users can authenticate with their user name and password to outbound SMTP servers. SPF also poses problems to forwarding-e-mail services, like alumni accounts from universities. These providers must rewrite the envelope sender to be effective, a technique called remailing.

» Click here for a wizard to help your organization create SPF records. More than 1.7 million domains had published SPF records as of November 2005. The IETF draft is here .

Sender IDSender ID, a backward-compatible extension to SPF, is a sender-authentication scheme backed by Microsoft and in use for Hotmail and MSN services. These services display Sender ID results in their Web mail interfaces. Sender ID, formally named Caller ID, makes use of the same DNS TXT resource records that SPF requires. The main difference between SPF and Sender ID is that the Sender ID spec checks the "from" address contained in the body, instead of the one presented to the SMTP layer.

Sender ID is better than SPF at preventing phishing attacks because it checks the "from" address users see in their e-mail clients. And, like SPF, it won't cost you anything to get started because you simply add DNS records to let others act on your records. However, like SPF, Sender ID will break forwarding services and mobile users who employ different SMTP servers.

The biggest concern we have with Sender ID revolves around Microsoft's Royalty-Free Sender ID Patent License Agreement. Organizations like the Apache Software Foundation, AOL, the Free Software Foundation (FSF) and Open Source Initiative (OSI) have expressed their discontent with the licensing language to the IETF; this appears to have slowed Sender ID's momentum. These groups believe that no one company should be given intellectual property rights over such a core Internet specification. Microsoft and open-source providers seem stalled over "nontransferable, non-sublicenseable" terms and language that requires competitors to contact Microsoft when rebranding or redistributing Sender ID, giving the software giant valuable insight into its competitors' business intentions.

Our recommendation is to implement SPF and wait for the patent license disagreements to be resolved. Both SPF and Sender ID are in IETF draft states, but SPF seems to have more momentum, so our bets are on SPF to become an IETF standard first.

» Click here for an overview of the Sender ID technology. IETF drafts are here, and here.And How About DKIM?

Yahoo's Domain Keys merged with a similar spec developed by Cisco, called Internet Identified Mail, and is now referred to as DKIM. The two companies announced the joint effort in mid-2005, submitting updated documentation to the IETF backed by at least a dozen major companies. The DKIM sender-authentication approach is to sign outbound messages with a private key. Receiving e-mail servers can validate that the e-mail originated from a known source using public keys available in DNS, and they also can validate the integrity of the message--a capability SPF and Sender ID lack. The DKIM specification is not yet a standard.

Setting up a DKIM-aware e-mail infrastructure is more involved than implementing SPF and Sender ID. Because outbound e-mail must be digitally signed, your outbound server e-mail software must be modified to let it compute and modify the message. As with SPF and Sender ID, if you want to validate DKIM messages, receiving e-mail systems must be modified as well. Implementations are available for commonly used MTAs (Message Transfer Agents). To play with DKIM, you must upgrade your e-mail server software.

To enable signing of outbound e-mail messages, DKIM-aware outbound e-mail software must be installed--your current software may support it with some modifications. It also requires that a public-private key pair be generated so e-mail messages can be digitally signed. The public key is entered into a TXT resource record for the policy under the subdomain domainkey. Yahoo uses a policy called s1024 for signing e-mail messages, so its public key can be found at s1024._ domainkey.yahoomail.com. The private key is loaded into the DKIM-aware e-mail software and associated with a policy, so it can be used to digitally sign e-mail.

When a user sends an e-mail using a DKIM-aware outbound mail server, the server signs the message using particular headers and the body text of the message. The digital signature, along with information describing what headers and encryption algorithm were used, is inserted as a header into the e-mail message. A DKIM e-mail header looks like this:

Once the message reaches a receiving e-mail system that is DKIM-aware, it can use this header to validate the message. Using the information in the DomainKey-Signature, the receiving server retrieves the public key from DNS. The receiving server then retrieves the TXT resource records from DNS for s1024._domainkey.yahoo.com.

Using the public key found in DNS, correct algorithm (a=) and particular headers (h=) found in the e-mail message, the receiving system can generate and compare digital signatures. This lets the receiving system validate that the e-mail was indeed sent, and allowed to be sent, from the sending system. If the signature is validate, the receiving e-mail system can deliver the message to the user, or it can drop, reject or quarantine the mail if the signatures do not match.

To prevent spam and phishing attacks, receiving e-mail servers can create policies that disallow non-DKIM e-mail from e-mail providers known to use DKIM. For example, if an e-mail arrives without a DKIM header claiming to be from Yahoo, which employs DKIM technology, the message can be safely rejected or deleted. Major e-mail providers including Yahoo and Google use DKIM technology.

The main disadvantage of DKIM is that it requires both outbound and inbound DKIM-aware e-mail servers. DKIM inherently "fixes" the forwarding problem, found in SPF and Sender ID, that lets spammers continue to propagate unwanted e-mail. However, a spammer can send one e-mail through a DKIM-aware server and then forward or replay that message to other users. It will appear to have a valid signature but arrive from a questionable source, meaning further enhancements to inbound e-mail servers are needed to detect this forgery.

DKIM will also add overhead to e-mail processing, require more DNS lookups, and reduce the amount of e-mail your servers can process due to the significant processing resources needed to validating signatures.» An explanation of DKIM. IETF drafts are here , here, and here.

Christopher T. Beers is a Network Computing contributing editor and manager of systems operations for a large broadband ISP, where he oversees daily operations of high-speed data and VoIP for the Northeast U.S., including Solaris and Linux administration. Write to him at [email protected].

Read more about e-mail authentication and find messaging-related news and reviews on our Messaging and Collaboration channel.

Executive Summary

Ladies and gentlemen, place your bets: SPF (Sender Policy Framework) is winning the early battle between three sender authentication methods, and we expect it to retain that early lead over the next year or so. The specification's ease of deployment will let everyone publish SPF records quickly, and its record checking is supported by just about every e-mail server vendor.Likewise, Sender ID is easy to implement and provision on both the sending and receiving sides. But it's surrounded by licensing controversy that's slowing its uptake by other vendors.

Finally, DKIM (Domain Keys Identified Mail) has value, but its complex configuration requires significant changes to both inbound and outbound e-mail software and imposes a serious performance penalty on DNS traffic between sites.

No matter which authentication method you choose, your users and the Internet community at large will come out ahead. Anything that trips up spammers is a sure winner in our book.