Email
Authentication
David MacQuigg
updated March 12, 2006 {1}
Ensuring a valid identity on an email has become a vital first step
in stopping spam, forgery, fraud, and even more serious crimes. Unfortunately,
the Simple Mail Transfer Protocol (SMTP)
that handles most email today was designed in an era when users of the Internet
were mostly honest techies who expected others to be equally honest. This
article will explain how email identities are forged and the steps that are
being taken to prevent it.
Mail Transfer Chaos
In a simple mail transfer, there
are four key players: the author or originator of the email, the sender
or agent who first puts the email on the public Internet, the receiver
or agent who gets the email from the Internet, and the recipient who is
the person supposed to read the email.[1]
When we say Internet, with a capital I, we mean the world-wide network
that shares a common set of IP addresses, not the internal networks before the
sender or after the receiver. For example, the computer I am writing this
article on shares a local network with other computers having addresses I can
assign at will. My network connects via a router to the
network of my Internet Service
Provider ( ISP ), and he can assign whatever addresses he wants
within his network, including the address of my router. It is only when he
connects his router to the Internet that a real Internet IP address is needed.
Other than the sender's IP
address, there is no verification of any information in an email. It is quite
easy for a spammer to make an exact copy of an email from smithbarney.com,
including a long complicated sequence of headers and a genuine logo in the body
of an email, then change the content to send readers to a website that appears
to be genuine, but is actually a phishing
scam designed to capture names, passwords, and credit card numbers.
So why can't the sender's IP
address be used to identify the spammer? There are two problems. One is that
spammers often work through forwarders to hide their IP addresses (see below).
Another is that the sender is often a zombie
that has been infected by a computer
virus, and is programmed to send spam without the owner even knowing about it.
There are millions of insecure home computers, and they have now ( March 2005 )
become the major source of spam.
Attempts to stop spam by blacklisting
sender's IP addresses have failed to reverse the worldwide growth. Most IP
addresses are dynamic, i.e. they are frequently changing. An ISP, or any
organization directly connected to the Internet, gets a block of real Internet
addresses when they sign up with an Internet Registry.
Within that block, they assign individual addresses to customers as needed. A
dial-up customer may get a new IP address each time they connect. By the time
that address appears on blacklists all over the world, the spammer will have
new addresses for the next run. There are 4 billion possible IP addresses on
the Internet. The game of blocking these rapidly changing IP addresses has been
facetiously called "whack-a-mole".
There are a number of things that
ISPs have done to stop zombies and deliberate spamming by their customers.
Infected computers can be cleared of viruses and patched to resist further
infection. Outgoing email can be monitored for any sudden increase in flow or
in content that is typical of spam. Some ISPs have been quite successful {2},
but others don't care to make the effort. With spam now two thirds of all email
traffic {3}, there will always be ISPs willing to provide services for
spammers.
Authenticating Senders
Email authentication greatly simplifies
and automates the process of identifying senders. By quickly verifying a
claimed domain name, it is possible to triage the incoming flood of
mail. Forgeries and known spamming
domains can be rejected at the connection level, without wasting any time on
data transfer, or even testing a long list of possible recipient names from the
spammer's dictionary. Reputable senders
can be given a pass for an entire session, allowing them to bypass the IP blacklists
and statistical filters that always lose some valid messages. The remaining
flow can be treated the same as we now treat all email - rigorous filtering,
return challenges to the sender, etc. Successful authentication, coupled with a
domain-rating system, will reward reputable senders and encourage others to
clean up their outgoing mail.
There are a number of methods to
authenticate a sender's domain name ( SPF
[2], SenderID [3], CSV [4] , DomainKeys [5], and
others). All are very effective in
stopping the kind of forgery now prevalent.
None exclude the use of other methods, although there is a lot of
overlap in basic function, and some incompatibilites. There are small vulnerabilites in each
method, and it may be that a combination of two will be required to cover all
the cracks. The most widely used will
likely be the ones that require the least effort on the part of senders who are
reluctant to assume any responsibility for operating public mail servers.
CSV, SPF, and SenderID
authenticate just a domain name. DomainKeys uses a Digital Signature to authenticate domain names
and the entire content of a message. CSV and SPF can reject a forgery before
any data transfer. SenderID must see at least the headers, and DomainKeys must
transfer the entire message. CSV is the
quickest. DomainKeys is the most
thorough. CSV checks only the HELO name
at the start of each SMTP session. SPF
checks the return address on each message "envelope". SenderID checks the From address in the
headers of each message. Domainkeys can
detect any alteration in the headers or body of a message.
CSV, SPF, and SenderID work by
checking the IP address of the actual sender {4} against a list of addresses
authorized by the alleged sender. If the
sender says "HELO this is aol.com sending to you from address
192.168.72.19", the receiver can query AOL's records in the Domain Name System (
DNS ), and see if that is indeed an address authorized to send mail on behalf
of AOL. So far, it looks like DNS is
secure {5}.
DomainKeys also uses DNS to
retrieve secure information from the alleged sender, but instead of a list of
authorized addresses, the sender provides a public key for his
domain. This key can be used to verify
the signature on the message, independent of any IP address. Freedom from IP addressing means the message
can go by any route, including through a forwarder.
The use of forwarders is common
for small domains which prefer not to manage their own mail server, and for individual
recipients, who prefer to keep their personal address when they change jobs or
ISPs. SPF and SenderID can also work
with forwarders, but the extra steps add complexity and some vulnerability to
the system (see below). CSV limits its
focus to one-hop authentications, and assumes a signature method will be used
for end-to-end authentication.
Use of the DNS database to
register authentication information for a domain is relatively new. The new
information is added to existing DNS records, and queries for this information
are handled the same way as any other DNS query. Publishing authentication
records in DNS is voluntary, and many domains probably won't bother. However,
any legitimate domain, even those that don't intend to operate public mail servers,
will most likely want to block others from using their name to forge emails. A
simple code in their DNS record will tell the world, "Block all mail
claiming to be from our domain. We have no public mail servers."
The Problems with Forwarders
At this point, you probably know
all you need to know about email authentication, but there are some additional
details when an email forwarder is involved. Forwarders can complicate an
IP-authentication method like SPF or SenderID or invalidate a signature by modifying
a message, as some mailing lists like to do.
Use of a forwarder prevents the receiver from directly seeing the
sender's IP Address. The incoming IP packets have only the forwarder's IP
Address. Two solutions are
possible. Either you trust the forwarder
to authenticate the sender, in effect extending the receiver's boundary to the
Internet, or you trust the forwarder to enforce his own anti-spam policy, in
effect extending the sender's boundary to the Internet.
The situation gets even more
complicated when there are forwarders on both sides of the transaction. All it takes is one mismanaged link in the
chain-of-trust from sender to receiver, and it is no longer possible to
authenticate the sender. Even worse, a
forwarder could be actively involved in the crime, doing such things as a
"replay attack" in which one message with a valid signature is
relayed to a million recipients.
The solution to these problems
requires trust and effort on both ends.
Senders and receivers must trust all agents within their own
"administrative domains" to be honest and competent. Then we can focus our attention on the one
boundary that cannot be trusted, the one between unrelated sending and
receiving domains. Providing trust at
this boundary is the essential role of email authentication, and what must be
done to restore the email system to what it once was – a reliable and
universal communications medium, available to any sender who meets some very
minimal requirements of honesty and competence.
Footnotes
{1} This is a simplified article,
based on the original Wikipedia article
by David MacQuigg, including some of the contributions from subsequent authors.
{2} America Online claims
to have eliminated outgoing spam. A small sample of reports
from SpamCop seems to
validate this.
{3} For current spam data see "Threat Statistics" at http://www.messagelabs.com
{4} IP Address forgery is possible, but generally involves a lower level of
criminal behavior ( breaking and entering, wiretapping, etc.), and these crimes
are neither exciting to a hacker, nor sufficiently risk-free for a typical
spammer.
{5} There have been attacks on DNS servers, but doing this on a large scale
over a long period of time may be orders of magnitude more difficult than
spreading zombie infections among millions of insecure home computers. The much
smaller number of DNS servers could be upgraded to use DNSSEC if such
attacks were to become commonplace.
References
[1] How mail flows through the
Internet http://www.openspf.org/mailflows.html
[2] Sender Policy Framework (SPF) http://www.openspf.org
[3] SenderID http://en.wikipedia.org/wiki/SenderID
[4] Certified Server Validation (CSV) http://en.wikipedia.org/wiki/Certified_Server_Validation
[5] DomainKeys http://en.wikipedia.org/wiki/DomainKeys