This article is part of the Identity and Access Management Security School lesson on VPNs and remote access. Visit the VPNs and remote access lesson page for more learning resources.

IPsec VPNs extend a network's security perimeter by connecting individual hosts or entire networks. A secure VPN starts with verifying the identity of those tunnel endpoints, but poor authentication choices can cause interoperability issues or network compromise. This tip explores common IPsec VPN identity and authentication options, and their security and deployment implications.

Standard IPsec VPN features

IPsec VPN tunnels can be configured statically ("nailed up") or established dynamically using messages defined by the Internet Key Exchange (IKE) standard. IKE lets two VPN gateways (or two VPN hosts, or a gateway and host) authenticate each other, negotiate security parameters, and generate keys for data encryption and integrity.

Authentication is critical to avoid establishing tunnels with unauthorized parties. In a site-to-site VPN, it lets us spot an attacker masquerading as a peer VPN gateway. In a remote access VPN, it lets us deny access to intruders who pose as legitimate users. To enable authentication, IKE peers identify themselves using standard Identity Types: an

To defeat ID spoofing, we require proof of claim using an IKE standard Authentication Method: a Pre-Shared Key (PSK), an RSA or DSS digital signature , or an encrypted public key . To use PSKs, the same secret value must be configured into both peers before they can authenticate each other. To use digital signatures, each peer must be issued a certificate by a Certificate Authority (CA) that the other trusts. To use encrypted public keys, each peer must generate its own RSA key pair, and then configure its public key into every IKE peer with whom it plans to communicate. IKE peers must use the same Authentication Method; for example, [email protected] and vpn.mycorp.com might both use the shared PSK "opensesame." Or they might both use RSA signatures, each generated with the sender's own certificate.

Because these messages cross untrusted networks where eavesdropping may occur, IKE uses cryptography to protect credentials. For example, IKE PSKs are never transmitted; peers just exchange signed hashes that prove they known the same PSK. But the same cannot be said for IKE IDs. IKE operates in two modes: a 5-message Main Mode that prevents ID sniffing, or a 3-message Aggressive Mode that sends the initiator's ID "in the clear." Both Modes support any IKE standard ID Type and Authentication Method, with one exception: if Main Mode is used with PSK, the ID must be an IP address. This makes Main Mode/PSK impractical for remote access VPNs, because mobile users rarely connect from static IP addresses.

Facilitating remote access

The mutual authentication methods supported by IKE are a good fit for site-to-site VPNs, but IKE does not support asymmetric user authentication methods like passwords, challenge/response exchanges and two-factor tokens, which are commonly used for remote access. To overcome this, many vendors implement non-standard enhancements like Extended Authentication (XAUTH ).

XAUTH is an Internet Draft that supports asymmetric authentication by inserting a new message exchange after Main/Aggressive Mode (IKE Phase 1) and before IPsec parameter negotiation (IKE Phase 2). VPN gateways that use XAUTH can prompt remote users for a secondary login. If user login succeeds, IPsec setup continues; otherwise, setup is abandoned. XAUTH -- and an improvement called Hybrid -- are widely implemented by VPN concentrators and clients to enable user authentication with "legacy" credentials like Windows logins or SecurID tokens.

Although many vendors implement the same XAUTH draft, this extension was not approved as an Internet standard because it is vulnerable to man-in-the-middle attacks . In particular, using weak credentials to complete IKE Phase 1 makes it too easy for an attacker to masquerade as a VPN gateway, prompting other users for their credentials. To reduce this risk, the XAUTH draft strongly discourages using IKE with the same (group) PSK for all users. Unfortunately, many remote access VPNs are configured in precisely this manner! To learn more, consult the XAUTH draft and these references:

Another legacy user authentication option is to run the Layer 2 Tunneling Protocol (L2TP ) over IPsec. This approach was originally used by the VPN client built into Windows 2000/XP, and is now also supported by many other VPN vendors. L2TP authentication occurs after IPsec is completely established.

Selecting an authentication mode for your IPsec VPN

Given this collection of ID Types and standard/extended Authentication Methods, how can you make the best choices for your own VPN? There are many factors to consider, including the number and type of endpoints involved, existing credentials and authentication databases (if any), interoperability needs, corporate security policies, and risk tolerance.

PSKs are the easiest VPN credentials to generate -- and the easiest to compromise. PSKs, like passwords, are vulnerable to social engineering, and valid ID and PSK values are easy to share with those who should not have them. PSKs that are too short or simple are relatively easy to guess, and tools like ikecrack can automate this attack. Clearly, authenticating every user with the same PSK is a high-risk scenario. Authenticating a single gateway pair with a very long, random PSK known only to one admin presents less risk. If you must use PSKs, avoid disclosing associated IDs with Aggressive Mode.

Raw public keys are not hard to generate and configure into a small set of endpoints, like a site-to-site VPN of modest size. The security afforded by encrypted public keys is far stronger than PSKs, but this method doesn't scale well and can pose interoperability problems in multi-vendor VPNs. In fact, RFC 4109 downgraded this method from "SHOULD" to "MAY" implement due to its lack of deployment.

Digital signatures use certificates to bind a subject's identity to its public key, vouched for by a trusted CA. This strengthens security without requiring every endpoint to be configured with every other endpoint's public key. Instead, endpoints need only be configured with one CA's certificate. RSA signatures are widely supported by VPN products; DSS is less common. Digital signatures scale well in companies that already have a Public Key Infrastructure. However, companies without an established platform for issuing certificates -- especially to remote users -- may consider this method too costly. Some VPN products have a built-in CA (or offer an online CA service) to make certificate management easier.

Even companies that bite the bullet and issue certificates to VPN gateways can be reluctant to part with legacy user authentication. As we have explained, there are cases where XAUTH poses high risk and cannot be recommended. However, there are ways to reduce XAUTH's man-in-the-middle risk while leveraging the added value of two-factor user authentication. For example, use IKE Main Mode with Certificates, issuing unique certificates to each VPN gateway, and a group certificate to your users. Follow this with SecurID authentication to verify individual user identity. This combo deters VPN gateway spoofing and PSK cracking, without requiring per-user certificate management. L2TP-over-IPsec is another option, if supported by your VPN Gateway and all clients.

ID Type depends on endpoint type, IKE Mode and Authentication Mode. VPN gateways often use IP address or (for more flexibility) FQDN. VPN hosts often use User FQDN, except for Main Mode/PSK, which requires the initiator's IP address. Depending on the product, remote access VPNs with a group PSK may treat a range of IP addresses as one ID, or use a group ID (FQDN) in Aggressive Mode. X.500 DNs are typically used only with Digital Certificate authentication. Alternatively, an IP address, hostname or email address can be included in a certificate's altSubjectName attribute.

On the horizon: IKEv2

After completing the original IKE standard, the IETF began work on a major revision that was published as RFC 4306 in December 2005. This new standard, IKEv2 , is not interoperable with IKEv1. IKEv2 is a new, streamlined key management protocol for use with IPsec. IKEv2 supports similar-but-different peer authentication options, with built-in support for asymmetric user authentication through the Extensible Authentication Protocol (EAP). ICSA Labs is now accepting applications for VPN products that wish to be tested for IKEv2 certification. Any company kicking off a new IPsec VPN from scratch should take a close look at IKEv2 vendor support and deployment timeframe. Those with deployed VPNs should keep an eye on IKEv2 progress and consider the business benefits of future migration. In any case, consider your authentication options carefully to make best use of today's mature, widely-deployed IKEv1 products.

About the author
Lisa Phifer owns Core Competence Inc., a consulting firm specializing in network security and management technology. Lisa has been involved in the design, implementation and evaluation of data communications, internetworking, security and network management products for over 20 years. At Core Competence, she has advised large and small companies regarding security needs, product assessment and the use of emerging technologies and best practices. Before joining Core Competence, Lisa was a Member of Technical Staff at Bell Communications Research where she won a president's award for her work on ATM Network Management.