IPSec (Internet Protocol Security)

• Suite of protocols that provides authentication, integrity, and confidentiality for IP communications at Layer 3
• Works by encrypting and authenticating IP packets between endpoints (host-to-host, network-to-network, or host-to-network)
• Key principle: IPSec is a framework, not a single protocol - it combines multiple protocols and algorithms

IPSec Protocol Components

• Authentication Header (AH) - Provides authentication and integrity but no encryption (Protocol 51)
• Encapsulating Security Payload (ESP) - Provides authentication, integrity, and encryption (Protocol 50)
• Internet Key Exchange (IKE) - Negotiates security associations and manages encryption keys (UDP 500/4500)

IPSec Modes

Security Associations (SA)

• Unidirectional security contract between two endpoints
• Each SA identified by three elements: Security Parameter Index (SPI), destination IP, and protocol
• Requires two SAs for bidirectional communication

IKE Phases

Phase 1 (Main Mode):

• Establishes secure, authenticated channel between peers
• Negotiates encryption/hashing algorithms and authentication method
• Creates IKE SA

Phase 2 (Quick Mode):

• Uses secure channel from Phase 1 to negotiate IPSec SAs
• Defines what traffic to protect
• Establishes ESP/AH parameters

Vocabulary

• Security Association: Unidirectional agreement defining security parameters
• Transform Set: Collection of security protocols and algorithms
• Crypto Map: Configuration binding peer information and transforms
• Perfect Forward Secrecy: Ensures compromise of long-term keys doesn’t compromise past session keys

Notes

• IPSec operates at Layer 3 - can protect any upper-layer protocol
• NAT traversal (NAT-T) required when IPSec peers are behind NAT devices
• Pre-shared keys don’t scale beyond small deployments - consider certificates
• IPSec can cause MTU issues - consider adjusting TCP MSS