Transport Layer Security Protocols

The following sections describe the security protocols that operate over TCP/IP or some other reliable but insecure transport. They are categorized as Transport layer security protocols because their intent is to secure the Transport layer as well as to provide methods for implementing privacy, authentication, and integrity above the Transport layer.

The Secure Socket Layer Protocol

The Secure Socket Layer (SSL) is an open protocol designed by Netscape; it specifies a mechanism for providing data security layered between application protocols (such as HTTP, Telnet, NNTP, or FTP) and TCP/IP. It provides data encryption, server authentication, message integrity, and optional client authentication for a TCP/IP connection.

The primary goal of SSL is to provide privacy and reliability between two communicating applications. This process is accomplished with the following three elements:

• The handshake protocol. This protocol negotiates the cryptographic parameters to be used between the client and the server session. When an SSL client and server first start communicating, they agree on a protocol version, select cryptographic algorithms, optionally authenticate each other, and use public-key encryption techniques to generate shared secrets.

• The record protocol. This protocol is used to exchange Application layer data. Application messages are fragmented into manageable blocks, optionally compressed, and a MAC (message authentication code) is applied; the result is encrypted and transmitted. The recipient takes the received data and decrypts it, verifies the MAC, decompresses and reassembles it, and delivers the result to the application protocol.

• The alert protocol. This protocol is used to indicate when errors have occurred or when a session between two hosts is being terminated.

Let's look at an example using a Web client and server. The Web client initiates an SSL session by connecting to an SSL-capable server. A typical SSL-capable Web server accepts SSL connection requests on a different port (port 443 by default) than standard HTTP requests (port 80 by default). When the client connects to this port, it initiates a handshake that establishes the SSL session. After the handshake finishes, communication is encrypted and message integrity checks are performed until the SSL session expires. SSL creates a session during which the handshake must happen only once.

The SSL handshake process is shown in Figure 2-20. (Refer to "Public Key Infrastructure and Distribution Models," later in this chapter, for more information about digital certificates.) The steps in the process are as follows:

Step 1 The SSL client connects to the SSL server and requests the server to authenticate itself.

Step 2 The server proves its identity by sending its digital certificate. This exchange may optionally include an entire certificate chain, up to some root certificate authority (CA). Certificates are verified by checking validity dates and verifying that the certificate bears the signature of a trusted CA.

Step 3 The server may then initiate a request for client-side certificate authen-tication. However, because of a lack of a public key infrastructure, most servers today do not do client-side authentication.

Step 4 The message encryption algorithm for encryption and the hash function for integrity are negotiated. Usually the client presents a list of all the algorithms it supports, and the server selects the strongest cipher available.

Step 5 The client and server generate the session keys by following these steps:

(a) The client generates a random number that it sends to the server, encrypted with the server's

Security Technologies public key (obtained from the server's certificate).

(b) The server responds with more random data (encrypted with the client's public key, if available; otherwise, it sends the data in cleartext).

(c) The encryption keys are generated from this random data using hash functions.

Figure 2-20: The SSL Handshake Process

The advantage of the SSL protocol is that it provides connection security that has three basic properties:

• The connection is private. Encryption is used after an initial handshake to define a secret key. Symmetric cryptography is used for data encryption (for example, DES and RC4).

• The peer's identity can be authenticated using asymmetric, or public key, cryptography (for example, RSA and DSS).

• The connection is reliable. Message transport includes a message integrity check using a keyed MAC. Secure hash functions (such as SHA and MD5) are used for MAC computations.

Acceptance of SSL has only been within HTTP. The other protocols have been demonstrated to work but are not yet widely deployed.

Note A new protocol is being defined by the IETF called Transport Layer Security(TLS). It is based on the SSL 3.0 protocol specification as published by Netscape; it is likely that the industry will move towards TSL for a standardized protocol for Transport layer security. There are, however, significant differences between TLS and SSL 3.0---mostly in the cryptographic algorithms they support---such that TLS 1.0 and SSL 3.0 do not necessarily interoperate.

The Secure Shell Protocol

The Secure Shell (SSH) is a protocol for secure remote login and other secure network services over an insecure network. It provides support for secure remote login, secure file transfer, and the secure forwarding of TCP/IP and X Window system traffic. It can automatically encrypt, authenticate, and compress transmitted data. The work in progress to define the SSH protocol ensures that the SSH protocol can provide strong security against cryptanalysis and protocol attacks, can work reasonably well without a global key management or certificate infra-structure, and can use existing certificate infrastructures (such as DNSSEC and X.509) when available.

The SSH protocol consists of three major components:

• The Transport layer protocol, which provides server authentication, confidentiality, and integrity with perfect forward secrecy. Optionally, it may also provide compression.

Security Technologies

• The user authentication protocol, which authenticates the client to the server.

• The connection protocol, which multiplexes the encrypted tunnel into several logical channels.

The SSH transport layer is a secure low-level transport protocol. It provides strong encryption, cryptographic host authentication, and integrity protection. Authentication in SSH is host-based; this protocol does not perform user authentication. A higher-level protocol for user authentication can be designed on top of SSH.

The protocol has been designed to be simple and flexible enough to allow parameter negotiation and to minimize the number of round trips. The key exchange method, the public key algorithm, the symmetric encryption algorithm, the message authentication algorithm, and the hash algorithm are all negotiated.

Data integrity is protected by including with each packet a message authentication code (MAC) computed from a shared secret, a packet sequence number, and the contents of the packet.

SSH implementations can be found for UNIX, Windows, and Macintosh systems. It is a widely accepted protocol that uses well-known and well-established encryption, integrity, and public key algorithms.

The SOCKS Protocol

Socket security (SOCKS) is a Transport layer-based secure networking proxy protocol. It is designed to provide a framework for client/server applications in both the TCP and UDP domains to conveniently and securely use the services of a network firewall.

SOCKS was originally developed by David and Michelle Koblas; the code was made freely available on the Internet. Several major revisions have occurred since then, but the software has remained freely available. SOCKS Version 4 provides for unsecured firewall traversal for TCP-based client/server applications (including Telnet, FTP, and the popular information discovery protocols such as HTTP, WAIS, and Gopher). SOCKS Version 5, defined in RFC 1928, extends the SOCKS Version 4 model to include UDP, extends the framework to include provisions for generalized strong authentication schemes, and extends the addressing scheme to encompass domain-name and IPv6 addresses.

A proposal currently exists to create a mechanism for managing the entrance or exit of IP multicast through a firewall. It does this by defining extensions to the existing SOCKS Version 5 protocol, which provides a framework for user-level, authenticated firewall traversal of unicast TCP and UDP traffic. However, because the current UDP support in SOCKS Version 5 has scalability problems as well as other deficiencies (which must be addressed before multicast support can be achieved), the extensions are defined in two parts:

• Base-level UDP extensions

• Multicast UDP extensions

SOCKS works by replacing the standard network system calls in an application with special versions. (This is why SOCKS is sometimes referred to as an application-level proxy.) These new system calls open connections to a SOCKS proxy server (configured in the application by the user, or by a system configuration file) on a well-known port (usually 1080/TCP). If the connection request succeeds, the client enters a negotiation for the authentication method to be used, authenticates with the chosen method, and then sends a relay request. The SOCKS server evaluates the request and either establishes the appropriate connection or denies it. After the connection is established with the SOCKS server, the client application sends the server the name of the machine and the port number to which the user wants to connect. The SOCKS server actually makes the connection with the remote host and then transparently moves data back and forth between the application and the remote machine. The user has no idea that the SOCKS server is even in the loop (see Figure 2-21).

Figure 2-21: The SOCKS Security Model

The difficulty with using SOCKS is that somebody has to replace the network system calls with the SOCKS versions (this process is generally referred to as SOCKS-ification or SOCKS-ifying an application). Fortunately, most of the common network applications (such as Telnet, FTP, finger, and whois) have already been SOCKS-ified, and many vendors are now including SOCKS support in commercial applications.

Continue reading here: Network Layer Security

Was this article helpful?

+1 0