TCP Split Handshake Attack and How it affect Server Security

Many computer network and IT Security professionals know the furious discussion going on about the last NSS lab report about security products (with Firewall protection). They tested 6 different situations and one of them "TCP Split Handshake Attack" was very successful for most of the products they tested. They successfully breached the security of most of the Firewall products from different vendors using TCP Split Handshake Attack method. So what is TCP Split Handshake Attack ? How it is a threat for current network ? If we refer RFC 793 (where TCP is explained), we can see how a TCP connection is established. TCP (Transmission Control Protocol) is a connection oriented protocol and thus it need handshaking process to establish a successful TCP connection. Blow you can see how the TCP connection is established between two TCP devices A and B.

What is TCP Handshake ? 

Before we go to TCP Split Handshake, we should understand how the three way handshake (normal TCP handshake) happen to establish a TCP connection. Let us say we have Two TCP devices A and B. Here A want to start a TCP connection with B and so A act as client and B act as server. In TCP connection the SYN\ACK packet is determining the server and client. So a proper TCP handshake is necessary to establish the right server client relation. So a simplified TCP handshake between A and B is:

Client (A) ISN =X

Server (B) ISN =Y

ISN= Initial Sequence Number

Step 1: A --- (connection request)-->B SYN=1, Seq= X

Step 2: B --- (Connection Granted) -->A SYN=1, Seq=Y, ACK= X+1

Step 3: A ----(acknowledgment) -->B SYN=0, Seq= X+1, ACK= Y+1
What is TCP Split Handshake 

Now let us check how TCP Split Handshake occur. The above three way handshake method can be written as below.

Step 1: A ----->B SYN, Seq= X

Step 2: B ----->A ACK sequence number of A is X

Step 3: B -----> A SYN, Sequence number of B is Y

Step 4: A ---> SYN\ACK Sequence number of A is X, Sequence number of B is Y+1 ( This is a possible error in many vendor products )

Step 5: Again the Three way handshake continue and the server will be A and B act like client. So Firewall get confused.

The above error in TCP three way handshake lead to TCP Split Handshake Attack where Firewall will be confused to find the actual server and client. So firewall think like B is the client (where in reality A is the client and requested connection) and it may lead to security vulnerability. In this split handshake signals some products (where the signal is strange ) drop the entire connection but some respond to it in unexpected ways.