Showing posts with label SIP. Show all posts
Showing posts with label SIP. Show all posts

November 26, 2011

Performance analysis of jNetPcap library

We are working on creation of VoIP sniffer application which must to capture RTP packets and SIP messages. We had to make the performance analysis to find the quickest way to free internal jNetPcap ring buffer and avoid network packets loss. Our application is used on multi core PCs, e.g. Xeon E5630.

For analysis of the performance jVisualVM profiler 1.3.3. was used. For the test I used trace pcap file with 262000 network packets including 1500 RTP packets and some small number of SIP messages. So useful traffic was just 1%.

Our first implementation of a capture thread was like following:

public void nextPacket(final PcapPacket packet, final String unused) {
   
    NetworkPacket outputPacket = null;
    boolean tcp = false;

    try {
        if ((packet.hasHeader(updHeader) || (tcp=packet.hasHeader(tcpHeader))) && !packet.hasHeader(icmpHeader)) {
           
            if (packet.hasHeader(rtpHeader)) {
                if ( packet.hasHeader(ipHeader) ) {
                    outputPacket = new RtpPacket(ipHeader, updHeader, rtpHeader);
                }
            } else if (packet.hasHeader(sipHeader)) {
                if ( packet.hasHeader(ipHeader) ) {
                    if (tcp) {
                        outputPacket = new SipPacket(ipHeader, tcpHeader, sipHeader);
                    } else {
                        outputPacket = new SipPacket(ipHeader, updHeader, sipHeader);
                    }
                }
            }

        if (outputPacket != null) {
            if (!packetProcessingQueue.offer(outputPacket)) {
                logger.error("packetProcessingQueue is full when packet [{}] is received", packet.getFrameNumber());
            }
        }

    } catch (ExceptionBadPacket e) {
        logger.info("Incorrect packet has been received from JNetPacket", packet.getFrameNumber());
    }
}

The RtpPacket and SipPacket constructors make copy of the network packets payload.
The profiler showed following results.


After that I tried to replace hasHeader() by getHeader() function call. It is less convenient because an user have to compare with null. However this function enabled us to increase the speed twice.


I was sure that the filtering enabled us to get higher speed because we skipped unnecessary packets and copied only useful RTP and SIP packets. However the results was absolutely unexpected for me. jNetPcap library copies network packets faster than checks its type. So most efficient way is to capture ALL network packets into the second thread and make analysis there.

The source code for the capture thread looks like:

public void nextPacket(PcapHeader header, JBuffer buffer, String any) {
   
    PcapPacket packet = new PcapPacket(header,buffer);
   
    if (packet != null) {
        if (!packetProcessingQueue.offer(packet)) {
            logger.error("packetProcessingQueue is full when packet [{}] is received", packet.getFrameNumber());
        }
    }
}


October 23, 2011

SIP protocol parsing: How to match call legs.

One of the main tasks for any protocol parser is how to find parts of the same call (call legs).

Any call is established through PBX and looks like it is showed in the picture 1 or 2.


Subscriber A calls to subscriber B. In the beginning a connection from Party A to PBX is established. Then PBX establishes the connection to Party B. From SIP protocol point of view two different calls are established with different Call-Id 12 and 23. After that RTP streams are transmitted from Party A to Party B and vice versa. There are two possible routes. The RTP streams could be passed through the PBX or directly between the participants.
 
You can see that one conversation consists of two SIP calls with different Call-Id. One call is established to PBX and another call is established from PBX. These calls are usually called "call legs".

Lets look into one real trace of a basic call scenario.


In the picture above PBX has 192.168.2.102 address.

Two calls with different call ids were established. They depicted by wireshark tool in different colors. Lets look into details of the calls.

The first call has Call-Id="***b3b1b". 
The calling party has extension "49602150014015" and media IP address 192.168.2.63.
The called party has extension  "4100" and media IP address 192.168.2.59.

 

The second call has Call-Id="***JPwcK6".
The calling party has extension "49602150014015" and media IP address 192.168.2.63.
The called party has extension  "49602150014100" and media IP address 192.168.2.59.



You can see that the PBX substitutes short phone extension for long extension and vice-versa. That is why it is not possible to use phone extensions to match call legs. On the contrary media IP addresses are constant.

So SIP call legs can be matched only by using media IP addresses.

June 9, 2011

Pitfalls of the JNetPcap library

During some software development we have found the following problems with using the JNetPcap library (1.3.0). Points 1 and 4 are still valid for the version 1.4.r1300 (I didn't check the other points). I would like to share our experience.


1. SIP messages support.

In order to check if some packet contains SIP message, we use the following code:

public void nextPacket(PcapPacket packet, String arg1) {
    Sip sipHeader = new Sip();
    if (packet.hasHeader(sipHeader)) {
        doSomething();
    }
}

However JNetPcap doesn't recognize all SIP messages. For example, the messages NOTIFY (RFC3265), SUBSCRIBE (RFC3265) and  REFER (RFC3515) will not be parsed as a SIP message and hasHeader(sipHeader) call will return false. Probably the messages INFO (RFC2976), UNSUBSCRIBE (RFC3265) and some other methods are not also supported by the library.

Update: Version 1.4.r1300 doesn't still support these messages.

2. SIP messages payload

To get a SIP message we used method getPayload() of Sip class. However, sometimes the method returned null for good SIP message. We realised that it is necessary to use Udp or Tcp class to have reliable access to the payload. Following code shows the work around solution:

Udp udpHeader = new Udp();
Tcp tcpHeader = new Tcp();

if ( packet.hasHeader(udpHeader) ) {
    sipMessage = new String(udpHeader.getPayload());
} else if ( packet.hasHeader(tcpHeader) ) {
    sipMessage = new String(tcpHeader.getPayload());
}

3. RTP packets with G.729 payload are not supported

The following code caused the exception "ArrayIndexOutOfBoundsException" when we tried to execute it against RTP packet with G.729 payload inside:

if ( packet.hasHeader(rtpHeader)) {
    payloadType = rtpHeader.typeEnum();
}

Complete exception trace:

Exception in thread "pool-1-thread-2" java.lang.ArrayIndexOutOfBoundsException: 18
    at org.jnetpcap.protocol.voip.Rtp$PayloadType.valueOf(Unknown Source)
    at org.jnetpcap.protocol.voip.Rtp.typeEnum(Unknown Source)
    ...
    at org.jnetpcap.Pcap.loop(Native Method)

The author of JNetPcap promised us to fix this issue in the next release 1.4.0.

4. Incorrect validation of RTP packets

Some time ago I wrote about how to determine if some packet is RTP or not. In that article I showed example from JNetPcap library. It considers RTP packets with zero sequence number and zero timestamp as invalid RTP packets. In practice we received RTP packet with sequence number = 0 and this packet was rejected by JNetPcap. According to the specification RFC3550 it is correct situation. I couldn't find restriction to have zero sequence number and timestamp.

Update: It is still valid for the version 1.4.r1300.

5. Pcap file reopen issue

When some file is opened, parsed, closed and opened again, the frame numbers are not initialized from the zero. The method getFrameNumber() of PcapPacket class returns the next number after the last one. For example, my pcap file contains 20 packets. After reopening it the method getFrameNumber() will return 21 for the first packet. From my point of view the frame numbers should be taken from the file. However this shows something different.

6. Performance issues

Let's consider the code to read a SIP message:

Tcp tcpHeader = new Tcp();
Udp udpHeader = new Udp();
Sip sipHeader = new Sip();

if ( packet.hasHeader(udpHeader) && packet.hasHeader(sipHeader) ) {

    sipMessage = new String(udpHeader.getPayload());

} else if ( packet.hasHeader(tcpHeader) && packet.hasHeader(sipHeader) ) {

    sipMessage = new String(tcpHeader.getPayload());

}


The methods getPayload() don't return a reference to a payload. They copy data from packet to a byte array. After that String constructor will copy the byte array into itself.

The method getByteArray() is also a copying method. I could not find a way to get the reference to the payload.

I suppose that the methods hasHeader() are also copying methods. They copy header data into newly created object.

To sum up, in order to read SIP message we have to allocate 3 objects in the memory, we have to make complete copy of different headers 3 times just to check if they are TCP, UDP or SIP, and at last we need to make double copy of the payload.

It doesn't look efficient.

PS. Since the JNetPcap doesn't support number of SIP messages, this example is not complete solution. UDP or TCP payload is necessary. It will require more efforts.

7. TCP getPayload() problem.

The method getPayload() of Tcp class works incorrectly. Please look the following code example:
JPacket packet = new JMemoryPacket(JProtocol.ETHERNET_ID,ethernetPacketWithRtpInTcp);

Ip4 ipHeader = new Ip4();
Tcp tcpHeader = new Tcp();
Rtp rtpHeader = new Rtp();

packet.getHeader(ipHeader);
packet.getHeader(tcpHeader);
packet.getHeader(rtpHeader);

RTP header is not parsed and exception "com.sun.jdi.InvocationException occurred invoking method" happens.
We found the workaround:
JPacket packet = new JMemoryPacket(JProtocol.ETHERNET_ID,ethernetPacketWithRtpInTcp);

Ip4 ipHeader = new Ip4();
Tcp tcpHeader = new Tcp();
Rtp rtpHeader = new Rtp();

packet.getHeader(ipHeader);
packet.getHeader(tcpHeader);

byte[] payload = tcpHeader.getPayload();

JPacket rtpJPacket = new JMemoryPacket(JProtocol.RTP_ID, payload);
rtpJPacket.getHeader(rtpHeader);

The similar problem happens when you try to get TCP payload with SIP message inside. However it looks a little bit different.
JPacket packet = new JMemoryPacket(JProtocol.ETHERNET_ID, ethernetPacketWithSipInTcp);

Ip4 ipHeader = new Ip4();
Tcp tcpHeader = new Tcp();

packet.getHeader(ipHeader);
packet.getHeader(tcpHeader);

String msg = new String(tcpHeader.getPayload());

msg will contain only last 20 bytes of the payload.

The workaround hasn't been found.

8. Big trace files processing.

I found that big files (in my test I used 200 MBytes file) are not processed by jNetPcap correctly. We used parameters -Dnio.mx=1024mb -Dnio.ms=512mb -Dnio.blocksize=2mb.

The following exception happens:

Exception in thread "pool-3-thread-3" java.nio.BufferUnderflowException
        at org.jnetpcap.nio.JBuffer.check(Unknown Source)
        at org.jnetpcap.nio.JBuffer.getUByte(Unknown Source)
        at org.jnetpcap.protocol.tcpip.Tcp.hlen(Unknown Source)
        at org.jnetpcap.protocol.tcpip.Tcp.decodeHeader(Unknown Source)
        at org.jnetpcap.packet.JHeader.decode(Unknown Source)
        at org.jnetpcap.packet.JPacket.getHeaderByIndex(Unknown Source)
        at org.jnetpcap.packet.JPacket.hasHeader(Unknown Source)
        at org.jnetpcap.packet.JPacket.hasHeader(Unknown Source)


9. DTMF payload type


In the chapter 4 I wrote about incorrect RTP packet identification. There is one more issue with that code. It rejects all payload types greater than 25. There are 127 standard payload types totally. So 102 types are not supported. DTMF uses some of dynamic payload types which are 95-127.


February 25, 2011

Jain SIP introduction

IM application (IP active) implementation.

There is a good article "An Introduction to the JAIN SIP API"
which can be used as first step guide for development of "IP active" component.

SIP parser application (IP passive) implementation.

However we need to use JainSIP for parsing SIP as well.
This topic is not covered in the article. But it is enough easy.

First, we need to create and initialize SipFactory and MessageFactory instances:

sipFactory = SipFactory.getInstance();
sipFactory.setPathName("gov.nist");
messageFactory = sipFactory.createMessageFactory();

Second, we need to get instance of SIPRequest or SIPResponse and initialize them by input buffer with a real SIP message.

SIPRequest sipRequest = (SIPRequest)messageFactory.createRequest(sipMessage);

How to understand if the income message request or response:

if ( sipMessage.startsWith( SIPConstants.SIP_VERSION_STRING ) ){
processResponse( sipMessage );
}
else {
processRequest( sipMessage );
}

Now we can get any information from the SIP message.

ListIterator<String> itHeaders = sipRequest.getHeaderNames();
String requestMethod = sipRequest.getMethod();

Detailed description of JainSIP API can be found here.

February 22, 2011

Basic connections and identifiers of SIP protocol

1. Transaction

"A SIP transaction occurs between a client and a server and comprises all messages from the first request sent from the client to the server up to a final (non-1xx) response sent from the server to the client." (RFC3261)

In other words the SIP transaction includes a request and all responses on it.

The  SIP transaction is identified by a branch parameter in the top of  "Via" header field and "CSeq" header field value. These identifiers are showed in the picture below in cyan color.


2. Dialog

"A dialog is a peer-to-peer SIP relationship between two UAs that persists for some time." (RFC3261)

A dialog is identified by a call identifier in "Call-ID"header field, local tag and a remote tag in "To" and "From" header fields. The identifiers are depicted in green color.


3. Session

"A multimedia session is a set of multimedia senders and receivers and the data streams flowing from senders to receivers." (RFC2327)

The session is defined by the concatenation of the SDP user name, session id, network type, address type,  and address elements in the origin field "o=". The session identifiers are showed in yellow color.

One of the best illustrated SIP introductions - Tech-invite.

In the picture below you can see example of a SIP response message.

SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.168.2.102:5060;branch=z9hG4bKSEC-a01a8c0-a01a8c0-1-oW11d3T02H
From: "Test 4100" <sip:4100@192.168.2.102>;tag=snl_951dhI11AG
To: <sip:49602150014101@192.168.2.70>;tag=4138389295
Call-ID: SEC11-a01a8c0-a01a8c0-1-D9KQeE4Yc8gu
CSeq: 1235 INVITE
Allow:  INVITE,ACK,CANCEL,BYE,REFER,NOTIFY,MESSAGE,UPDATE
Allow-Events: talk, hold
Contact: 49602150014101 <sip:49602150014101@192.168.2.70:5060;transport=udp>
Server: OpenStage_20_V2 R0.30.2      SIP  100120
Supported: replaces, timer
X-Siemens-Call-Type: ST-insecure
Content-Type: application/sdp
Content-Length: 265

v=0
o=MxSIP 0 814456601 IN IP4 192.168.2.70
s=SIP Call
c=IN IP4 192.168.2.70
t=0 0
m=audio 5062 RTP/AVP 8 0 101
a=rtpmap:8 PCMA/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:101 telephone-event/8000
a=silenceSupp:off - - - -
a=fmtp:101 0-15
a=ptime:20
a=sendrecv