June 9, 2011

Pitfalls of the JNetPcap library

During some software development we have found the following problems with using the JNetPcap library (1.3.0). Points 1 and 4 are still valid for the version 1.4.r1300 (I didn't check the other points). I would like to share our experience.


1. SIP messages support.

In order to check if some packet contains SIP message, we use the following code:

public void nextPacket(PcapPacket packet, String arg1) {
    Sip sipHeader = new Sip();
    if (packet.hasHeader(sipHeader)) {
        doSomething();
    }
}

However JNetPcap doesn't recognize all SIP messages. For example, the messages NOTIFY (RFC3265), SUBSCRIBE (RFC3265) and  REFER (RFC3515) will not be parsed as a SIP message and hasHeader(sipHeader) call will return false. Probably the messages INFO (RFC2976), UNSUBSCRIBE (RFC3265) and some other methods are not also supported by the library.

Update: Version 1.4.r1300 doesn't still support these messages.

2. SIP messages payload

To get a SIP message we used method getPayload() of Sip class. However, sometimes the method returned null for good SIP message. We realised that it is necessary to use Udp or Tcp class to have reliable access to the payload. Following code shows the work around solution:

Udp udpHeader = new Udp();
Tcp tcpHeader = new Tcp();

if ( packet.hasHeader(udpHeader) ) {
    sipMessage = new String(udpHeader.getPayload());
} else if ( packet.hasHeader(tcpHeader) ) {
    sipMessage = new String(tcpHeader.getPayload());
}

3. RTP packets with G.729 payload are not supported

The following code caused the exception "ArrayIndexOutOfBoundsException" when we tried to execute it against RTP packet with G.729 payload inside:

if ( packet.hasHeader(rtpHeader)) {
    payloadType = rtpHeader.typeEnum();
}

Complete exception trace:

Exception in thread "pool-1-thread-2" java.lang.ArrayIndexOutOfBoundsException: 18
    at org.jnetpcap.protocol.voip.Rtp$PayloadType.valueOf(Unknown Source)
    at org.jnetpcap.protocol.voip.Rtp.typeEnum(Unknown Source)
    ...
    at org.jnetpcap.Pcap.loop(Native Method)

The author of JNetPcap promised us to fix this issue in the next release 1.4.0.

4. Incorrect validation of RTP packets

Some time ago I wrote about how to determine if some packet is RTP or not. In that article I showed example from JNetPcap library. It considers RTP packets with zero sequence number and zero timestamp as invalid RTP packets. In practice we received RTP packet with sequence number = 0 and this packet was rejected by JNetPcap. According to the specification RFC3550 it is correct situation. I couldn't find restriction to have zero sequence number and timestamp.

Update: It is still valid for the version 1.4.r1300.

5. Pcap file reopen issue

When some file is opened, parsed, closed and opened again, the frame numbers are not initialized from the zero. The method getFrameNumber() of PcapPacket class returns the next number after the last one. For example, my pcap file contains 20 packets. After reopening it the method getFrameNumber() will return 21 for the first packet. From my point of view the frame numbers should be taken from the file. However this shows something different.

6. Performance issues

Let's consider the code to read a SIP message:

Tcp tcpHeader = new Tcp();
Udp udpHeader = new Udp();
Sip sipHeader = new Sip();

if ( packet.hasHeader(udpHeader) && packet.hasHeader(sipHeader) ) {

    sipMessage = new String(udpHeader.getPayload());

} else if ( packet.hasHeader(tcpHeader) && packet.hasHeader(sipHeader) ) {

    sipMessage = new String(tcpHeader.getPayload());

}


The methods getPayload() don't return a reference to a payload. They copy data from packet to a byte array. After that String constructor will copy the byte array into itself.

The method getByteArray() is also a copying method. I could not find a way to get the reference to the payload.

I suppose that the methods hasHeader() are also copying methods. They copy header data into newly created object.

To sum up, in order to read SIP message we have to allocate 3 objects in the memory, we have to make complete copy of different headers 3 times just to check if they are TCP, UDP or SIP, and at last we need to make double copy of the payload.

It doesn't look efficient.

PS. Since the JNetPcap doesn't support number of SIP messages, this example is not complete solution. UDP or TCP payload is necessary. It will require more efforts.

7. TCP getPayload() problem.

The method getPayload() of Tcp class works incorrectly. Please look the following code example:
JPacket packet = new JMemoryPacket(JProtocol.ETHERNET_ID,ethernetPacketWithRtpInTcp);

Ip4 ipHeader = new Ip4();
Tcp tcpHeader = new Tcp();
Rtp rtpHeader = new Rtp();

packet.getHeader(ipHeader);
packet.getHeader(tcpHeader);
packet.getHeader(rtpHeader);

RTP header is not parsed and exception "com.sun.jdi.InvocationException occurred invoking method" happens.
We found the workaround:
JPacket packet = new JMemoryPacket(JProtocol.ETHERNET_ID,ethernetPacketWithRtpInTcp);

Ip4 ipHeader = new Ip4();
Tcp tcpHeader = new Tcp();
Rtp rtpHeader = new Rtp();

packet.getHeader(ipHeader);
packet.getHeader(tcpHeader);

byte[] payload = tcpHeader.getPayload();

JPacket rtpJPacket = new JMemoryPacket(JProtocol.RTP_ID, payload);
rtpJPacket.getHeader(rtpHeader);

The similar problem happens when you try to get TCP payload with SIP message inside. However it looks a little bit different.
JPacket packet = new JMemoryPacket(JProtocol.ETHERNET_ID, ethernetPacketWithSipInTcp);

Ip4 ipHeader = new Ip4();
Tcp tcpHeader = new Tcp();

packet.getHeader(ipHeader);
packet.getHeader(tcpHeader);

String msg = new String(tcpHeader.getPayload());

msg will contain only last 20 bytes of the payload.

The workaround hasn't been found.

8. Big trace files processing.

I found that big files (in my test I used 200 MBytes file) are not processed by jNetPcap correctly. We used parameters -Dnio.mx=1024mb -Dnio.ms=512mb -Dnio.blocksize=2mb.

The following exception happens:

Exception in thread "pool-3-thread-3" java.nio.BufferUnderflowException
        at org.jnetpcap.nio.JBuffer.check(Unknown Source)
        at org.jnetpcap.nio.JBuffer.getUByte(Unknown Source)
        at org.jnetpcap.protocol.tcpip.Tcp.hlen(Unknown Source)
        at org.jnetpcap.protocol.tcpip.Tcp.decodeHeader(Unknown Source)
        at org.jnetpcap.packet.JHeader.decode(Unknown Source)
        at org.jnetpcap.packet.JPacket.getHeaderByIndex(Unknown Source)
        at org.jnetpcap.packet.JPacket.hasHeader(Unknown Source)
        at org.jnetpcap.packet.JPacket.hasHeader(Unknown Source)


9. DTMF payload type


In the chapter 4 I wrote about incorrect RTP packet identification. There is one more issue with that code. It rejects all payload types greater than 25. There are 127 standard payload types totally. So 102 types are not supported. DTMF uses some of dynamic payload types which are 95-127.