Showing posts with label Java. Show all posts
Showing posts with label Java. Show all posts

May 17, 2012

How to read byte buffer in Java

In telecommunications we need to read and analyze raw byte buffers frequently because this is usual representation of a network packet. However there is an issue with the Java language which does not have unsigned integer types. All integer types in Java are signed. So it is not possible just to read a byte and then analyze it.

For example, if we have byte with value 0x96 we will get -106 instead 150. Why does this happen and what to do?

Negative numbers are stored in the memory in two's complement system. According to this approach we need to make the following conversation to write negative number into the memory:
Step 1. invert all bits
Step 2. add 1

For example, let's try to convert a number -6 to the two's complement system representation.
The binary representation of the number 6 is 00000110.
Step 1. After invert conversion we will have 11111001.
Step 2. 11111001 + 1 = 11111010
So the negative number -6 will look in the two's complement system as 11111010. This number corresponds to unsigned byte 250.

Now let's see what Java makes with our number 0x96 or 150. Since there is no unsigned types in Java, this number is considered as number with a sign. Maximum value for signed byte is 127. So Java makes reverse conversion.
The binary representation of the number 0x96 is 10010110.
Step 1. subtract 1:  10010110-1=10010101
Step 2. invert all bits: ~10010101=01101010
01101010 is -106.
That is why Java considers the byte 0x96 as the negative number -106.

How to fix this Java feature? We need to make two things:
1. Convert a byte into int because the maximum value of a signed byte is only 127. At this step Java will fill the high-order bits with 1 to keep the number negative.
2. Then we need to zero high-order bits to get the original value.

The following code does this:

byte b = (byte) 0x96;
int i = (int) b & 0xFF; 

February 1, 2012

How to build jNetPcap library


Access to the source code of jNetPcap
 
jNetPcap source code is located under SVN control.
To access the repository you can use TortoiseSVN client.
Then you can get the source code using following link:https://jnetpcap.svn.sourceforge.net/svnroot/jnetpcap/jnetpcap/trunk

Preparation

The library should be linked statically with the libraries LIBGCC_S_DW2-1.DLL and LIBSTDC++-6.DLL which is located in bin folder of MinGW. The default build.xml doesn't contain necessary options. So before to start build procedure you need to update the build.xml file.

Both sections with compilation options for 32 and 64 bits platforms should be updated.

Find section "comp-jni-win32" and "comp-jni-win64" and update the section <exec executable...> inside both of them. The following lines should be added there "<arg value="-static" />"
So in result section <exec executable..."> should look like this code:

 <exec executable="${compiler.ld.cmd}">
   <arg value="-shared" />
   <arg value="-g" />
   <arg value="-Wl,--add-stdcall-alias" />
   <arg value="-static" />
   <arg line="-o ${jni.build.filepath}" />
   <arg line="${jni.obj.files}" />
   <arg line="-L ${pcap.lib.dir}" />
   <arg line="-l ${pcap.lib.name}" />
   <arg line="-l ${ms.lib.name}" />
</exec>

How to build jNetPcap library for 32-bits platform

Download and install Apache Ant building software.
It could be taken from the official site. To install it you need just open zip archive and extract the files into any directory. Then you need to create Environment variable ANT_HOME with the path to the ant directory. It is also necessary to add the path to ant bins into the PATH environment variable.

Download and install compiler MinGW.
It could be taken from the official site. During the setup you need to select "Download latest repository catalogues". It is important at least for Windows XP. Then you need to update the PATH environment variable again and add the path to MinGW bins there.

Build the jNetPcap library
Go into the jnetpcap directory with the source code and run the following command to get the complete list of possible targets:
ant help
You need to compile jnetpcap.jar and jnetpcap.dll files. Execute the following command one by one.
ant comp-java
ant build-jar
ant comp-jni

 
How to build jNetPcap library for 64-bits platform


Download and install Apache Ant building software.
It could be taken from the official site. To install it you need just open zip archive and extract the files into any directory. Then you need to create Environment variable ANT_HOME with the path to the ant directory. It is also necessary to add the path to ant bins into the PATH environment variable.

Download and install compiler MinGW64.
It could be taken from the official site. But there is no binaries ready to execute. You can find there only source code and you will need to compile the compiler yourself. The compilation procedure is enough complicated.
So it would be much faster to get already build binaries from another site. You can use Windows XP/2003 x64 binaries for Windows 7. It is ok.

Build the jNetPcap library
Go into the jnetpcap directory with the source code and run the following command to get the complete list of possible targets:
ant help

You need to compile jnetpcap.jar and jnetpcap.dll files. Execute the following command one by one.
ant comp-java
ant build-jar
ant comp-jni

November 26, 2011

Performance analysis of jNetPcap library

We are working on creation of VoIP sniffer application which must to capture RTP packets and SIP messages. We had to make the performance analysis to find the quickest way to free internal jNetPcap ring buffer and avoid network packets loss. Our application is used on multi core PCs, e.g. Xeon E5630.

For analysis of the performance jVisualVM profiler 1.3.3. was used. For the test I used trace pcap file with 262000 network packets including 1500 RTP packets and some small number of SIP messages. So useful traffic was just 1%.

Our first implementation of a capture thread was like following:

public void nextPacket(final PcapPacket packet, final String unused) {
   
    NetworkPacket outputPacket = null;
    boolean tcp = false;

    try {
        if ((packet.hasHeader(updHeader) || (tcp=packet.hasHeader(tcpHeader))) && !packet.hasHeader(icmpHeader)) {
           
            if (packet.hasHeader(rtpHeader)) {
                if ( packet.hasHeader(ipHeader) ) {
                    outputPacket = new RtpPacket(ipHeader, updHeader, rtpHeader);
                }
            } else if (packet.hasHeader(sipHeader)) {
                if ( packet.hasHeader(ipHeader) ) {
                    if (tcp) {
                        outputPacket = new SipPacket(ipHeader, tcpHeader, sipHeader);
                    } else {
                        outputPacket = new SipPacket(ipHeader, updHeader, sipHeader);
                    }
                }
            }

        if (outputPacket != null) {
            if (!packetProcessingQueue.offer(outputPacket)) {
                logger.error("packetProcessingQueue is full when packet [{}] is received", packet.getFrameNumber());
            }
        }

    } catch (ExceptionBadPacket e) {
        logger.info("Incorrect packet has been received from JNetPacket", packet.getFrameNumber());
    }
}

The RtpPacket and SipPacket constructors make copy of the network packets payload.
The profiler showed following results.


After that I tried to replace hasHeader() by getHeader() function call. It is less convenient because an user have to compare with null. However this function enabled us to increase the speed twice.


I was sure that the filtering enabled us to get higher speed because we skipped unnecessary packets and copied only useful RTP and SIP packets. However the results was absolutely unexpected for me. jNetPcap library copies network packets faster than checks its type. So most efficient way is to capture ALL network packets into the second thread and make analysis there.

The source code for the capture thread looks like:

public void nextPacket(PcapHeader header, JBuffer buffer, String any) {
   
    PcapPacket packet = new PcapPacket(header,buffer);
   
    if (packet != null) {
        if (!packetProcessingQueue.offer(packet)) {
            logger.error("packetProcessingQueue is full when packet [{}] is received", packet.getFrameNumber());
        }
    }
}


November 11, 2011

Thread states in VisualVM profiler

VisualVM uses following states to indicate thread activity status:
  • Running; 
  • Sleeping; 
  • Wait; 
  • Monitor. 
 

What do these states mean?

Running.

Thread.State = RUNNABLE.
"Thread state for a runnable thread. A thread in the runnable state is executing in the Java virtual machine but it may be waiting for other resources from the operating system such as processor."

Sleeping

Thread.State = TIMED_WAITING.

 Thread state for a waiting thread with a specified waiting time. A thread is in the timed waiting state due to calling one of the following methods with a specified positive waiting time:

Wait


Thread.State = WAITING.

Thread state for a waiting thread. A thread is in the waiting state due to calling one of the following methods:
A thread in the waiting state is waiting for another thread to perform a particular action. For example, a thread that has called Object.wait() on an object is waiting for another thread to call Object.notify() or Object.notifyAll() on that object. A thread that has called Thread.join() is waiting for a specified thread to terminate.


Monitor

Thread.State = BLOCKED.

Thread state for a thread blocked waiting for a monitor lock. A thread in the blocked state is waiting for a monitor lock to enter a synchronized block/method or reenter a synchronized block/method after calling Object.wait.





April 13, 2011

How to create initialized enum in Java.

One day I needed to create enum initialized to values other than its default values in Java. It can be done easily in C++. For example, look at the code below:

typedef enum ResponseType{
     OK = 200,
     TRYING = 100,
     RINGING = 180
} TResponseType;

However it is not so obvious in Java.

public enum ResponseType {
     OK(200),
     TRYING(100),
     RINGING(180),
     SESSION_PROGRESS(183),
     BAD_REQUEST(400),
     NOT_FOUND(404);

     private int value;

     ResponseType(int value){
           this.value = value;
     }

     public int getValue() {
          return value;
     }
};

March 2, 2011

C++/Qt vs Java

From my point of view, this article is useful for to realize possible risks.

Please find extracts from it below. The original article is attached at the end of the page.

Programmer efficiency

"Programmer-efficiency describes how efficiently (i.e. how quickly and accurately) a programmer with a given degree of experience and knowledge can implement a certain set of requirements in a particular programming language, including debugging and project setup time."

"Java has several features designed to make it more programmer-efficient. For example, unlike C++ (or C), the programmer does not have to explicitly "free" (give back) allocated memory resources to the operating system."

"Research shows that in practice, garbage collection and other Java features, do not have a major influence on the programmer-efficiency."

"In conclusion: both research and practice contradict the claim that Java programmers achieve a higher programmer-efficiency than C++ programmers."

Runtime efficiency

"Prechelt provides useful data. The amount of data he provides is huge, but he arrives at the conclusion that "a Java program must be expected to run at least 1.22 times as long as a C/C++ program".

"Note that he says at least; the average runtime of Java programs is even longer. Our own experience shows that Java programs tend to run about 2-3 times as long than their equivalent C/C++ programs for the same task."

Cross platform approaches

"It is the semi-interpreted nature of Java programs that makes the "compile once, run anywhere" approach of Java possible in the first place. Once a Java program is compiled into bytecode, it can be executed on any platform which has a JVM. In practice, this is not always the case, because of implementation differences in different JVMs, and because of the necessity to sometimes use native, non-Java code, usually written in C or C++, together with Java programs."

"But is the use of platform-independent bytecode the right approach for crossplatform applications? With a good cross-platform toolkit like Qt and good compilers on the various platforms, programmers can achieve almost the same by compiling their source code once for each platform: "write once, compile everywhere"."

"The question arises why it should be necessary to run the Java Virtual Machine in software; if a program can be implemented in software, it should also be possible to have hardware implement the same functionality."

Memory efficiency

"Java and C++ take completely different approaches to memory management. In C++, all memory management must be done explicitly by the programmer …. Java automatically de-allocates (frees) unused memory"

"Garbage collection is very convenient, but the trade offs are greater memory consumption and slower runtime speed. …. In addition to the higher memory requirements, the garbage collection process itself requires processing power … Since the garbage collector runs periodically, it can occasionally lead to Java programs "freezing" for a few seconds…..".

"The Qt toolkit takes a more efficient approach to easing the memory management task for its programmers: when an object is deleted, all dependant objects are automatically deleted too. Qt’s approach does not interfere with the programmer’s freedom to delete manually when they wish to."

"To sum up this discussion, we have found C++ to provide much better runtime- and memory-efficiency than Java, while having comparable programmer-efficiency."

Available libraries and tools

"The Java platform includes an impressive number of packages that provide hundreds of classes for all kinds of purposes, including graphical user interfaces, security, networking and other tasks. This is certainly an advantage of the Java platform. For each package available on the Java platform, there is at least one corresponding library for C++."

"However, this strength of Java is also one of its weaknesses. It becomes increasingly difficult for the individual programmer to find their way through the huge APIs."

"If Java has an advantage on the side of available libraries, C++ clearly has an advantage when it comes to available tools"

Conclusion

Both independent academic research and industrial experience demonstrate that the hype favouring Java is mostly unjustified, and that the C++/Qt combination is superior. This is mainly due to the runtime and memory efficiency problems in Java programs … and Java’s failure to deliver increased programmer efficiency. In various programming projects we have been involved in, junior-level programmers learnt Java faster, but more experienced and senior-level programmers (which are usually in charge of the application design and the implementation of the critical parts of an application) achieved better and faster results using C+.

References


Qt vs. Java: A Comparison of Qt and Java for Large- Scale, Industrial-Strength GUI Development