LOW LATENCY

Low Latency

Low latency allows human-unnoticeable delays between an input being processed and the corresponding output providing real time characteristics. 

Low latency is a function of many things, the two most important ones being:

·         network latency - i.e. the time taken on the network to transmit/receive messages.

·         processing latency - i.e. the time taken by your application to act on a message/event.

Classloading is a sequential process that involves IO to disk. Make sure all the classes for your main transaction flows are loaded upfront and that they never get evicted from the perm generation.

Model your business domain and ensure all your algorithms are O(1) or at least O(log n). This is probably the biggest cause of performance issues in my experience. Make sure you have performance tests to cover the main cases.

Cache misses are your biggest cost to performance. Use algorithms that are cache friendly and set affinity to processor cores either with taskset or numactl for a JVM or JNI for individual threads.

Understand Object Allocation and Garbage Collection. This is a massive topic, but basically you want to avoid GC pauses (often caused by the Stop the World nature of various GC collections). Specialist GC collectors like the Azul collector can in many cases solve this problem for you out of the box, but for most people they need to understand how to tune the Sun/Oracle GCs (CMS, G1, etc).

Use in-memory computing. Memory is cheap, you can have tera byte of data in memory.

Generally you want lock free algorithms and I/O. Even the most well designed concurrent application (that uses locks) is at risk of blocking, blocking in low latency is generally bad :-).

Be aware of the impact of logging. logging binary or using coded output that you can parse off line is probably a good idea.

Use mechanical sympathy - Refer lmax disruptor, excellent framework.

http://www.infoq.com/articles/G1-One-Garbage-Collector-To-Rule-Them-All

http://stackoverflow.com/questions/2574579/what-is-your-development-checklist-for-java-low-latency-application

Reduce the number of methods to the stack chain. Try to implement the code only in a single method…

Here's my short-notes about Low Latency which I have got exposure to.

I used to consider some of the below factors while building the Low Latency application:

• Clear understanding of Java Memory Management will help to tune Garbage Collection.
• Always ensure most of the time there's a 'Cache Hit' rather than a 'Cache Miss' for better performance.
• Applied flyweight design pattern to create a pool of shared objects, which helps minimizes the number of objects created.
• Optimize I/O operations
• Use threads and ensure to have multiple core processor’s as well
• Use StringBuffer/StringBuilder classes instead of immutable String objects
• Use ArrayLists, HashMap as opposed to Vector, Hashtable where possible; as the methods in ArrayList, HashMap are not synchronized