Low latency programming

0 votes
asked Jul 30, 2009 by sambatyon

I've been reading a lot about low latency financial systems (especially since the famous case of corporate espionage) and the idea of low latency systems has been in my mind ever since. There are a million applications that can use what these guys are doing, so I would like to learn more about the topic. The thing is I cannot find anything valuable about the topic. Can anybody recommend books, sites, examples on low latency systems?

11 Answers

0 votes
answered Jul 30, 2009 by paul-sonier

Anything on realtime programming would fit the bill. It's not precisely what you're after, I suspect, but it's an extremely good place to start.

0 votes
answered Jul 31, 2009 by starblue

If I remember correctly real time Java (RTSJ) is used in this area, though I couldn't find a good article to link to now.

0 votes
answered Jan 30, 2010 by sunil

If you are speaking about low latency server design, these are some good pointers: http://www.kegel.com/c10k.html http://pl.atyp.us/content/tech/servers.html

0 votes
answered Jul 8, 2010 by paul-sonier

Typically, work in low-latency environments means having an understanding of call dependencies and how to reduce them to minimize the dependency chain. This includes the use of data structures and libraries to store desired cacheable data as well as refactoring existing resources to reduce interdependencies.

0 votes
answered Jul 8, 2010 by adamski

I work for a financial company that produces low latency software for communication directly with exchanges (for submitting trades and streaming prices). We currently develop primarily in Java. Whilst the low latency side isn't an area I work in directly I have a fair idea of the skillset required, which would include the following in my opinion:

  • Detailed knowledge of the Java memory model and techniques to avoid unnecessary garbage collection (e.g. object pooling). Some of the techniques used might typically be regarded as "anti-patterns" in a traditional OO-environment.
  • Detailed knowledge of TCP/IP and UDP multicast including utilities for debugging and measuring latency (e.g. DTrace on Solaris).
  • Experience with profiling applications.
  • Knowledge of the java.nio package, experience developing NIO-based scalable server applications, experience designing wire protocols. Also note that we typically avoid using frameworks and external libraries (e.g. Google Protobuf), preferring to write a lot of bespoke code.
  • Knowledge of FIX and commercial FIX libraries (e.g. Cameron FIX).

Unfortunately many of the skills can only be developed "on the job" as there's no substitute for the experience gained implementing a price server or trading engine based on a spec. from an exchange or vendor. However, it's also worth mentioning that our company at least tend not to look for specific experience in this (or other) niche areas, instead preferring to hire people with good analytical and problem solving skills.

0 votes
answered Jul 8, 2010 by ed-james

Take a look at ZeroMQ. http://www.zeromq.org

Read the whitepapers on that site and you'll get some insight into what is required for low latency programming.

0 votes
answered Jul 21, 2010 by michel

http://g-wan.com/ does it all in 200 KB with ANSI C scripts.

0 votes
answered Jul 5, 2011 by inder

Low latency is a function of many things, the two most important ones being:

  • network latency - i.e. the time taken on the network to transmit/receive messages.
  • processing latency - i.e. the time taken by your application to act on a message/event.

So, if you are say writing an Order Matching system, the network latency would represent how soon within your network were you able to receive the order matching request. And processing latency would represent the time taken by your application to match the Order against existing, open orders.

Multicast, UDP, reliable multicast, Kernel bypass (supported by Java 7, Informatica Ultra Messaging, and many others) on Infiniband networks are some common technologies used by all companies in this field.

Additionally, there are low latency programming frameworks like disruptor (http://code.google.com/p/disruptor/) which implement design patterns for dealing with low latency applications. What could kill you is having to write to a DB or log files as part of your main workflow. You will have to come up with unique solutions that fulfill the requirements of the problem you are trying to solve.

In languages like Java, implementing your app such that it creates (almost) zero garbage becomes extremely important to latency. As Adamski says, having a knowledge of Java memory model is extremely important. Understand different JVM implementations, and their limitations. Typical Java design patterns around small object creation are the first things that you will throw out of the window - one can never fix the Java Garbage Collector enough to achieve low latency - the only thing that can be fixed is the garbage.

Good luck!

0 votes
answered Jul 10, 2012 by ashkrit-sharma

There are many good answer in this post. I would like to add my experience also

  • To achieve low latency in java you have to take control of GC in java, there are many ways to do that for eg pre-allocate objects(i.e use flyweight design pattern), use primitive objects - trove is very good for that, all data structure are based on primitive, Reuse object instance for eg create system wide dictionary to reduce creating new objects, very good option when reading data from stream/socket/db

    • Try to use wait-free algo( which is bit difficult), lock free algo. You can find tons of example for that

    • Use in-memory computing. Memory is cheap, you can have tera byte of data in memory.

    • If you can master bit-wise algo then it gives very good performance.

    • Use mechnical sympathy - Refer lmax disruptor, excellent framework

0 votes
answered Jul 12, 2014 by rdalmeida

If you are interested in Java low-latency developing, you should know that it can be done without a RTSJ (Real-Time) JVM provided that you keep the Garbage Collector under control. I suggest you take a look on this article that talks about Java Development without GC overhead. We also have many other articles in our site that talk about low-latency Java components.

Welcome to Q&A, where you can ask questions and receive answers from other members of the community.
Website Online Counter

...