I do most my concurrent realtime analysis in C or Java. The biggest issue I run into with realtime analysis, I work in flow measurement systems so you may neve

I do most my concurrent realtime analysis in C or Java. The biggest issue I run into with realtime analysis, I work in flow measurement systems so you may never encounter this but communication errors. Having a value randomly flip from 50 to -30 just for one data point can throw you stand deviation, averages, etc. Straight out the window. And potentially ruin a solid test. What I find you want is a very threaded model. I normally just throw threads at the scheduler and let it sort out the details. Typically you want IO/error handling done in its own heavy thread for each from of IO. Or each source, Ethernet, DAQ card, etc. Luckily ethernet does most of this for you. Next your post-error processed IO should get sorted into something, normally a structure of some sort, and dumped into a generally read only structure. This structure is read by 2 threads. 1 logs it, 1 processes it further (before moving to another thread to be logged). This lets your processing be largely independent and have no IO slow down. Which is handy when your approaching ~20GB/hr+ of data streaming. Most of what you get, and generate internally doesn't have to be logged, unless you really like buying hard drives. Also by keeping these separate you can have multiple processing threads. But when you do be on guard. Haveing 1 IO -> N processing -> 1 IO will result in data arriving at the 2 be logged IO not in the order it was recieved. You will likely want a catch all thread sitting between logging and IO to sort the last ~500 items by time stamp, so they can be logged properly. TL;DR 1) Make sure you keep track of when it arrived log it in the same order 2) More threads are better then none. 3) Make sure communication errors don't occur (if your using TCP the OS does this for you).