Sorry for being vague, that wasn't our intention, so much as to focus on laying out the problem that we've experienced personally as algorithm developers. We wi

Sorry for being vague, that wasn't our intention, so much as to focus on laying out the problem that we've experienced personally as algorithm developers. We will be posting more blog posts with much more detail about the system in the near future. The data question is huge for us. We have built our system on top of Hadoop + Spark in order to handle large amounts of data, and be able to apply computations to it. You're absolutely right that data is very heavy, and often the limiting factor, so we are doing everything we can to both get data into the system (currently with a focus on streaming data, since that gets around the heavy data problem), and getting algorithms to where the data is. As far as specific algorithms, there are many that could apply. My personal experience is in machine learning algorithms, so I'm personally biased toward those. There are numerous algorithms like Classification, Clustering, Optimization, Anomaly Detection, etc. which are very CPU-intensive. We will be doing a blog post soon with a demo of live algorithms already in our system to give more concrete examples.