3 ms·
Yeah there is some startup time required to get to know the tools. In my experience it's worth it, though. I've seen some absolutely mind boggling spreadsheets
by kyzyl 14y ago
Yeah there is some startup time required to get to know the tools. In my experience it's worth it, though. I've seen some absolutely mind boggling spreadsheets trying to do what is a couple of lines of work with numpy/pandas. Spreadsheets are dead simple, but get unwieldy very quickly as your data & problem become more complex.
I suppose you're right, Quantopian could turn into a galaxy-like tool, but I think there is a great deal of experimentation that goes on in finance that doesn't fit into the pipeline model very well. As well, a lot of the fitting routines run in finance are quite computationally intensive and wouldn't really work on a framework like quantopian. For example, good luck hooking your risk model into a recurrent neural network on quantopian. That isn't to say that bioinf. is not computationally bound, of course it is, but a lot of what's done is more oriented to data pipelines as opposed to fancy computations (correct me if I'm wrong), which is why a thing like Galaxy fits so well.
- rglullis 14y agoOh, genomics can be very demanding in their computating power requirements. We had a few applications with the running time reported in days/weeks. But still... with all the talk about cloud computing and PaaS, this is the only real use-case where I see a benefit in tying up a platform to something like EC2. In the case of Galaxy, the developers provided an AMI that you could deploy and use your own account. For the Quantopian case, one of their services could be exactly in doing the managing and scaling of these tools. So, the neural network would be a tool that you'd have access to, and you would define how much power should be allocated when running it.