5 ms·

it's pretty crazy what people are willing to do to make Python faster: Pyston, Numba, Cython, RipTable. Instead of just biting the bullet and adandoning Python,

by smabie 6y ago

it's pretty crazy what people are willing to do to make Python faster: Pyston, Numba, Cython, RipTable. Instead of just biting the bullet and adandoning Python, they spend thousands of manhours trying to hack a fundamentally slow and broken language into something that is acceptable. I see these hacks as a cognitive bias and rarely think they are the rational and correct decision. Just use Julia and call it a day. It really is that easy.

disgruntledphd2 6y ago
So, perhaps it makes sense in your industry to go all-in on Julia. Personally, I'm still burned by the time they announced 1.0 mere days after announcing 0.7, and thus meaning that literally nothing in the ecosystem worked at 1.0. If that transition had been managed better, then I would be much more comfortable evangelising Julia now, but it didn't and the decisions behind this make me incredibly uncomfortable in adopting it anywhere where I'm on the hook for failures. Additionally, Python has libraries for everything, and is known (badly at least), by almost everyone in Data Science/programming. When your modelling is a part of a larger project, this matters a lot (and is why I no longer write my production code in R). That being said, I really like Julia, and look forward to checking it out again, hopefully with less breakages in the future.
aldanor 6y ago
It's not the Python itself but the ecosystem and tooling around it, especially in machine learning / data science area. I find it strange that many people don't understand it and instead compare languages is. "Just use Julia" is not really an option in enterprise world either, when you already have hundreds of kloc, hundreds/thousands of packages and hundreds of people (not all of them professional developers, e.g. various analysts and quants) working on / using that on daily basis in production, plus all that code almost certainly depends on external packages (eg machine learning libraries) that are simply not available in Julia. For personal projects - sure, go use Julia or ditch it and go straight for Rust.
- smabie 6y ago
  They don't need to be available on Julia, calling Python libraries from Julia is incredibly painful. I'm not saying you should migrate your whole data analysis pipeline over, I'm saying you should write new services/analysis in Julia. There are production trading systems that are written in Python. And everyday, the quants and devs fruitlessly try and squeeze out more performance from the system instead of just accepting that Python is inappropriate.
  - aldanor 6y ago
    Well, this repo is definitely about data analysis pipelines and not about production trading systems, no one is arguing that writing live prod trading systems in Python is inappropriate. And in order to write "new analysis" in another language, you would need to have access to the ecosystem that's been already developed in Python (both internally and externally), even to just get as far as loading all the data from all the data sources you need. Also worth a mention, "Python" often really means "C++ wrapped in pybind or the like"; again, it's pretty hard to match if there's a developed ecosystem of C/C++ libraries and extensions.

by smabie 6y ago

disgruntledphd2 6y ago

So, perhaps it makes sense in your industry to go all-in on Julia. Personally, I'm still burned by the time they announced 1.0 mere days after announcing 0.7, and thus meaning that literally nothing in the ecosystem worked at 1.0. If that transition had been managed better, then I would be much more comfortable evangelising Julia now, but it didn't and the decisions behind this make me incredibly uncomfortable in adopting it anywhere where I'm on the hook for failures. Additionally, Python has libraries for everything, and is known (badly at least), by almost everyone in Data Science/programming. When your modelling is a part of a larger project, this matters a lot (and is why I no longer write my production code in R). That being said, I really like Julia, and look forward to checking it out again, hopefully with less breakages in the future.

aldanor 6y ago

It's not the Python itself but the ecosystem and tooling around it, especially in machine learning / data science area. I find it strange that many people don't understand it and instead compare languages is. "Just use Julia" is not really an option in enterprise world either, when you already have hundreds of kloc, hundreds/thousands of packages and hundreds of people (not all of them professional developers, e.g. various analysts and quants) working on / using that on daily basis in production, plus all that code almost certainly depends on external packages (eg machine learning libraries) that are simply not available in Julia. For personal projects - sure, go use Julia or ditch it and go straight for Rust.

smabie 6y ago
They don't need to be available on Julia, calling Python libraries from Julia is incredibly painful. I'm not saying you should migrate your whole data analysis pipeline over, I'm saying you should write new services/analysis in Julia. There are production trading systems that are written in Python. And everyday, the quants and devs fruitlessly try and squeeze out more performance from the system instead of just accepting that Python is inappropriate.
- aldanor 6y ago
  Well, this repo is definitely about data analysis pipelines and not about production trading systems, no one is arguing that writing live prod trading systems in Python is inappropriate. And in order to write "new analysis" in another language, you would need to have access to the ecosystem that's been already developed in Python (both internally and externally), even to just get as far as loading all the data from all the data sources you need. Also worth a mention, "Python" often really means "C++ wrapped in pybind or the like"; again, it's pretty hard to match if there's a developed ecosystem of C/C++ libraries and extensions.