4 ms·
Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python ove
by stephc_int13 6y ago
Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++).
I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...
- andrew_v4 6y agoPython (in common ML frameworks) is really just wrapping well optimized native code. The overhead is very minimal. The advantage here is about easy access to the internals. I don't know that any material speedup is expected just because its written in C++
- stephc_int13 6y agoI've worked at some point on a commercial game engine written in C++ at the core but with many Lua components and API, for convenience. The reasoning wat the same, all the heavy lifting done with fast native code, and everything written in Lua was mostly glue code without real performance impact. Turns out the engine was slow and difficult to maintain because of the many interfaces. They ditched it a few years later... I think you could say the same thing about javascript on the web, this is mostly glue code, all the heavy lifting is done by optimized C++ or Rust code in the browser. And yet it is difficult to run a web mail client if your computer is a bit old... This is probably related in some ways to Amdahl's law https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
- pjmlp 6y agoThat was the same reasoning why Unreal dropped UnrealScript. https://www.gamasutra.com/view/news/213647/Epics_Tim_Sweeney_lays_out_the_case_for_Unreal_Engine_4.php https://www.gamasutra.com/view/news/213647/Epics_Tim_Sweeney... However a couple of years later and Epic is bringing scripting back with Verse https://twitter.com/saji8k/status/1339709691564179464?s=20 https://twitter.com/saji8k/status/1339709691564179464?s=20 Because the problem is not the scripting, but how it is done. I imagine that the engine you mention did not use any kind of compiler for Lua, e.g. LuaJIT, nor batch calls across the marshalling layer. Same happens on the Web, JavaScript JIT can outperform WebAssembly if one keeps switching all the time between layers, instead of doing batch requests.
- stephc_int13 6y agoThe engine used LuaJIT, but this is not available on iOS. The code was a mess, absolutely no batching, quite the contrary.
- Joky 6y agoPython isn't really driving the compute intensive part of ML actually, whether it's JAX, PyTorch, or TensorFlow the code is really mostly native. Convolution are implemented by hand in highly optimized libraries (Intel MKL-DNN, Nvidia cuDNN) and the Python glue is really just a light "dispatcher". A lot of it is also asynchronous for performance: the Python code just enqueues more work to a queue which some native C++ code processes. For TensorFlow the Python code traces an entire computation graph that is stored a protobuf and then executed by a C++ native stack, potentially remotely/distributed. Serving ML with TensorFlow does not involve any Python code in many scenarios. Python is still quite useful for scientist to quickly glue everything together, and to describe their dataset, or when they collect result and need to produce graphs or other data analyses.
- Kranar 6y agoI work at a quant firm and we use Python because of how painful it is to build things in C++. Our framework is built in C++ but we then expose all of it to Python using pybind11 (amazing library). Most quants do not want to learn complex build systems that have quirky behavior on different platforms, wait for very long compile times when making small changes to the code, dense and incomprehensible error messages, and a host of painful problems that one has to consider when writing C++. Python just works really well on almost all platforms. The biggest downside of Python is its parallelism, which means there is a lot of hackyness around writing parallel code. In most cases we can break things down and run different tests independently of one another, but in many other cases we have to make use of awkward workarounds, use multiprocessing, and other tricks.
- deleted 6y ago[deleted]
- marcinzm 6y agoPython is basically a configuration file for the native code that actually does the machine learning. Python is easier to write than native code or most other complex configuration formats (see terraform issues).
- cma 6y agoNotebooks may also be a big player, being able to visualize experimental changes rapidly.