3 ms·
because it turns out that optimizing performance of a programming language designed for use-cases where runtime performance doesn't matter ... doesn't matter
by semiinfinitely 1y ago
because it turns out that optimizing performance of a programming language designed for use-cases where runtime performance doesn't matter ... doesn't matter
- lucb1e 1y agoMight as well take the work that's already done though? I can't think of a logical reason why you'd want to run it at potentially half the speed (depending on the hot code specifics how much, if any, speedup you get of course)
- morganherlocker 1y agoThere's currently talk of adding gigawatts of data center capacity to the grid just for use cases where python dominates development. While a lot of that will be compiled into optimized kernels on CPU or GPU, it only takes a little bit of 1000x slower code to add up to a significant chunk of processing time at training or inference time.
- deleted 1y ago[deleted]
- MobiusHorizons 1y agoWhat percentage of the CPU cycles are actually spent running Python though? My impression is _very_ low in production LLM workloads. I think significantly less than 1%. There are almost certainly better places to spend the effort, and if it did matter, I think they would replace Python with something like C++ or Rust.