2 ms·
Python software is to a large extent either doing things in not-python (c, c++, rust, etc.) or doing things that are not cpu bound (io bound, async, etc.). If y
by bob001 4mo ago
Python software is to a large extent either doing things in not-python (c, c++, rust, etc.) or doing things that are not cpu bound (io bound, async, etc.). If you're cpu bound then you can either take a 2x jit improvement or take a 10x non-python improvement. There's few companies of a scale where the non-hot path cost of 2x cpu is so massive as to be worth caring about.
- thatguysaguy 4mo agoThe python overhead of launching big ML jobs is nontrivial, so I think speeding that up would be meaningful. (I mean the initial tracing and other setup, not things once the GPUs are actually doing the work).
- bob001 4mo agoThat seems more like the tracing overhead than the python overhead. The original jit proposal would I believe not help at all with that since ML workloads basically do their own jit. The post being discussed however pushes for a general framework and good tracing support so might help ML workloads.
- jbvlkt 4mo agoAlso you can use projects like numba https://numba.pydata.org/ https://numba.pydata.org/