4 ms·

Those teams still wouldn't switch because they wouldn't gain any performance from switching. After all CPython is just calling highly optimized, already compile

• lumost 5y ago
  many/most datascience processes end up slowing down when inevitably the data must move back to python or a python function must be invoked on some data. A significant performance improvement in python would benefit many ds related tasks.
  • Filligree 5y ago
    Very much this. Anyone who does machine learning will notice their CPU sitting at 100% of one core a significant fraction of the time. Doesn't matter how fast a GPU you have; Python and the GIL is the bottleneck.
  • nerdponx 5y ago
    This is very true, especially when pre-processing text and other unstructured data. It ends up being a lot of loops, string manipulation, and dict lookups. Fortunately, with a tool like DVC or even Make, you usually don't have to (or want to) put that code in the same script as the actual machine learning part. So you can theoretically run the former with PyPy and the latter with CPython, if you really need to maximize both.
• agravier 5y ago
  I would, I find that there are lots of data transformations and non-deep modelling happening in python still. E.g string processing, Json munching, business rules if-this-then-remove, etc.