4 ms·
There are many cases where the objects are too big to be passed around. Python is used a huge amount in Machine learning and datascience, where being able to do
by make3 9y ago
There are many cases where the objects are too big to be passed around. Python is used a huge amount in Machine learning and datascience, where being able to do parallel work on stuff already in memory would be great.
- leereeves 9y agoAre those applications often bottlenecked by the CPU, as opposed to GPU or data transfer?
- rspeer 9y agoThe world of algorithms that run well on a CPU is still much, much bigger than the world of algorithms that run well on a GPU, even in machine learning. And even if you're fortunate enough that Nvidia designs their GPUs to solve your problem, why should the CPU cores sit idle?
- dec0dedab0de 9y agoSo work on data that can not be broken down into smaller chunks? That makes sense, and is something I never come across.
- chrisseaton 9y agoI'm sure they can be broken down into smaller chunks, but is it more efficient if they aren't broken down and instead shared memory is used? If you want parallelism you're obviously already worried about performance.
- smaddox 9y agoCan't this already be handled by calling out to a C/C++ or FORTRAN procedure that processes the data in multiple threads? For number crunching, Python is almost exclusively used as glue.
- ams6110 9y agoToday's machine learning and data science students don't know how to code in those languages. They know python, and maybe java.
- orangejewce 9y agoDon't forget R... shudders
- sin7 9y agoWhat's wrong with R? I know it's not a programmers language but it's great for getting things done.
- foobarchu 9y agoYou CAN handle it, but why should you have to? If it's possible to remove that barrier, then it absolutely should be removed. If the only answer to a problem is "use another language", then the language in question has a limitation that needs to be addressed.
- Drdrdrq 9y agoIt is not a limitation at all in this case. Python is just a front to Tensorflow and similar libraries/frameworks so GIL doesn't matter there.
- deleted 9y ago[deleted]
- fulafel 9y agoSomething like the web worker primitives might work there (transferables & sharing read only data).