5 ms·

Are there any real world examples (with actual measurement) of languages where map uses hardware parallelisation to be faster than a c loop? Serious question!

by dzdt 10y ago

Are there any real world examples (with actual measurement) of languages where map uses hardware parallelisation to be faster than a c loop? Serious question! My impression is, no, outside of something like hadoop.

lallysingh 10y ago
https://donsbot.wordpress.com/2007/11/29/use-those-extra-cores-and-beat-c-today-parallel-haskell-redux/ https://donsbot.wordpress.com/2007/11/29/use-those-extra-cor... This was a simple fib(), not a map(), but I think the point still stands pretty well.
kutkloon7 10y ago
FPGA implementations usually are made to do this, although there is usually a human writing dedicates VHDL. Also google course-grained reconfigurable architectures (which can sometimes be mapped automattically from code).
deleted 10y ago
[deleted]
kstrauser 10y ago
I don't have any off the top of my head, but I used a forking map in Python where the inner call was an image processing operation, and it was approximately #cores faster. I think it'd be enormously dependent on workload: if the inner operation is adding two ints, the gains might be lost in the parallelization overheard. Map is just one example of a high-level construct that permits faster code, though. A lot of us here have done async IO in C because that was the option we had available to us for doing it, but I'd vastly rather do such stuff in Python (3), Go, or Node where the language itself gives you constructs for describing what you're trying to accomplish. My impression is, no, outside of something like hadoop. That's a pretty big exclusion, though! We use Hadoop-like stuff on, ahem, sizable quantities of data. That's not exactly an edge case.
Ono-Sendai 10y ago
Here's an example (something of a proof-of-concept/toy example though) Auto-vectorisation and parallelisation in Winter: http://www.forwardscattering.org/post/22 http://www.forwardscattering.org/post/22
pjmlp 10y ago
Check the now dead Connection Machine and *Lisp.

by dzdt 10y ago

lallysingh 10y ago

https://donsbot.wordpress.com/2007/11/29/use-those-extra-cores-and-beat-c-today-parallel-haskell-redux/ https://donsbot.wordpress.com/2007/11/29/use-those-extra-cor... This was a simple fib(), not a map(), but I think the point still stands pretty well.

kutkloon7 10y ago

FPGA implementations usually are made to do this, although there is usually a human writing dedicates VHDL. Also google course-grained reconfigurable architectures (which can sometimes be mapped automattically from code).

deleted 10y ago

[deleted]

kstrauser 10y ago

I don't have any off the top of my head, but I used a forking map in Python where the inner call was an image processing operation, and it was approximately #cores faster. I think it'd be enormously dependent on workload: if the inner operation is adding two ints, the gains might be lost in the parallelization overheard. Map is just one example of a high-level construct that permits faster code, though. A lot of us here have done async IO in C because that was the option we had available to us for doing it, but I'd vastly rather do such stuff in Python (3), Go, or Node where the language itself gives you constructs for describing what you're trying to accomplish. My impression is, no, outside of something like hadoop. That's a pretty big exclusion, though! We use Hadoop-like stuff on, ahem, sizable quantities of data. That's not exactly an edge case.

Ono-Sendai 10y ago

Here's an example (something of a proof-of-concept/toy example though) Auto-vectorisation and parallelisation in Winter: http://www.forwardscattering.org/post/22 http://www.forwardscattering.org/post/22

pjmlp 10y ago

Check the now dead Connection Machine and *Lisp.