3 ms·

Interesting that the clojure stuff is slower than python. From what I gather most clojure stuff never actually gain any of the possible multi-core performance b

Interesting that the clojure stuff is slower than python. From what I gather most clojure stuff never actually gain any of the possible multi-core performance boost that is one of promises of the language. Not that I don't like clojure, but part of the hype is that it's supposed to do a lot of the scaling inherently just by the design of the language. But in reality it's never that simple. Switching from map to pmap doesn't do much, actually a whole different way of writing the code is necessary. I'd like to see a version of this that tries to maximise multi core performance, and see what a 12-core mac pro can do with the execution times.

nickik 16y ago
> Interesting that the clojure stuff is slower than python. From what I gather most clojure stuff never actually gain any of the possible multi-core performance boost that is one of promises of the language. Clojure always sad its designed for CONCURENCY and he said that Clojure has great primitives for parallism. Most people just dont know the diffrence. > Not that I don't like clojure, but part of the hype is that it's supposed to do a lot of the scaling inherently just by the design of the language. That wasn't a designgoal of clojure to provid parallism just out of box. It maybe is a designgoal now because everybody wants it to be. > But in reality it's never that simple. Switching from map to pmap doesn't do much, actually a whole different way of writing the code is necessary. I'd like to see a version of this that tries to maximise multi core performance, and see what a 12-core mac pro can do with the execution times. pmap does work great for some things but just thinking you can throw in some 'p'-s want wrok. Rich or anybody else never claimed that this will be possible he infact always said that it wan't but people in the internet just dont listen that well.
- Luyt 16y ago
  Clojure always said its designed for concurrency and he said that Clojure has great primitives for parallelism. Most people just dont know the difference. Well, here's the difference (according to Wikipedia): Parallelism - a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). There are several different forms of parallel computing: bit-level, instruction level, data, and task parallelism. Parallelism has been employed for many years, mainly in high-performance computing, but interest in it has grown lately due to the physical constraints preventing frequency scaling. Concurrency - a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. The computations may be executing on multiple cores in the same chip, preemptively time-shared threads on the same processor, or executed on physically separated processors. A number of mathematical models have been developed for general concurrent computation including Petri nets, process calculi, the Parallel Random Access Machine model and the Actor model.
  - scott_s 16y ago
    I would remove the word "simultaneously" from the concurrency definition. Two threads can be executing concurrently on the same processor - strangely, the next sentence seems to acknowledge this. My personal definitions, based on my experience in systems and high performance computing research: Parallelism: simultaneous calculations executing in the service of a single problem, usually with the goal of improved performance. Concurrency: executions in the same time granularity, but not necessarily simultaneous. Also not necessarily in the service of the same problem, but some form of synchronization is required.
- tmountain 16y ago
  Having hacked a decent amount of Clojure, I've learned that Clojure can be very fast or very slow depending on which features of the language one chooses to take advantage of. A perusal of the Clojure Google group will illustrate many examples where a few small changes to a program give an order of magnitude performance boost (or penalty for that matter). This makes it somewhat difficult to write performant code in Clojure (for me at least) as a lot of experimentation is required. Additionally, a given program has to run for at least a little while before hotspot JVM optimizations kick in which make a huge difference.
swannodette 16y ago
It's a simple port that avoids mutation entirely. The fact that it's only slightly slower than the Python version says a lot about Clojure performance. In anycase, the OP was clear, the goal was to write a comparison of the original design, not to write the fastest Clojure program (and there are many ways to make the Clojure version faster, yet it's not clear to me how to best Norvig's design in pure Python).
fogus 16y ago
but part of the hype The gulf between hype and reality rarely converges. One of Clojure's primary goals is to allow sane mutation and out of that falls a facilitation of concurrency and parallelism. However, it does not solve the problem of effectively utilizing multicore processors automatically. How would it ever do that without programmer intervention?
- sunkencity 16y ago
  exactly.

nickik 16y ago

> Interesting that the clojure stuff is slower than python. From what I gather most clojure stuff never actually gain any of the possible multi-core performance boost that is one of promises of the language. Clojure always sad its designed for CONCURENCY and he said that Clojure has great primitives for parallism. Most people just dont know the diffrence. > Not that I don't like clojure, but part of the hype is that it's supposed to do a lot of the scaling inherently just by the design of the language. That wasn't a designgoal of clojure to provid parallism just out of box. It maybe is a designgoal now because everybody wants it to be. > But in reality it's never that simple. Switching from map to pmap doesn't do much, actually a whole different way of writing the code is necessary. I'd like to see a version of this that tries to maximise multi core performance, and see what a 12-core mac pro can do with the execution times. pmap does work great for some things but just thinking you can throw in some 'p'-s want wrok. Rich or anybody else never claimed that this will be possible he infact always said that it wan't but people in the internet just dont listen that well.

Luyt 16y ago
Clojure always said its designed for concurrency and he said that Clojure has great primitives for parallelism. Most people just dont know the difference. Well, here's the difference (according to Wikipedia): Parallelism - a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). There are several different forms of parallel computing: bit-level, instruction level, data, and task parallelism. Parallelism has been employed for many years, mainly in high-performance computing, but interest in it has grown lately due to the physical constraints preventing frequency scaling. Concurrency - a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. The computations may be executing on multiple cores in the same chip, preemptively time-shared threads on the same processor, or executed on physically separated processors. A number of mathematical models have been developed for general concurrent computation including Petri nets, process calculi, the Parallel Random Access Machine model and the Actor model.
- scott_s 16y ago
  I would remove the word "simultaneously" from the concurrency definition. Two threads can be executing concurrently on the same processor - strangely, the next sentence seems to acknowledge this. My personal definitions, based on my experience in systems and high performance computing research: Parallelism: simultaneous calculations executing in the service of a single problem, usually with the goal of improved performance. Concurrency: executions in the same time granularity, but not necessarily simultaneous. Also not necessarily in the service of the same problem, but some form of synchronization is required.
tmountain 16y ago
Having hacked a decent amount of Clojure, I've learned that Clojure can be very fast or very slow depending on which features of the language one chooses to take advantage of. A perusal of the Clojure Google group will illustrate many examples where a few small changes to a program give an order of magnitude performance boost (or penalty for that matter). This makes it somewhat difficult to write performant code in Clojure (for me at least) as a lot of experimentation is required. Additionally, a given program has to run for at least a little while before hotspot JVM optimizations kick in which make a huge difference.

swannodette 16y ago

It's a simple port that avoids mutation entirely. The fact that it's only slightly slower than the Python version says a lot about Clojure performance. In anycase, the OP was clear, the goal was to write a comparison of the original design, not to write the fastest Clojure program (and there are many ways to make the Clojure version faster, yet it's not clear to me how to best Norvig's design in pure Python).

fogus 16y ago

but part of the hype The gulf between hype and reality rarely converges. One of Clojure's primary goals is to allow sane mutation and out of that falls a facilitation of concurrency and parallelism. However, it does not solve the problem of effectively utilizing multicore processors automatically. How would it ever do that without programmer intervention?

sunkencity 16y ago
exactly.