3 ms·

My reaction to this is that I am sure he is right that embracing nondeterminism and relaxing synchronization will allow us to scale up to these many core PCs. H

My reaction to this is that I am sure he is right that embracing nondeterminism and relaxing synchronization will allow us to scale up to these many core PCs. However, this does not solve the problem of how 'the rest of us' program them effectively. One of the biggest challenges I see practically are that concurrency and parallelism are the hardest thing to push down under a layer of abstraction. These effects leak out all the way through your application (and even into other applications). It strikes me that this kind of non-deterministic approach will leak everywhere. And then who can program in this way? Who among us can reason effectively about just how far from the true answer we are. If this is the only way we can really harness the many-many-core PCUs of the future then we will have the change the way we make software. The IBM approach of armies of programmers will not work. As much as I hate this approach to software it is the way most software (at least in the enterprise) is made. Will we go back to the long beards? Or will we end up in even more aggressively restrictive frameworks that work harder to prevent us from doing this wrong. If this approach to parallelism turns out to be important I hope that the problem of reasoning about these types of program is tractible. To push well understood software development even further away from the reach of most programmers. If we all end up having to learn a bit of statistics that wouldn't be a bad thing.

deleted 15y ago
[deleted]
mooism2 15y ago
> And then who can program in this way? Who among us can reason effectively about just how far from the true answer we are. If floating point math is anything to go by, most programmers won't be able to cope with this. (Then again, most languages do nothing to help programmers keep track of how large the inaccuracies get.)
- fmstephe 15y ago
  An interesting point. I suppose if this becomes a really important technique we may see some very interesting language support. Fun times :)
yummyfajitas 15y ago
When you write concurrent software, you typically build two kinds of components. The first are the cogs. These are typically either immutable data structures, or mutable components designed to be accessed only from a single thread. The second is a concurrent abstraction which sits on top of the cogs. For example, in my project Hobo, I've got an immutable, read-only index with procedural, deterministic methods. Then sitting in front of it I've got a TThreadPoolServer which handles multithreaded access. Single threaded: https://github.com/stucchio/Hobo/blob/master/src/org/styloot/hobo/hoboindex/SimpleHoboIndex.java https://github.com/stucchio/Hobo/blob/master/src/org/styloot... Library handles multithreaded part: https://github.com/stucchio/Hobo/blob/master/src/org/styloot/hobo/thriftserver/Server.java https://github.com/stucchio/Hobo/blob/master/src/org/styloot... Another thing that will work for data where contention is rare is software transactional memory. Most programmers are already familiar with SQL transactions, I don't see a reason they can't use it in memory as well.
- fmstephe 15y ago
  "The first are the cogs. These are typically either immutable data structures, or mutable components designed to be accessed only from a single thread. The second is a concurrent abstraction which sits on top of the cogs." This is a very good illustration of how concurrency potentially leaks out across an entire application. You can see here that your concerns for concurrency appears in the cogs. It is certainly true that you have provided a layer of abstraction but only for a certain concurrency model, and your cogs will reflect that in their implementation. I would also note that I wouldn't claim that all concurrency problems pose these kinds of problem. Graphics pipelines being the easiest (and best abstracted) example. But in general for the day to day code we write, concurrency concerns have a habit of being very hard to tidy away. For a really good treatment on the virtues and limitations of transactional memory there is a great conversation between Rich Hickey and Cliff Click on the clojure mailing list. I can't find the original, but here is a good cleaned up summary of the exchange. http://blogs.azulsystems.com.sharedcopy.com/cliff/2008/05/97fd3e9e643ee6fa1009713c970381d7.html http://blogs.azulsystems.com.sharedcopy.com/cliff/2008/05/97...
  - yummyfajitas 15y ago

deleted 15y ago

[deleted]

mooism2 15y ago

> And then who can program in this way? Who among us can reason effectively about just how far from the true answer we are. If floating point math is anything to go by, most programmers won't be able to cope with this. (Then again, most languages do nothing to help programmers keep track of how large the inaccuracies get.)

fmstephe 15y ago
An interesting point. I suppose if this becomes a really important technique we may see some very interesting language support. Fun times :)

yummyfajitas 15y ago

When you write concurrent software, you typically build two kinds of components. The first are the cogs. These are typically either immutable data structures, or mutable components designed to be accessed only from a single thread. The second is a concurrent abstraction which sits on top of the cogs. For example, in my project Hobo, I've got an immutable, read-only index with procedural, deterministic methods. Then sitting in front of it I've got a TThreadPoolServer which handles multithreaded access. Single threaded: https://github.com/stucchio/Hobo/blob/master/src/org/styloot/hobo/hoboindex/SimpleHoboIndex.java https://github.com/stucchio/Hobo/blob/master/src/org/styloot... Library handles multithreaded part: https://github.com/stucchio/Hobo/blob/master/src/org/styloot/hobo/thriftserver/Server.java https://github.com/stucchio/Hobo/blob/master/src/org/styloot... Another thing that will work for data where contention is rare is software transactional memory. Most programmers are already familiar with SQL transactions, I don't see a reason they can't use it in memory as well.

fmstephe 15y ago
"The first are the cogs. These are typically either immutable data structures, or mutable components designed to be accessed only from a single thread. The second is a concurrent abstraction which sits on top of the cogs." This is a very good illustration of how concurrency potentially leaks out across an entire application. You can see here that your concerns for concurrency appears in the cogs. It is certainly true that you have provided a layer of abstraction but only for a certain concurrency model, and your cogs will reflect that in their implementation. I would also note that I wouldn't claim that all concurrency problems pose these kinds of problem. Graphics pipelines being the easiest (and best abstracted) example. But in general for the day to day code we write, concurrency concerns have a habit of being very hard to tidy away. For a really good treatment on the virtues and limitations of transactional memory there is a great conversation between Rich Hickey and Cliff Click on the clojure mailing list. I can't find the original, but here is a good cleaned up summary of the exchange. http://blogs.azulsystems.com.sharedcopy.com/cliff/2008/05/97fd3e9e643ee6fa1009713c970381d7.html http://blogs.azulsystems.com.sharedcopy.com/cliff/2008/05/97...
- yummyfajitas 15y ago