3 ms·

A brief retrospective on transactional memory (2010)

24 points by luu 4y ago

signa11 4y ago
this paper "software transactional memory why is it only a research toy?" presents a different point of view, and imho, has proven to be largely correct.
- JonChesterfield 4y ago
  PDF at https://dl.acm.org/doi/pdf/10.1145/1454456.1454466?download=true https://dl.acm.org/doi/pdf/10.1145/1454456.1454466?download=...
  - int_19h 4y ago
    This seems to be arguing that the main problem with STM is performance? I don't think that's a particularly convincing argument in this day and age, when we had plenty of experience with concurrency-induced bugs on one hand, and that much more compute power on the other. End-to-end ST could give all the same safety guarantees as e.g. Rust, but packaged in a mental model that is easier to teach and much, much easier to write code for. Even if it means, say, a 10x slowdown, I think that there are many use cases where this would be acceptable if it meant significant improvements in safety. The problem is the end-to-end part. You don't just need STM - ideally, you need everything that the code can do to be transactional, including any other system that it interacts with in a way that causes a state change. This isn't feasible, so in practice you have to restrict STM to a subset of code which does not perform operations that cannot be rolled back. But with any existing platform, it ends up being such a small subset that it's not worth the bother; and even if you disregarded backwards compatibility and designed something from scratch specifically for end-to-end ST, it would still be limited wrt networking.
throwawaylinux 4y ago
> Throughout all of this, we searched and searched for the killer TM app. It’s unfair to pin this on TM, because the industry as a whole still searches for a killer concurrency app. If this means what I think it does, it's wrong. This was written in 2010. Concurrency was everywhere by then. I mean it was everywhere for many years in databases, web servers, parallel compiles, all sorts of high performance computation from supercomputers to that SETI program. But by 2010 even desktops and laptops had several cores. git could use several threads, so could web browsers, video games.
- mrkeen 4y ago
  To add to this: if you're going to throw away TM because you "can't find a killer concurrent app", you'd better throw away locks and other mutexes too.
JonChesterfield 4y ago
I'm under the impression that transactional memory works fine - clojure as an existence proof - but Intel's hardware implementations of it are all broken and disabled. So presumably it's a nice programming model with substantial performance overhead. edit: one of the hazards is exceptions, but the obvious fix of 'no non-local control flow' somehow doesn't seem to be on the table.
- dkersten 4y ago
  One thing I’ve realised after ~13 years of using Clojure is that I’ve never used its STM and have never actually seen other code use it. It works, but its usefulness is, to me, questionable. YMMV of course, nah just be the nature of code I’ve written or looked at.
- deleted 4y ago
  [deleted]
- int_19h 4y ago
  As the article explains, STM works great until you need to do something other than twiddling bits in your own process memory (basically any kind of I/O or IPC). Which, on one hand, happens to exclude most practical apps. And on the other hand, real-world tasks that require concurrent processing of in-memory data generally don't require much if any processing of shared data. So you're paying that very hefty performance price for something that you can't use most of the time.

24 points by luu 4y ago

this paper "software transactional memory why is it only a research toy?" presents a different point of view, and imho, has proven to be largely correct.

JonChesterfield 4y ago
PDF at https://dl.acm.org/doi/pdf/10.1145/1454456.1454466?download=true https://dl.acm.org/doi/pdf/10.1145/1454456.1454466?download=...
- int_19h 4y ago
  This seems to be arguing that the main problem with STM is performance? I don't think that's a particularly convincing argument in this day and age, when we had plenty of experience with concurrency-induced bugs on one hand, and that much more compute power on the other. End-to-end ST could give all the same safety guarantees as e.g. Rust, but packaged in a mental model that is easier to teach and much, much easier to write code for. Even if it means, say, a 10x slowdown, I think that there are many use cases where this would be acceptable if it meant significant improvements in safety. The problem is the end-to-end part. You don't just need STM - ideally, you need everything that the code can do to be transactional, including any other system that it interacts with in a way that causes a state change. This isn't feasible, so in practice you have to restrict STM to a subset of code which does not perform operations that cannot be rolled back. But with any existing platform, it ends up being such a small subset that it's not worth the bother; and even if you disregarded backwards compatibility and designed something from scratch specifically for end-to-end ST, it would still be limited wrt networking.

throwawaylinux 4y ago

> Throughout all of this, we searched and searched for the killer TM app. It’s unfair to pin this on TM, because the industry as a whole still searches for a killer concurrency app. If this means what I think it does, it's wrong. This was written in 2010. Concurrency was everywhere by then. I mean it was everywhere for many years in databases, web servers, parallel compiles, all sorts of high performance computation from supercomputers to that SETI program. But by 2010 even desktops and laptops had several cores. git could use several threads, so could web browsers, video games.

mrkeen 4y ago
To add to this: if you're going to throw away TM because you "can't find a killer concurrent app", you'd better throw away locks and other mutexes too.

JonChesterfield 4y ago

I'm under the impression that transactional memory works fine - clojure as an existence proof - but Intel's hardware implementations of it are all broken and disabled. So presumably it's a nice programming model with substantial performance overhead. edit: one of the hazards is exceptions, but the obvious fix of 'no non-local control flow' somehow doesn't seem to be on the table.

dkersten 4y ago
One thing I’ve realised after ~13 years of using Clojure is that I’ve never used its STM and have never actually seen other code use it. It works, but its usefulness is, to me, questionable. YMMV of course, nah just be the nature of code I’ve written or looked at.
deleted 4y ago
[deleted]
int_19h 4y ago
As the article explains, STM works great until you need to do something other than twiddling bits in your own process memory (basically any kind of I/O or IPC). Which, on one hand, happens to exclude most practical apps. And on the other hand, real-world tasks that require concurrent processing of in-memory data generally don't require much if any processing of shared data. So you're paying that very hefty performance price for something that you can't use most of the time.