4 ms·

Kay starts off by citing Moore's law, but misses that Moore's Law hasn't been kind to Smalltalk. Specifically, CPU power has grown at a faster rate than memory

Kay starts off by citing Moore's law, but misses that Moore's Law hasn't been kind to Smalltalk. Specifically, CPU power has grown at a faster rate than memory bandwidth, to the point that going to main memory is hundreds of times slower than the L1 cache. I believe the pendulum is swinging back from dynamic languages to compiled languages with stronger static guarantees. Languages that are parsimonious with memory and give you control over the layout when you need it. Don't get me wrong. Smalltalk is a hugely influential language and we're better for having it, but it was designed in an era where hardware had very different performance characteristics from today's hardware.

xfer 9y ago
Citing performance (especially related to micro-optimization) everytime we discuss language design is not a right thing to do. Most applications don't require or benefit from such micro-optimization(tight control of layout). > but it was designed in an era where hardware had very different performance characteristics from today's hardware. Exactly, the era of 256k memory computers are over. Programmer time and correctness is more valuable than making sure everything fits in L1 cache.
- nickm12 9y ago
  You're missing my point. When we had 256k memory computers we didn't have caches because memory access was uniform. As time goes on, fitting in cache is getting more important, not less.
- loup-vaillant 9y ago
  The difference between random access and linear access is easily 10x, possibly more. Given how much time your average huge application (Word, Photoshop…) takes to even start up, I think this is still an important point. Programmer time is important, but so is user time. And in many cases, there are many more users than programmers.
davelnewton 9y ago
There's a big difference between Smalltalk implementations being relatively slow and Smalltalk being intrinsically slow. If Smalltalk VMs had the same resources thrown at them as other VMs/compilers/etc. they could be more competitive speed-wise. Noting, of course, that Smalltalk systems can be plenty fast-enough. Smalltalk-as-glue is also perfectly reasonable, interfacing with multiple back-end systems, controlling business logic, etc.
- nickm12 9y ago
  What I'm saying is that the "lots of little objects" model of Smalltalk goes against the grain of modern machines and how they access memory. I'm not sure what it takes for something to be _intrinsically slow_ but it appears physics is against us. If we could build machines that had fast uniform access to memory we probably would have.
  - fulafel 9y ago
    I think the current mindset of taking data layout as fixed is a big illusion that we just haven't broken out of. Optimizers should make data layout optimizations. For example, a JIT could do AoS->SoA optimizations on an array containing multi-field objects and vectorize loops that manipulate a small subset of the fields. Or replace object representations with limited, live data-specialized versions that consume a fraction of the space. There may be some changes to language semantics needed of course, but doesn't preclude Smalltalky languages. Another property of the current software/hardware world is that the vast majority of programs don't care about performance at this level. There is a vast size difference between the oceans of production code in the world, and the amount of "hot" code that people pay money to make faster. (And the levels of "make faster" are usually more low hanging fruit than switch of programming language)
  - abecedarius 9y ago
    Consider Concurrent Smalltalk on the http://cva.stanford.edu/projects/j-machine/ http://cva.stanford.edu/projects/j-machine/ -- a processor-in-memory massively parallel architecture. This seems like a better match to physics than a Von Neumann architecture with a few fast processors separated from a giant memory, because physics is local. I suspect systems like the J-machine haven't gotten very far for historical path-dependent reasons more than intrinsic ones. It's true though that a parallel object language like Concurrent Smalltalk isn't the historical Smalltalk we usually mean.
fulafel 9y ago
To be fair, the poor "memory bandwidth" of CPUs has not been fought very hard and is not really a direct consequence of Moore's law. Constantly growing caches just work very well, and we've grown to treat the DRAM level of the memory hierarchy as just another storage device, faster than your SSD but slower than your L4 cache. The industry has just emphasized capacity/$ and traditional, commodity DRAM protocols. Consider for example the potential, and near-absence[1], of CPU + DRAM MCMs. [1] The latest rebranding of this obvious idea is "HBM" in GPUs. Though these don't really take advantage of the idea. But still CPUs have not been interested in even reaching this little distance.

xfer 9y ago

Citing performance (especially related to micro-optimization) everytime we discuss language design is not a right thing to do. Most applications don't require or benefit from such micro-optimization(tight control of layout). > but it was designed in an era where hardware had very different performance characteristics from today's hardware. Exactly, the era of 256k memory computers are over. Programmer time and correctness is more valuable than making sure everything fits in L1 cache.

nickm12 9y ago
You're missing my point. When we had 256k memory computers we didn't have caches because memory access was uniform. As time goes on, fitting in cache is getting more important, not less.
loup-vaillant 9y ago
The difference between random access and linear access is easily 10x, possibly more. Given how much time your average huge application (Word, Photoshop…) takes to even start up, I think this is still an important point. Programmer time is important, but so is user time. And in many cases, there are many more users than programmers.

davelnewton 9y ago

There's a big difference between Smalltalk implementations being relatively slow and Smalltalk being intrinsically slow. If Smalltalk VMs had the same resources thrown at them as other VMs/compilers/etc. they could be more competitive speed-wise. Noting, of course, that Smalltalk systems can be plenty fast-enough. Smalltalk-as-glue is also perfectly reasonable, interfacing with multiple back-end systems, controlling business logic, etc.

nickm12 9y ago
What I'm saying is that the "lots of little objects" model of Smalltalk goes against the grain of modern machines and how they access memory. I'm not sure what it takes for something to be _intrinsically slow_ but it appears physics is against us. If we could build machines that had fast uniform access to memory we probably would have.
- fulafel 9y ago
  I think the current mindset of taking data layout as fixed is a big illusion that we just haven't broken out of. Optimizers should make data layout optimizations. For example, a JIT could do AoS->SoA optimizations on an array containing multi-field objects and vectorize loops that manipulate a small subset of the fields. Or replace object representations with limited, live data-specialized versions that consume a fraction of the space. There may be some changes to language semantics needed of course, but doesn't preclude Smalltalky languages. Another property of the current software/hardware world is that the vast majority of programs don't care about performance at this level. There is a vast size difference between the oceans of production code in the world, and the amount of "hot" code that people pay money to make faster. (And the levels of "make faster" are usually more low hanging fruit than switch of programming language)
- abecedarius 9y ago
  Consider Concurrent Smalltalk on the http://cva.stanford.edu/projects/j-machine/ http://cva.stanford.edu/projects/j-machine/ -- a processor-in-memory massively parallel architecture. This seems like a better match to physics than a Von Neumann architecture with a few fast processors separated from a giant memory, because physics is local. I suspect systems like the J-machine haven't gotten very far for historical path-dependent reasons more than intrinsic ones. It's true though that a parallel object language like Concurrent Smalltalk isn't the historical Smalltalk we usually mean.

fulafel 9y ago

To be fair, the poor "memory bandwidth" of CPUs has not been fought very hard and is not really a direct consequence of Moore's law. Constantly growing caches just work very well, and we've grown to treat the DRAM level of the memory hierarchy as just another storage device, faster than your SSD but slower than your L4 cache. The industry has just emphasized capacity/$ and traditional, commodity DRAM protocols. Consider for example the potential, and near-absence[1], of CPU + DRAM MCMs. [1] The latest rebranding of this obvious idea is "HBM" in GPUs. Though these don't really take advantage of the idea. But still CPUs have not been interested in even reaching this little distance.