4 ms·
30k doesn't even sound like that much to me given Moore's law. I'd expect more improvement since 1989. Supercomputer performance increased more than a million s
by Aardwolf 3y ago
30k doesn't even sound like that much to me given Moore's law. I'd expect more improvement since 1989. Supercomputer performance increased more than a million since then
- ben_w 3y agoIt isn't much, but as the link says, the neural network they were reimplementing is too small to take advantage of modern hardware.
- swores 3y agoMy (wrong) intuition on reading your comment was that you were over-estimating the expected growth in performance over that time period, but actually after checking the maths based on Moore's Law, i.e. doubling every two years (though of course I understand that was a rough estimate, more of a concept prediction than expected to be precise) you're right so I'll share the maths for anyone else whose intuition might be as poor as mine: Doubling every 2 years = compound annual growth rate (CAGR) of ~41.42% CAGR = ((End Value / Start Value)^(1 / Number of Years)) - 1 ((2 / 1)^(1 / 2)) - 1 = 0.41421356237 Therefore in 34 years since then: 1 * (1 + 0.41421356237)^34 = ~131,072 So x30k is ~4.4x less than 131k. Then again, that's equivalent to ~x1.833 every two years, compared to Moore's Law of x2 every two years, so only ~8% less growth per two years, which coming back to the fact that Moore's Law is a rough estimate concept not an exact fact, doesn't seem to far off!
- jacquesm 3y agoThe rest of that difference can easily be explained by the difference in the class of hardware used. A desktop made today vs a laptop is roughly that factor 4. Not sure if back then there would have been laptops that you could have done this on for a more apples-to-apples comparison. Modern laptops give great efficiency, when I went for solar power here the first thing to go was the desktop computer. I still have it, but it hasn't run in over a year and the elderly thinkpad that is now my daily driver uses far less power and still has enough compute to serve my modest needs. But if I would dive into something requiring much more compute I'd have to start the desktop again. Unfortunately power management is not such that computers can really throttle down to 'miser mode' when you don't need it, it's a good step but not as good as the jump between desktop and laptop.
- swores 3y agoHadn't thought of that, good point
- DesiLurker 3y agoalso the 'memory wall', remember memory b/w did not grow at pace with moore's law. sure there are ways to mitigate it but that eat into chip budget and reflect when real world performance is calculated.
- jacquesm 3y agoYes, true and in a way that wall is still there. The way GPUs are limited in how much RAM they have because there is a way to sell you that memory at a multiple of the cost. Imagine a GPU with a 128G or even 256G slot based memory section that is sold unpopulated. 8 SODIMM slots or so.
- 2Gkashmiri 3y agoimagine that we are discussing "proving" a law with historical data from our POV but at the time, it must've seem like a theory at best or comical at the least. 8% less growth is not the point. The "law" has stood the test of time which says something about the guy and his vision
- fooker 3y agoAmdahl's law
- cma 3y ago33 years ago is 2000/1999