3 ms·
>A single modern games console has more RAM than all of the Atari 2600s ever manufactured put together In other mind boggling movie lore, an RTX 4090 has 2x th
by FirmwareBurner 3y ago
>A single modern games console has more RAM than all of the Atari 2600s ever manufactured put together
In other mind boggling movie lore, an RTX 4090 has 2x the TFLOPS computing power than the killer AI Skynet used in Terminator 3[1].
The writers back then probably though 60 TFLOPS is such a ridiculously high sci-fi number for the world-ending AI, that nothing could possibly come close to it, and 20 years later consumers can have twice more computing power in their home PCs.
It's also a nice reminder how far technology has progressed in the last decades even if the pace has slowed down in the last years.
[1]https://youtu.be/_Wlsd9mljiU?t=155 https://youtu.be/_Wlsd9mljiU?t=155
- pdimitar 3y agoThe writers could still turn out to be right. I am not sure we are making good use of all that hardware yet.
- MegaSec 3y agoDamm high level programming languages. Just go back to assembly, that'll fix everything.
- pdimitar 3y agoYeah how dare they. ;) Truth be told though, I believe we are in for some more innovation in the area, especially with the advent of ARM lately. It's always kinda funny how these mega-machines we have still manage to stutter.
- SmellTheGlove 3y ago> It's always kinda funny how these mega-machines we have still manage to stutter. I just figured that’s the trade-off for general purpose computing. We can optimize for whatever covers the wide swath of use cases, but we can’t optimize for everything, and some will continue to be mutually exclusive. Mind you I’m no expert, I’m just extrapolating on how differently what CPUs and GPUs are optimized for these days and historically.
- pdimitar 3y agoNah you are very correct, I just feel that our compilers can still do a better job with the general purpose code because they are usually completely blind to the systems the compiled code runs in (as in, I/O takes orders of magnitude more time for example and this can be used to auto-parallelize code; but I know compiler authors will never auto-spawn threads). I feel this can be improved a lot but for various (and likely good) reasons our tooling is not as good as it can be.
- SmellTheGlove 3y agoGood points. I wish I knew more about compilers sometimes, and this is one of those times.
- automatic6131 3y ago>I am not sure we are making good use of all that hardware yet. Dunno, working out the color of 8 million pixels every 6ms seems pretty good to me
- pdimitar 3y agoTrue, though I was talking about the AI workloads.
- koolba 3y agoThe only thing keeping us alive is that skynet is an electron app.
- iamgopal 3y agoIsn’t it a great reminder that technology is not progressed enough to even take advantage of 60TFLOPS.
- FirmwareBurner 3y ago> to even take advantage of 60TFLOPS Rendering Electron apps and mining Dodgecoins?
- segfaultbuserr 3y agoIn scientific computing, it has become a serious problem. Because of the memory wall, many important algorithms can never take advantage of 60 TFLOPS due to their low arithmetic intensity. The only solutions are (1) stop using these algorithms, (2) stop using von Neumann computers (e.g. in-memory computing). The stop-gap solution is HBM or 3D-VCache.
- CyberDildonics 3y agoIn other mind boggling movie lore, an RTX 4090 has 2x the TFLOPS computing power than the killer AI Skynet used in Terminator 3[1]. That isn't really mind boggling since you are quoting fiction.
- pests 3y agoIt's not fiction that the writers thought 60TFLOPS would be huge today.
- CyberDildonics 3y agoSo what? It's a made up number that's supposed to sound fancy. It is for people who don't know much about computers. It's probably just there because people have heard the prefix 'tera', but wouldn't know what 'exa' or any other prefix means. It doesn't mean anything. Documentation made by people having more pages than a CPU which was also made by people is interesting because these are real things made for specific purposes, not a number pulled out of thin air for fiction. There is nothing 'mind blowing' about an uninformed person just being wrong. Is it 'mind blowing' that the original terminator was supposed to run on a 6502? In Johnny Mnemonic 320 GB was supposed to be a lot of data in 2021 when it costs the same as lunch for two people. https://www.imdb.com/title/tt0113481/plotsummary/ https://www.imdb.com/title/tt0113481/plotsummary/
- 7speter 3y agoIt kinda seems like the writers (writer?) either consulted with or did the math and calculated where pretty powerful computers would be by now, and that the t800 was more of a mid tier model and that there were higher tier models (or ai’s that ran in data centers) that individually ran on 4090 power and above
- FirmwareBurner 3y ago>That isn't really mind boggling since you are quoting fiction Fiction of the past plays an important role in seeing how far tech has progressed, that what was once fiction is now a commodity.
- deleted 3y ago[deleted]
- segfaultbuserr 3y ago> RTX 4090 has 2x the TFLOPS computing power than the killer AI Skynet used in Terminator 3. The writers back then probably though 60 TFLOPS is such a ridiculously high sci-fi number Also a fact worth noting but is routinely ignored in the popular press is that, these astronomical peak floating-point ratings of modern hardware are only achievable for a small selection of algorithms and problems. In practice, realizable performance is often much worse, efficiency can be as low as 1%. First, not all algorithms are best suited for the von Neumann architecture. Today, the memory wall is higher than ever. The machine balance (FLOPS vs. load/store) of modern hardware is around 100:1. To maximize floating-point operations, all data must fit in cache. This requires the algorithm to have a high level of data reuse via cache blocking. Some algorithms do it especially well, like dense linear algebra (Top500 LINPACK benchmark). Other algorithms are less compatible with this paradigm, they're going to be slow no matter how well the optimization is. Examples include many iterative physics simulation problems, sparse matrix code, and graph algorithms (Top500 HPCG benchmark). In the Top500 list, HPCG is usually 1% as fast as LINPACK. Best-optimized simulation code can perhaps reach 20% of Rpeak. This is why both Intel and AMD started offering special large-cache CPUs, either using on-package HBM or 3D-VCache. They're all targeted for HPC. Meanwhile in machine learning, people also made the switch to FP16, BF16 and INT8 largely because of the memory wall. Doing inference is a relatively cache-friendly problem, many HPC simulations are much worse in this aspect. Next, even if the algorithm is well-suited for cache blocking, peak datasheet performance is usually still unobtainable because it's often calculated from the peak FMA throughput. This is unrealistic in real problems, you can't just do everything in FMA - 70% is a more realistic target. In the worst case, you get 50% of the performance (disappointing, but not as bad as the memory wall). In contrast to datasheet peak performance, the LINPACK peak performance Rpeak is measured by a real benchmark.
- jcranmer 3y agoWhen you measure peak FLOPS, especially "my desktop computer has X FLOPS", you're generally computing N FMA units * f frequency, theoretical maximum FLOPS unit. This number, as you note, has basically no relation to anything practical: we've long been at the point where our ability to stamp out ALUs greatly exceeds our ability to keep those units fed with useful data. Top500 measures FLOPS on a different basis. Essentially, see how long it takes to solve an N×N equation Ax=b (where N is large enough to stress your entire system), and use a synthetic formula to convert N into FLOPS. However, this kind of dense linear algebra is an unusually computation-heavy benchmark--you need to do about n^1.5 FLOPS per n words of data. Most kernels tend to do more like O(n) or maybe as high as O(n lg n) work for O(n) data, which requires a lot higher memory bandwidth than good LINPACK numbers does. Furthermore, graph or sparse algorithms tend to do really bad because the amount of work you're doing isn't able to hide the memory latency (think one FMA per A[B[i]] access--you might be able to do massive memory bandwidth fetches on the first B[i] access, but you end up with a massive memory gather operation for the A[x] access, which is extremely painful).
- kabdib 3y agoAn SF book published in the 1950s (I have forgotten title and author, sigh) featured a then-imagined supercomputer with - 1M bits of storage - A mile on a side - Located in Buffalo, NY, and cooled by Niagra Falls (vacuum tubes, natch) - Able to surveil every citizen in the nation for suspicious activity No mention of clock speed, cache-line size, or instruction set. I guess SF writers aren't computer designers :-)
- merob 3y ago> The writers back then probably though 60 TFLOPS is such a ridiculously high sci-fi number for the world-ending AI, that nothing could possibly come close to it, and 20 years later consumers can have twice more computing power in their home PCs. If you look at the top500 supercomputer list of the time [1], they actually nailed it, the #1 rank at the time hitting a peak of 40TFLOPS [1] https://www.top500.org/lists/top500/2003/06/ https://www.top500.org/lists/top500/2003/06/