5 ms·
It's worth remembering that this is all happening because of video games ! It is highly unlikely that the hardware which makes LLMs possible would have been de
by delegate 8mo ago
It's worth remembering that this is all happening because of video games !
It is highly unlikely that the hardware which makes LLMs possible would have been developed otherwise.
Isn't that amazing ?
Just like internet grew because of p*rn, AI grew because of video games.
Of course, that's just a funny angle.
The way I see it, AI isn't accidental. Its inception has been in the first chips, the Internet, Open Source, Github, ...
AI is not just the neural networks - it's also the data used to train it, the OSes, APIs, the Cloud computing, the data centers, the scalable architectures.. everything we've been working on over the last decades was inevitably leading us to this.
And even before the chips, it was the maths, the physics ..
Singularity it seems, is inevitable and it was inevitable for longer than we can remember.
- blibble 8mo agowhat a load of utter tripe
- sealeck 8mo agoI think it's a bit hard to say that this is definitively true: people have always been interested in running linear algebra on computers. In the absence of NVIDIA some other company would likely have found a different industry and sold linear algebra processing hardware to them!
- senbrow 8mo agoAlmost certainly not at the scale of the consumer gaming industry, however!
- willis936 8mo agoGoogle is making millions of TPUs per year. Nvidia ships more gaming GPUs, but it's not like multiple orders of magnitude off.
- senbrow 8mo agoI'm willing to bet TPUs wouldn't be nearly as successful or sophisticated without the decades of GPU design and manufacturing that came before them. Current manufacturing numbers are a small part of the story of the overall lineage.
- hparadiz 8mo agoIt's pretty interesting that consumer GPUs started to really be a thing in the early 90s and the first Bitcoin GPU miner was around 2011. That's only 20 years. That caused a GPU and asic gold rush. The major breakthroughs around LLMs started to snowball in the academic scene right around that time. It's been a crazy and relatively quick ride in the grand scheme of things. Even this silicone shortage will pass and we'll look back on this time as quaint.
- danielmarkbruce 8mo agoOf course you are right, but in addition they wouldn't have even made them if GPUs hadn't made ML on CPU so relatively incapable. Competition drives a lot of these decisions, not just raw performance.
- danielmarkbruce 8mo agoYou are missing his point. They very likely don't start building TPUs if there were no GPUs.
- willis936 8mo agoI'm not missing the point. If you recall your computer architecture class there are many vector processing architectures out there. Long before there was nvidia the world's largest and most expensive computers were vector processors. It's inaccurate to say "gaming built SIMD".
- danielmarkbruce 8mo agoYou are missing the point - it's an economic point. Very little R&D was put into said processors. The scale wasn't there. The software stack wasn't there (because the scale wasn't there). No one is suggesting gaming chips were the first time someone thought of such an architecture or built a chip with such an architecture. They are suggesting the gaming industry produced the required scale to actually do all the work which lead to that hardware and software being really good, and useful for other purposes. In chip world, scale matters a lot.
- dwd 8mo agoGoogle DeepMind can trace part of it's evolution back to a playtester for the video game Syndicate who saw an opportunity to improve the AI of game NPCs.
- BatteryMountain 8mo agoRemember that games are just simulations. Physics, light, sound, object boundaries - it not real, just a rough simulation of the real thing. You can say that ML/AI/LLM's are also just very distilled simulations. Except they simulate text, speech, images, and some other niche models. It is still very rough around the edges - meaning that even though it seems intelligent, we know it doesn't really have intelligence, emotions and intentions. Just as game simulations are 100% biased towards what the game developers, writers and artists had in mind, AI is also constrained to the dataset they were trained on.
- dboon 8mo agoType the word porn. Self censorship is wrong.
- tim333 8mo agoSci-fi ideas of robots have been around for ages and work on AI and the term singularity kicked off around 1950, so a while ago - well before chips or me being born.