5 ms·
First transformer != LLM. Imagine a world where you had to use AMD or CPUs, there would be no AlexNet, there would be no LLMs. Nvidia seeded universities with g
by KaoruAoiShiho 2y ago
First transformer != LLM. Imagine a world where you had to use AMD or CPUs, there would be no AlexNet, there would be no LLMs. Nvidia seeded universities with gifted hardware accelerators for a over a decade. Nvidia built the foundations for modern ML on which transformer lies, it's just one step on a long road to LLMs.
- dagmx 2y agoAMD had GPU compute APIs at the time as well. They also used to contribute GPUs (albeit in much smaller quantities). They just ended up killing them in favor of OpenCL which then withered on the vine. NVIDIA absolutely contributed to the foundation but they are not the foundation alone. Alexnet was great research but they could have done the same on other vendors at the time too. The hardware didn’t exist in a vacuum.
- physicsguy 2y agoOpenCL itself is kinda fine, but the libraries never existed because every academic from postdoc up could get a free NVidia card. You literally filled a form out and they sent you it.
- dagmx 2y agoI agree the library situation for OpenCL never stood up. Khronos tried at the start but then lost steam. It’s one of the projects I think the Khronos group mishandled the most unfortunately.
- DeathArrow 2y agoDo they have a project that achieved success?
- echoangle 2y agoOpenGL, Vulkan, WebGL?
- dagmx 2y agoHere’s a list of projects https://www.khronos.org/ https://www.khronos.org/ If I were to categorize the successful ones: glTF, KTX, SPIR-V, OpenGL and its variants, WebGL People will say Vulkan but it has the same level of adoption as OpenCL, and has the same issue that it competes against vendor specific APIs (DX and Metal) that are just better to use. It’s still used though of course as a translation target but imho that doesn’t qualify it as a success. OpenCL was and is a failure of grand magnitude. As was colada.
- KaoruAoiShiho 2y agoAlexnet could not have been done on AMD, nope.
- dagmx 2y agoPlease, this is just revisionism. There’s nothing inherent to AlexNet that relied on NVIDIA hardware or even software. It was just what happened to be available and most straightforward at the time. To say it wouldn’t have been possible on AMD is ludicrous and there is a pattern to your comments where you dismiss any other companies efforts or capabilities, but are quite happy to lay all the laurels on NVIDIA. The reality is that multiple companies and individuals got us to where we are, and multiple products could have done the same. That's not to take away from NVIDIA's success, it's well earned, but if you took them out of the equation, there's nothing that would have prevented the tech existing.
- hnlmorg 2y agoThe crux of the issue is this: > It was just what happened to be available and most straightforward at the time AMD made better hardware for a while and people wanted OpenCL to succeed. The reason why nvidia became dominant was because their competitors simply weren’t good enough for general purpose parallel compute. Would AI still have happened without CUDA? Almost certainly. However nvidia still had a massive role in shaping what it looks like today.
- EVa5I7bHFq9mnYK 2y agoMany people here are heavily invested in NVDA stock, which certainly clouds the judgement.
- dagmx 2y agoThat’s my take away too, even though I’m invested myself. The discussion around them has transcended into mythology. Imho part of it is because GPUs themselves are so foreign to many people as a development target and so they’re not really as aware of the landscape
- deleted 2y ago[deleted]