13 ms·
As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You ha
by buildbot 3y ago
As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an Nvidia GPU.
Google is barely limping along with it's XLA interface to pytorch providing researchers a decent compatibility path. Same with Intel.
Any company in this space should basically setup a giant test suite of IDK, every model on hugging face and just start brute force fixing the issues. Then maybe they can sell some chips!
Intel is basically doing the same shit they always do here, announcing some open initiative and then doing literally the bare minimum to support it. 99% chance openvino goes nowhere. OpenAIs Triton already seems more popular, at least I've heard it referenced a lot more than openvino.
- _the_inflator 3y agoI agree. That’s what MS didn’t understand with cloud and Linux at first. There is more than just a hardware layer to adoption. CUDA is a platform is an ecosystem is also some sort of attitude. It won’t go away. Companies invested a lot into it.
- claytonjy 3y agohuh, I didn't even know openvino supported anything but CPUs! TIL
- UncleOxidant 3y agoThis right here. Until Intel (and/or AMD) get serious about the software side and actually invest the money CUDA isn't going anywhere. Intel will make noises about various initiatives in that direction and then a quarter or two later they'll make big cuts in those divisions. They need to make a multi-year commitment and do some serious hiring (and they'll need to raise their salaries to market rates to do this) if they want to play in the CUDA space.
- latchkey 3y agoLiterally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them. The larger issue is that they need to fix the access to their high end GPUs. You can't rent a MI250... or even a MI300x (yet, I'm working on that myself!). But that said, you can't rent an H100 either... there are none available.
- _7il4 3y ago<removed>
- latchkey 3y agoROCm is open source. and: https://www.youtube.com/watch?v=pVl25BbczLI https://www.youtube.com/watch?v=pVl25BbczLI
- deleted 3y ago[deleted]
- mepian 3y agoHow’s SYCL proprietary exactly?
- latchkey 3y agoWhat _7il4 removed were these two comments: "AMD is not serious, and neither is Intel for that matter. Their software are piles of proprietary garbage fires. They may say they are serious but literally nothing indicates they are." "Yes, and ROCm also doesn't work on anything non-AMD. In fact it doesn't even work on all recent AMD gpus. T"
- frognumber 3y agoIt's not too polite to repost what people removed. Errors on the internet shouldn't haunt people forever. However, my experience is that the comments about AMD are spot-on, with the exception of the word "proprietary." Intel hasn't gotten serious yet, and has a good track record in other domains (compilers, numerical libraries, etc.). They've been flailing for a while, but I'm curious if they'll come up with something okay.
- foobiekr 3y agoIt's not just Intel. Open initiatives and consortiums (the phase two of the same) are always the losers ganging up hoping that it will give them the leg up they don't have. If you're older you'll have seen this play out over and over in the industry - the history of Unix vs. Windows NT from the 1990s was full of actions like this, networking is going through it again for the nth time (this time with UltraEthernet) and so on. OpenGl was probably the most successful approach, barely worked, and didn't help any of the players who were not on the road to victory already. Unix 95 didn't work, unix 98 didn't work, etc.
- AnthonyMouse 3y agoYou're just listing the ones that didn't knock it out of the park. TCP/IP completely displaced IPX to the point that most people don't even remember what it was. Nobody uses WINS anymore, even Microsoft uses DNS. It's rare to find an operating system that doesn't implement the POSIX API. The past is littered with the corpses of proprietary technologies displaced by open standards. Because customers don't actually want vendor-locked technology. They tolerate it when it's the only viable alternative, but make the open option good and the proprietary one will be on its way out.
- pjmlp 3y ago> It's rare to find an operating system that doesn't implement the POSIX API. Except that it has barelly improved beyond CLI and daemons, still thinks terminals are the only hardware, everything else that matters isn't part of it, not even more modern networking protocols that aren't exposed in socket configurations.
- AnthonyMouse 3y agoIts purpose was to create compatibility between Unix vendors so developers could write software compatible with different flavors. The primary market for Unix vendors is servers, which to this day are still about CLI and daemons, and POSIX systems continue to have dominant market share in that market.
- lacker 3y agoThe funny thing to me is that so much of the "AI software ecosystem" is just PyTorch. You don't need to develop some new framework and make it popular. You don't need to support a zillion end libraries. Just literally support PyTorch. If PyTorch worked fine on Intel GPUs, a lot of people would be happy to switch.
- latchkey 3y agoThis is a big reason why AMD did this deal with PyTorch... https://pytorch.org/blog/experience-power-pytorch-2.0/ https://pytorch.org/blog/experience-power-pytorch-2.0/
- shihab 3y agoBut you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense because generating random numbers is not exact that complicated), and documentation (God it was awful). Now, that's a small slice of the larger pie. But imagine if this trend continues for other libraries.
- lumost 3y agoFolks also underestimate how complex these libraries are. There are dozens of projects to make BLAS alternatives which give up after ~3-6 months when they realize that this project will take years to be successful.
- eternityforest 3y agoHow does that work? Why not pick up where the previous team left off instead of everyone starting new ones? Or are they all targeting different backends and hardware?
- 3y ago
- papichulo2023 3y agoWasnt Intel the biggest supporter of OpenCV? I dont know any open source heavely supported by Nvidia
- Keyframe 3y agoGuys need to do ye olde embrace, extend maneuver. What wine did, what Javas of the world did. CUDA driver API and CUDA runtime API either translation or implementation layer that offers compatibility and speed. I see no way around it at this point, for now.
- joe_the_user 3y agoThey could do that. That would eliminate Nvidia's monopoly. AMD has made gestures in that direction with Hip. But they ultimately don't want to do that - Hip support is half-assed and inconsistent. AMD creates and abandons a variety of APIs. So the conclusion is the other chip makers whine about Nvidia's monopoly but don't want to end it - they just want maneuver to get their smaller monopolies of some sort or other.
- Keyframe 3y agoYou're right. At this stage CUDA is de facto what the standard is around. Just like in ISA wars x86 was. Doesn't matter if you have POWER whatever when everything's on the other thing. I get why not though, it would drag the battle onto Nvidia's home turf. At least it would be a battle though.
- ilaksh 3y agoNvidia is probably ten times more scared of this guy https://github.com/ggerganov https://github.com/ggerganov than Intel or AMD.
- ebb_earl_co 3y agoCan you expand on this? This is my first time seeing this guy’s work
- pbronez 3y agoHe’s the main developer of Llama.cpp, which allows you to run a wide range of open-weights models on a wide range of non-NVIDIA processors.
- fl0id 3y agobut it's all inference, and most of Nvidias moat is in training afaik.
- refulgentis 3y agoPeople have really bizarre overdramatic misunderstandings of llama.cpp because they used it a few times to cook their laptop. This one really got me giggling though.
- ilaksh 3y agoI am integrating llama.cpp into my application. I just went through one of their text generation examples line-by-line and converted it into my own class. This is a leading-edge software library that provides a huge boost for non-Nvidia hardware in terms of inference capability with quantized models. If you don't understand that, then you have missed an important development in the space of machine learning.
- refulgentis 3y ago
- deleted 3y ago[deleted]
- dheera 3y agoBoth AMD and Intel (and Qualcomm to some degree) just don't seem to get how you beat NVIDIA. If they want to grab a piece of NVIDIA's pie, they do NOT need to build something better than an H100 right away. There are a million consumers who are happy with a 4090 or 4080 or even 3080 and would love for something that's equally capable at half price, and moreover, actually available for purchase, from Amazon/NewEgg/wherever, and without a "call for pricing" button. AMD and Intel are much better at making their chips available for purchase than NVIDIA. But that's not enough. What they DO need to do to take a piece of NVIDIA's pie is to build "intelcc", "amdcc", and "qualcommcc" that accept the EXACT SAME code that people feed to "nvcc" so that it compiles as-is, with not a single function prototype being different, no questions asked, and works on the target hardware. It needs to just be a drop-in replacement for CUDA. When that is done, recompiling PyTorch and everything else to use other chips will be trivial.
- flamedoge 3y agothats what hip is though. a recompiler for cuda code. its not good enough for ptx assembly to amdgpu yet
- DeathArrow 3y ago>Both AMD and Intel (and Qualcomm to some degree) just don't seem to get how you beat NVIDIA. That's simple, just do what Nvidia did. Be better than the competition.
- treprinum 3y agoThat's not going to work because each GPU has a different internal architecture and aligning the way data is fed with how stream processors operate is different for each architecture (stuff like how memory buffers are organized/aligned/paged etc.). AMD is very incompatible to Nvidia at the lowest level so things/approaches that are fast on Nvidia can be 10x slower on AMD and vice versa.
- dheera 3y ago> things/approaches that are fast on Nvidia can be 10x slower on AMD and vice versa. That's fine but the job of software is to abstract that out. The code should at least compile, even if it is 10x less efficient and if multiple awkward sets of instructions need to be used instead of one. If Pytorch can be recompiled for AMD overnight with zero effort (only `ln -s amdcc nvcc`, `ln -s /usr/local/cuda /usr/local/amda`) they will gain some footing against Nvidia.
- 2OEH8eoCRo0 3y ago> the software incompatibilities mean you'll spend a ton of time getting it to work compared to an Nvidia GPU. Leverage LLMs to port the SW
- tw04 3y ago> but nobody gets that it's the software and ecosystem. Really? Because I’m confident Intel knows exactly what it’s about. Have you looked at their contributions to Linux and open source in general? They employ thousands of software developers.
- madaxe_again 3y agoWhich is why intel left nvidia and everyone else in the dust with CUDA, which they developed. Oh, wait…
- tw04 3y agoWhat exactly do you think Intel was going to write CUDA for? Their GPU products have been on the market less than 2 years, and they're still trying to get their arms wrapped around drivers. Them understanding what was coming doesn't mean they have a magic wand to instantly have a fully competitive product. You can't write a CUDA competitor until you've gotten the framework laid. The fact they invested so heavily in their GPUs makes it pretty obvious they weren't caught off-guard, but catching up takes time. Sometimes you can't just throw more bodies at the problem...
- madaxe_again 3y agoThat’s my point though - they’re catching up, not leading, which rather implies that they absolutely missed a beat, and don’t therefore understand where the market is going before it goes there.
- tw04 3y ago“Nobody gets it’s the software” If they didn’t get it’s the software, they wouldn’t catch up. You didn’t say they were late to the party, you said they don’t understand what needs to happen. My point is they understand exactly what needs to happen they just didn’t have the technology to even start to tackle the problem until recently.
- deleted 3y ago[deleted]
- goldenshale 3y agoSeriously, why don't they just dedicate a group to creating the best pytorch backend possible? Proving it there will gain researcher traction and prove that their hardware is worth porting the other stuff over to.
- rightbyte 3y agoYou can't "just" do stuff like this. You need the right guy and big corps have no clue who is capable.
- MrBuddyCasino 3y agoThey still don't get that those who are serious about hardware, must make their own software.
- nox100 3y agoIs Mojo trying to solve this? https://www.youtube.com/watch?v=SEwTjZvy8vw https://www.youtube.com/watch?v=SEwTjZvy8vw