4 ms·
Can we expect the price of 'Gaudi 3 PCIe' to be reasonable enough to put in a workstation? That would be a game changer for local LLMs
by sairahul82 2y ago
Can we expect the price of 'Gaudi 3 PCIe' to be reasonable enough to put in a workstation? That would be a game changer for local LLMs
- CuriouslyC 2y agoJust based on the RAM alone, let's just say if you can't just buy a Vision Pro without a second thought about the price tag, don't get your hopes up.
- wongarsu 2y agoProbably not. An 40GB Nvidia A100 is arguably reasonable for a workstation at $6000. Depending on your definition an 80GB A100 for $16000 is still reasonable. I don't see this being cheaper than an 80GB A100. Probably a good bit more expensive, seeing as it has more RAM, compares itself favorably to the H100, and has enough compelling features that it probably doesn't have to (strongly) compete on price.
- chessgecko 2y agoI think you're right on the price, but just to give some false hope. I think newish hbm (and this is hbm2e which is a little older) is around $15/gb so for 128 gb thats $1920. There are some other cogs, but in theory they could sell this for like $3-4k and make some gross profit while getting some hobbyist mindshare/research code written for it. I doubt they will though, it might eat too much into profits from the non pcie variants.
- p1esk 2y agoin theory they could sell this for like $3-4k You’re joking, right? They will price it to match current H100 pricing. Multiply your estimate by 10x.
- chessgecko 2y agoThey could, I know they wont, but they wouldn't lose money on the parts
- ksec 2y ago> is around $15/gb This figure is old and I dont think $15 cuts it anymore. My guess would be $20 if not exceeds it.
- Workaccount2 2y agoInterestingly they are using HBME2 memory which is a few years old at this point. The price might end up being surprisingly good because of this.
- 0cf8612b2e1e 2y agoSurely NVidia’s pricing is more what the market will bear vs an intrinsic cost to build. Intel being the underdog should be willing to offer a discount just to get their foot in the door.
- wmf 2y agoNvidia is charging $35K so a discount relative to that is still very expensive.
- tormeh 2y agoPricing is normally what the market will bear. If this is below your cost as supplier you exit the market.
- AnthonyMouse 2y agoBut if your competitor's price is dramatically above your cost, you can provide a huge discount as an incentive for customers to pay the transition cost to your system while still turning a tidy profit.
- narrator 2y agoIsn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?
- magic_hamster 2y agoMacs don't support CUDA which means all that wonderful hardware will be useless when trying to do anything with AI for at least a few years. There's Metal but it has its own set of problems, biggest one being it isn't a drop in CUDA replacement.
- doublepg23 2y agoI'm assuming this won't support CUDA either?
- adam_arthur 2y agoYou can do LLM inference without CUDA just fine. Download Ollama and see for yourself
- egorfine 2y agoFor LLM inference - yes absolutely.
- ipsum2 2y agoIt won't be under $10k.