4 ms·
The AMD tinybox is on hold until we can build and run the firmware on our GPUs
- Havoc 3y agoUnfortunate but understandable. AMD needs to move faster on software support in AI space if they want any of that money.
- brucethemoose2 3y agoBut is this going to blow over in a few days? Again? I can certainly appreciate frustration with the AMD stack, but be blunt, I was not impressed with Hotz's YouTube rant from before.[1] It didn't give the impression of a stable framework, and this doesn't either. Also (at least from the end user llm inference side of things) ROCm is not nearly as unusable as it used to be. We would certainly be renting MI300s over A100s (or even H100s) if we could get any, and we use a number of different inference backends. 1: https://news.ycombinator.com/item?id=36193625 https://news.ycombinator.com/item?id=36193625
- jauntywundrkind 3y agoThe post PC era of big expensive hardware you can't even buy if you do have the money is upon us, and mercy this is a scary scary time in computing for me/us.
- smoldesu 3y agoBesides the datacenter stuff, what exactly are people struggling to source these days? The 30/40-series prices should be fairly stable relative to the MSRP these days.
- brucethemoose2 3y agoUsed 3090 prices are absolutely outrageous. And the 4090 MSRP was outrageous to begin with.
- brucethemoose2 3y agoI was talking about renting! There are some boutique hosts like Hot Aisle serving MI300s (who I really should reach out to), but for the immediate future our little startup is stuck with the big cloud providers. No MI300s for us mere mortals, not even to rent.
- pjmlp 3y agoYet another thing to go back to 1990's ecosystem. We already have timesharing again, now we have the prices as well.
- whalesalad 3y agohe's always been nails on a chalkboard for me. rant, whine, cry, repeat. reminds me of larry david with a CS degree.
- brucethemoose2 3y agoI never followed Hotz, so perhaps I missed something cool. But I never understood the hype myself.
- deleted 3y ago[deleted]
- zachbee 3y agoWhen they originally announced tiny corp and the tinybox, the entire pitch was that AMD hardware were great, but their software was bad. [1] Now they're giving up on AMD hardware because the software is bad. Wasn't the whole point to solve that problem? I'm hopeful for tinygrad as a piece of software, but I'm skeptical about the future of the tinybox if they keep waffling on the hardware so much. [1] https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...
- wmf 3y agoThe software is now fixed but that only revealed that the firmware is bad.
- 1oooqooq 3y agonext stop: microcode.
- throwawaymaths 3y agoThere is a texture between hard and soft
- snissn 3y agogooey?
- patmorgan23 3y agoFirme(ware)
- patmorgan23 3y agoFirm(ware)
- jrflowers 3y agoSilken
- 3y ago
- chaostheory 3y agoDoes Nvidia have any real competition that’s already shipped?
- brucethemoose2 3y agoThe MI300 is the best accelerator you can buy, for many current workloads. It's technically way more advanced. Not as outrageously priced as an H100 either.
- pjmlp 3y agoThat is the thing, they can allow themselves to have all that proprietary stuff, because Intel, AMD just can't get their act together. Even the whole OpenCL versus CUDA, they had years to ship something that was great tooling alternative, instead they did everything but that.
- whywhywhywhy 3y agoWhy would there be? The writing was on the wall that this was important 10 years ago and no one moved on it, in fact AMD and Apple both fumbled OpenCL then proceeded to waste several more years after that. I know some people don't like Nvidia but like their competition had their opportunity and what needed to be done spelt out to them and did nothing.
- lostmsu 3y agoTBH I don't know what they were counting on. 4090 has almost 3x BF16 tensor ops/s vs 7900XTX. So you can just buy a regular PC with 2x4090 for half the price and have basically the same training performance with much less headache.
- renewiltord 3y agoFrequently people say on HN you should use AMD but looks like it's not going to work. I am glad to stick to straightforward Nvidia GPU / Epyc CPU stack. Don't want to innovate for this.
- convolvatron 3y agothere is no room for innovation in high performance tensor evaluation, cost-performance, or usability. we're just done.
- renewiltord 3y agoThere is room, but if you are not working on building the framework, it's not worth building the framework. The time cost is high.
- brucethemoose2 3y agoIt's not either or, you can use different vendors for different tasks. tinygrad isn't in the realm of production ready though, AFAIK.
- fisf 3y agoYes but the same could be said about rocm.
- wmf 3y agoSomething that isn't really talked about is that Tiny Corp has been kind of working against AMD's interests. Tinybox is/was about replacing MI300s with much cheaper 7900 XTXs. I'm not terribly surprised to discover that AMD is not investing in ROCm on consumer cards (which are a different architecture) even though they're technically supported.
- throwaway48476 3y agoNo one would be buying A100's if CUDA hadn't been supported on all desktop Nvidia cards for years. Desktop cards provide an accessible on ramp to the ecosystem. PhD grads with boxes of desktop cards turn into the purchasers of data center chips.
- cherioo 3y agoNvidia was forced to do that because it was so early, they had no customers except PhDs. I see no evidence AMD wants to do that right now, and instead focusing on extracting value from deep pocket enterprise customers. The way things go, I think the AMD consumer card experience will only get better once AMD manage to gimp consumer cards’ ML throughput or RAM.
- DaiPlusPlus 3y ago> The way things go, I think the AMD consumer card experience will only get better once AMD manage to gimp consumer cards’ ML throughput or RAM. Que? Making things worse will make things better?
- sjsdaiuasgdia 3y agoThey'll let you do some things on a consumer card as soon as they can make sure that you can't effectively use the consumer card in place of an enterprise card.
- throwaway48476 3y ago
- whalesalad 3y agoThis was destined to be a hard problem. Surprised to see they are giving up so easily.
- yinser 3y agoIdentifying MES & CP as the barriers to an AMD tinybox and letting the community know is a huge service and is still great engineering even if they decide it's an immoveable barrier and walk away.
- caycep 3y agowhat's so tiny about a box that has 6 gpus?
- zitterbewegung 3y agoWhat advantage will remain if tensorflow and PyTorch works on AMD cards? https://www.xda-developers.com/nvidia-cuda-amd-zluda/ https://www.xda-developers.com/nvidia-cuda-amd-zluda/ https://pytorch.org/ https://pytorch.org/ has a rocm support . This doesn’t make the outlook on this company very good …
- alecco 3y ago> We are also (sadly) exploring a 6x4090 box. $12k for 6 4090 for 144GB GDDR vs $20k H100 PCIe 80GB HBM2 (price likely dropping later this year when B100 is released). And H100 has a lot of features like async and loading directly to tensor cores not present in consumer cards. I want to root for the little guy, but it seems the AI hardware landscape will be Nvidia for the next few years. And us GPU poors accessing it via cloud (shudders).
- mdaniel 3y agogiven his background in the jailbreak community, I look forward to him jailbreaking the AMD GPU firmware load process :-D
- mnau 3y agoJailbreaking is not the problem. The problem is reverse engineering a large firmware that operates on HW they have no docs for and fixing the firmware (ie. doing it better that original manufacturer that has access to everything). The job of firmware is so close to the hw that it's nearly impossible to decode. You need to decode a custom CPU instructions for a IP block the microcode is running on (it's custom, no ARM/RISC/MIPS..). After that you have to decode what firmware actually does. It writes something to this.... What does it mean? It's completely opaque number written to a opaque memory... cache control? Delay? And you do that why? So you can ship tinybox (i.e. cheap consumer GPU). Let's says you succeed. Do the same thing Next gen will be similar challenge, except firmware will be better locked, because AMD will want consumer HW to be segregated from data center GPU, the same way NVidia does. The task itself is basically impossible and waste of time. There is the reason why NVidia driver driver for Linux was used basically only to install official driver.
- mdaniel 3y agorelevant to his tenstorrent mention: https://news.ycombinator.com/item?id=39658787 https://news.ycombinator.com/item?id=39658787 and an bunch more https://hn.algolia.com/?q=tenstorrent https://hn.algolia.com/?q=tenstorrent