ryao

24 ms·

301.
by ryao 1y ago
That is not true unless n^C / e^n = log(n) where C is some constant, which it is not. The difference between log(n) and some polynomial is logarithmic, not exponential.
302.
by ryao 1y ago
Carl Friedrich Gauss invented the FFT. Many reinvented it afterward until Cooley and Tukey popularized their reinvention of it.
303.
by ryao 1y ago
They ordinary people would be the ones that need this level of protection, since a scammer would talk them into sideloading malware if the device permits it.
304.
by ryao 1y ago
Netbeans is the Solaris IDE. It is free.
305.
by ryao 1y ago
In what software? llama.cpp and others divide things by layers.
306.
by ryao 1y ago
That is good to know. Thanks.
307.
by ryao 1y ago
I consider it to be a driver, or at least part of one. Userspace drivers exist. Graphic drivers originally were entirely in userspace, until portions of them were moved into the kernel for kernel mode setting and DRM. These days, graphics d
308.
by ryao 1y ago
I fixed some autocorrect typos that were in my comment. I do not find fault with UDNA and I have no idea why you think I do. I find fault with the CDNA/RDNA split. UDNA is what AMD should have done in the first place. As for Gaudi 3, I
309.
by ryao 1y ago
Ask Eric to consider writing a new blog post discussing the state of LLM training on AMD hardware. I would be very interested in reading what he has to say. AMD catching up in CPUs required that they become competent at hardware development
310.
by ryao 1y ago
The G stood for graphics. As for being stuck in the past, I got fed up in 2006 after 8 years of nothing but ATI graphics. I spent years hoping that the issues would be fixed after the latest update, but they never were. I had a fairly probl
311.
by ryao 1y ago
> ROCm isn't part of AMD drivers, its a software library that helps you support legacy compute APIs and stuff in the BLAS/GEMM/LAPACK end of things. AMD says otherwise: > AMD ROCm™ is an open software stack including dr
312.
by ryao 1y ago
GPUs were originally designed for gaming. Their ability to be used in HPC grew out of that. The history of issues goes back rather far.
313.
by ryao 1y ago
I hope your business works out for you and I am willing to believe that AMD has improved somewhat, but I do not believe AMD has improved enough to be worth people’s time when Nvidia is an option. I have heard too many nightmares and it is g
314.
by ryao 1y ago
Unfortunately, AMD and ATI before it have had driver quality issues for decades; and both they and their fans have claimed that they have solved the problems every year since. Even if they have made progress, I doubt that they have reached
315.
by ryao 1y ago
I read tinygrad’s website: https://tinygrad.org/#tinygrad Under driver quality for AMD, they say “developing” and point to their git repository. If AMD had fixed the issues, they would instead say the driver quality is grea
316.
by ryao 1y ago
There is nothing big about Groq’s chips. Their individual chips have only 230 MB RAM. Unlike Cerebras, which can load multiple layers into a single chip, grok must divide a layer across many chips.
317.
by ryao 1y ago
I have been hearing this about AMD/ATI drivers for decades. Every year, someone says that it is fixed, only for new evidence to come out that they are not. I have no reason to believe it is fixed given the history. Here is evidence to
318.
by ryao 1y ago
> As models become bigger, this does not scale anymore because the model weights will not fit into GPU memory anymore and you need to distribute them across GPUs or across nodes. Even with NVLink and Infiniband, these communications are
319.
by ryao 1y ago
It really does not matter how much memory AMD has if the drivers and firmware are unstable. To give one example from last year: https://www.tomshardware.com/pc-components/gpus/amds-lisa-su... They are currently de
320.
by ryao 1y ago
If he is doing multiturn conversations, he can reuse the kv cache from the last turn and skip the prompt processing on the history that would make time to first token too slow, by only doing prompt processing on his actual prompt for the cu
321.
by ryao 1y ago
Your “more efficient” remarks are nonsensical to me. Your “loading and unloading weights” remark would be slightly less nonsensical if you called it to Von Neumann bottleneck, but unfortunately for you, their hardware is so bottlenecked int
322.
by ryao 1y ago
Do you mean that they ordered stuff for you to buy that they did not carry?
323.
by ryao 1y ago
I had been unaware of the others. Anyway, you need writes to the KV cache for every token generated. You are going to hit that fast.
324.
by ryao 1y ago
Some context is needed for this. The only way to get a 4 orders of magnitude difference would be to compare incomparable things, like OpenAI’s most expensive model versus llama 3.1 8B.
325.
by ryao 1y ago
You replied really quickly when I had thought I could sneak in a revision, which dropped the estimates for production numbers. In any case, the Cerebras CSE-3 is extremely inefficient for what it does. Inference is memory bandwidth bound, s
326.
by ryao 1y ago
The CSE-3 is divided into 900,000 PEs, which each have only 48kB of RAM: https://hc2024.hotchips.org/assets/program/conference/day2/7... Similarly, the SMs in Blackwell have up to 228kB of RAM: https:&#
327.
by ryao 1y ago
They also support external DRAM over their 150GB/sec system IO link. They call it MemoryX and talk about it on these blog posts: https://www.cerebras.ai/blog/cerebras-cs-3-vs-nvidia-b200-20... https://w
328.
by ryao 1y ago
The other comment already clarified that 150GB/sec = 1.2Tbps. That said, the CSE-3 did not change this figure. It is buried in their specification sheets somewhere if you care to search for it. I did last year, which is how I know.
329.
by ryao 1y ago
How is that related to the claim that Cerebras themselves made about their hardware’s price performance ratio? https://www.cerebras.ai/press-release/cerebras-qualcomm-anno...
330.
by ryao 1y ago
The processes used for logic chips, and the processes used for NVM are typically different. The only case I know of the industry combining them onto a single chip would be Texas Instruments’ MSP430 microcontrollers with FeRAM, but the quant

More ›