georgehotz

9 ms·

61.
by georgehotz 2y ago
That OCP 3.0 card has the same link bandwidth as the GPUs, so you can scale out without much loss of all-reduce bandwidth. In practice, for all models except the largest, the ~16GB/s all-reduce is totally fine. You just need to make su
62.
by georgehotz 2y ago
It's the same way NVIDIA states bandwidth for PCIe and NVLink. https://www.nvidia.com/en-us/data-center/h100/
63.
by georgehotz 2y ago
The worst part of this critique is that it's not even true, and marcan knows better. He clearly dislikes me so much that he is willing to lie. The symmetric half of the metldr key was obtained with a novel exploit I found. Then the asy
64.
by georgehotz 2y ago
A GM at Ford absolutely will be paying this if they want a phone call. Joining our Discord is free however.
65.
by georgehotz 2y ago
We didn't just bypass all of ROCm, we bypassed HSA! The HSA parsing MEC firmware running on the GPUs is riddled with bugs, fortunately you can bypass 90% of it using PM4, which is pretty much direct sets of the GPU registers. That&#x27
66.
by georgehotz 2y ago
We'll get there as we push deeper into assemblies. RDNA3 probably first, since it's documented and a bit simpler.
67.
by georgehotz 2y ago
Cool to see one of these in C, particularly if it can be binary compatible. Why not s/libreCuInit/cuInit? If you are interested in open source runtimes, tinygrad has them in Python for both AMD and NVIDIA, speaking directly to the
68.
by georgehotz 2y ago
AMD is already on MLPerf in the form of the tinybox red :)
69.
by georgehotz 3y ago
I actually lost about $5k on cheapETH running servers. Nobody was "defrauded", I think these people don't understand how forks work. It's a precursor to the modern L2 stuff, I did this while writing the first version of
70.
by georgehotz 3y ago
It should if your 3090s have Resizable BAR support in the VBIOS. AFAIK most card manufacturers released BIOS updates enabling this. Re: 3090 NVLink, that only allows pairs of cards to be connected. PCIe allows full fabric switch of many car
71.
by georgehotz 3y ago
tinygrad supports uneven splits. There's no fundamental reason for 4 or 8, and work should almost fully parallelize on any number of GPUs with good software. We chose 6 because we have 128 PCIe lanes, aka 8 16x ports. We use 1 for NVMe
72.
by georgehotz 3y ago
This is my weekend hobby, I'm learning about GPUs. Besides, I'd like to see competition to NVIDIA. From a business side, tiny corp is selling boxes with both AMD and NVIDIA, we'll let the user choose how much they value money
73.
by georgehotz 3y ago
I have had multiple replies from Lisa Su on both Twitter and by e-mail. It doesn't help. AMD is structurally incapable of fixing these issues. They don't even have a 7900XTX in CI. Crashing the firmware is so trivial there's
74.
by georgehotz 3y ago
We have a product for sale: https://comma.ai We raised $18.1M and have made $28M in revenue to date. Where are you getting your narrative? Are you confusing comma with someone else?
75.
by georgehotz 3y ago
Are you interested in being factually correct, or are you interested in hating? If it's the former, I think you should do some research. If it's the latter, :salute: I had GPT-4 do some research for you, hopefully you will incorpo
76.
by georgehotz 3y ago
lol, you should see me bash my own code. I'm even more mean. https://github.com/tinygrad/tinygrad/blob/master/examples/hl... have a bunch of bounties on it, we're getting 94%+ now! mostly
77.
by georgehotz 3y ago
And...it's a good plan. There's two cooling problems, pulling heat off the wafer (water cooled loop or immersion cooling, this is easy) and dumping the heat somewhere. The second part is why the place being cool matters, I imagine
78.
by georgehotz 3y ago
Have you ever met someone with actual bipolar disorder? They think that they are Gods and see symbols and signs and shit. Or sometime they go negative and thing there's a conspiracy targeting them or something. They don't usually
79.
by georgehotz 3y ago
Oh this is nice! Is this the same quantization method as GGML? It's just described as "Efficient custom 8 bit and 4 bit quantization" And thanks for the https://pypi.org/project/lm-eval/ pointer, ha
80.
by georgehotz 3y ago
I question if perplexity on wikitext is enough, I wonder if someone has checked how well perplexity correlates with https://github.com/google/BIG-bench Fundamentally, I'm not sure if perplexity on a dataset that&#
81.
by georgehotz 3y ago
Which low level optimizations specifically are you referring to? I'm happy with most of the abstractions. We are pushing to assembly codegen. And if you meant things like matrix accelerators, that's my next priority. We are taking
82.
by georgehotz 3y ago
In the limit, AI is potentially very dangerous. All intelligence is. I am a lot more worried about human intelligence. Re: alignment. I'm not concerned about alignment between the machines and the owner of the machines. I'm concer
83.
by georgehotz 3y ago
I love that the "hater" thread turns into a discussion of uncountable infinities :) The "cardinality" section of that Wikipedia page describes my objection well. I don't doubt the real numbers are not recursively en
84.
by georgehotz 3y ago
For reference, comma released a new version of openpilot today, and we now sell comma threes. https://blog.comma.ai/092release/
85.
by georgehotz 4y ago
I’m not Elon! I’m talking about openpilot, see linked video.
86.
by georgehotz 4y ago
The FSD is actually going well https://www.youtube.com/watch?v=KF7X7s48jRc Never said it would be done in a year, give it until the end of the decade.
87.
by georgehotz 4y ago
This is why tinygrad.org is tiny.
88.
by georgehotz 4y ago
Whole net performance at comma, when we switch from BatchNorm to GroupNorm it adds 70ms to the training step time, and it's -70ms for no norm. We also wrote a custom AllNorm that's like 10% slower than BatchNorm (and I put several
89.
by georgehotz 4y ago
Agreed. For anything at all common, most of the gains will be from fusion, the rest is just free. PyTorch also uses tons of GPU memory after only initializing, I wonder if it's copying all the kernels in?
90.
by georgehotz 4y ago
That's a ReduceOp right now, more annoying to reason about than a UnaryOp. But in the limit, yea. Or add an elementwise BinaryOp for max. Submit a PR if you can improve something!

More ›