What's really amazing about a lot of these recent projects is they tend to provide benchmarks running on an Nvidia T4. They use these because they're relatively

What's really amazing about a lot of these recent projects is they tend to provide benchmarks running on an Nvidia T4. They use these because they're relatively cheap from cloud providers and you can usually actually get them (as opposed to requesting and getting denied for an A100 or whatever). For those that aren't familiar with it they are tiny power and density optimized GPUs. I have the successor (A2) and total max TDP is 60 watts. Single slot. Slot only power, and passively cooled. Depending on workload I observe it to be roughly 5-10x slower than a 3090, which means for most people at home with a spare Nvidia gaming card (or whatever) you'll see results from these project at a performance multiple of the benchmarks they provide. The one caveat is that the T4/A2 have 16GB VRAM, which makes them more capable (albeit slower) than a "low end" desktop card like the 3070 which has only 8GB VRAM. But as HN readers know there is incredible progress daily to reduce VRAM requirements for these models!