3 ms·

I'm not an apple fan (as I type on a mac that I am forced to use) but I gotta applaud their push for power efficiency. NVIDIA actually -does- have a few cards t

I'm not an apple fan (as I type on a mac that I am forced to use) but I gotta applaud their push for power efficiency. NVIDIA actually -does- have a few cards they make that really improve power efficiency but then they generally hamstring them with a lack of memory. NVIDIA is really good at making their high-end cards the only viable choice but I think that will backfire on them as people like me, that value quiet, cool and efficient over 25% faster inference start taking any viable alternative that comes out.

llm_trw 2y ago
Memory is king. Anything that has more memory and adequate compute will win the coming AI wars. At the rate at which power consumption is growing now that the shortage of current gen cards has started to work itself out people are realizing they need a fleet of nuclear reactors to keep the data centers running. This is not something that's getting fix with the coming generation, if anything it's worse.