4 ms·
Not really. I can generate images or get LLM answers in below 15 seconds on mundane hardware. The image generator draws many times faster than any normal perso
by dale_glass 10mo ago
Not really.
I can generate images or get LLM answers in below 15 seconds on mundane hardware. The image generator draws many times faster than any normal person, and the LLM even on my consumer hardware still produces output faster than I can type (and I'm quite good at that), let alone think what to type.
- beepbooptheory 10mo agoIs "faster" really what we are talking about right now? It could be a lot faster to take a helicopter to work everyday too, versus riding a bike. Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?
- FergusArgyll 10mo agoThere's a billion users. Why do we make massive cities and factories and fields if humans only need 2000 calories a day
- butlike 10mo agoidk, why?
- FergusArgyll 10mo agoBecause there's a lot of people
- discreteevent 10mo ago> It could be a lot faster to take a helicopter to work everyday too, versus riding a bike. Great analogy.
- Dylan16807 10mo agoIt's not a good analogy at all, because of what they said about mundane hardware. They're specifically not talking about any kind of ridiculous wattage situation, they're talking about single GPUs that need fewer watts than a human in an office to make text faster than a human, or that need 2-10x the watts to make video a thousand times faster.
- dale_glass 10mo agoIt's a Framework Desktop motherboard. I believe the CPU on that maxes out somewhere around 150W.
- dale_glass 10mo agoSpeed highly correlates with power efficiency. I believe my hardware maxes out somewhere around 150W. 15 seconds of that isn't much at all. > Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"? I presume that's mostly training, not inference. But in general anything that serves millions of requests in a small footprint is going to look pretty big.
- butlike 10mo agoAn LLM gives AN answer. If you ask for not many more than that it gets confused, but instead of acting in a human-like way, it confidently proceeds forward with incorrect answers. You never quite know when the context got poisoned, but reliability drops to 0. There's many things to say on this. Free is worthless. Speed is not necessarily a good thing. The image generation is drivel. But... The main nail in the coffin is accountability. I can't trust my work if I can't trust the output of the machine. (and as a bonus, the machine can't build a house. It's single purpose).
- Dylan16807 10mo agoOkay, but this has vanishingly little to do with the comment chain you replied to, which was about energy efficiency.