4 ms·
On the plus side, lots of cheap servers to swoop up :)
by winrid 2mo ago
On the plus side, lots of cheap servers to swoop up :)
- dgellow 2mo agoLook at their power supply, it’s not something you can run in a home lab. Unfortunately most of that will likely go to the bin eventually :(
- winrid 2mo agoNo but dedicated server prices will likely drop or at least you'll get more for your money
- sroussey 2mo agoBut power hungry. In that 5+ year timeline, the compute per watt could change by three orders of magnitude. GPUs are to LLMs what CPUs are to gaming — not a good fit.
- amluto 2mo agoA cursory estimate courtesy of ChatGPT suggests that there is a grand total of one order of magnitude or less of power efficiency improvement available compared to current Blackwell if the entire system’s power consumption outside the ALUs went all the way to zero. If you want three orders of magnitude improvement, you probably need to find two of those orders of magnitude somewhere else: process improvements, different ALU design, model architecture changes, etc.
- sroussey 2mo agoOh, you could go analog rather than digital. Ever look at ALU design? Nothing efficient about it!