Now that I'm back at my computer, can search harder and it seems one can legitimately buy 256GB sticks at approximately USD$2000 a pop <https://www.ebay.com/itm

Now that I'm back at my computer, can search harder and it seems one can legitimately buy 256GB sticks at approximately USD$2000 a pop <https://www.ebay.com/itm/267177294719 https://www.ebay.com/itm/267177294719> or 128GB for $790 <https://www.ebay.com/itm/205354052535 https://www.ebay.com/itm/205354052535> Since it seems A100s top out at 80GB, and appear to start at $10,000 I'd say it's a steal Yes, I'm acutely aware that bandwidth matters, but my mental model is the rest of that sentence is "up to a point," since those "self hosted LLM" threads are filled to the brim with people measuring tokens-per-minute or even running inference on CPU I'm not hardware adjacent enough to try such a stunt, but there was also recently a submission of a BSD-3-Clause implementation of Google's TPU <https://news.ycombinator.com/item?id=44111452 https://news.ycombinator.com/item?id=44111452>