2 ms·
Similar titled discussion from a day ago: https://news.ycombinator.com/item?id=48168198 https://news.ycombinator.com/item?id=48168198
by seltzered_ 5mo ago
Similar titled discussion from a day ago: https://news.ycombinator.com/item?id=48168198 https://news.ycombinator.com/item?id=48168198
- kristianp 5mo agoFrom the other discussion: "running at full capacity, 24/7. Why would you do that?". Also, I'd say if you're using Mac anyway that changes the economics, depends whether you've had to upgrade specs to run the size models you want. Going up to 64, 96 or 128GB adds 1000s to the price. However when I'm doing non local inference, I rarely use a model as small as 32B.