3 ms·
I've had an endgame idea in mind for a while. Models, probably first open weight ones like Kimi K3 class, are etched into silicon like this and sold as cartrid
by api 2mo ago
I've had an endgame idea in mind for a while.
Models, probably first open weight ones like Kimi K3 class, are etched into silicon like this and sold as cartridges almost like old school game cartridges.
You buy a USB-C dongle that the cartridge goes into, or for data centers you have PCI cards that take these in slots.
- wmf 2mo agoEach cartridge costs $1,000. Do you still want it?
- api 2mo agoMe? Probably not. A business or a hoster, sure. There'd probably end up being an aftermarket in used cartridges with slightly older but still good models on them.
- anigbrowl 2mo agoFor fast Kimi K3? You're damn right I do
- wmf 2mo ago$1,000 only gets you the Qwen 27B cartridge. For Kimi K3 it would be more like $100,000 (and the "cartridge" is the size of a refrigerator).
- trollbridge 2mo agoI would gladly pay $100,000 for local K3 running at 18,000 tok/sec.
- anigbrowl 2mo agoSome hurried goalpost-moving there! If we take Qwen 27B as the benchmark of equivalent power (rather than its default configuration with no fine-tuning), I can still think of applications. Auto summarization for legal document databases and other high security/offline contexts, contraband detection, moderating websites like HN etc :)
- singingtoday 2mo agoYeah. I have 3 max20 plans.