3 ms·
What ... how ...? Just bought two yesterday for playing with local llm but thought 3.8 was totally out of reach!?
by runekaagaard 1mo ago
What ... how ...? Just bought two yesterday for playing with local llm but thought 3.8 was totally out of reach!?
- cmrdporcupine 1mo agoA PC with a small GPU coordinating and then llama.cpp using the llama RPC stuff (over RDMA to reduce latency) talking to the two nodes. I dunno, maybe he'll do a write-up someday.
- runekaagaard 1mo agoOK thanks! Would love to read :)