3 ms·
Something about having a very fast, very small harness written in C only to drive LLMs amuses me. Like putting a big ass spoiler on a minivan or something.
by losvedir 2mo ago
Something about having a very fast, very small harness written in C only to drive LLMs amuses me. Like putting a big ass spoiler on a minivan or something.
- tjoff 2mo agoWell, you'd think so but given how slow and resource hungry the ones written in javascript are it makes a lot of sense.
- deleted 1mo ago[deleted]
- para_parolu 1mo agoAre they? I use pi and it doesn’t drain more than other software.
- pm25coder 1mo ago[flagged]
- snvzz 1mo agoThe LLM can run elsewhere, such as the cloud. For the part that runs locally, efficiency might matter.