4 ms·
My workstation runs a local LLM that allows me to ask deeply technical questions in plain English and receive useful, usually highly accurate (95%+ of the time)
by aappleby 2y ago
My workstation runs a local LLM that allows me to ask deeply technical questions in plain English and receive useful, usually highly accurate (95%+ of the time) responses in a few seconds.
It has no memory and requires no Internet access, so I feel generally comfortable interacting with it in a way that I do not when using Google or Facebook - if I randomly ask about drill presses or brownie recipes I will not suddenly be deluged by ads for toolboxes and Ozempic.
This to me is a radically powerful tool, and one that makes me skeptical of the "AI Bust" memes I see on occasion. There may be a lot of dumb money being poured into applying AI to problems it's ill suited for, but the things it is good at is such a positive step-function increase over the old tools that I cannot possibly imagine losing it.
- tuatoru 2y agoAnd does that get your groceries bagged or your leaky tap fixed? Acemoglu readily concedes that AI helps a lot with some jobs, but points out that those are a small proportion of total employment anyway.
- llm_trw 2y agoAttach it to a robot arm and yes it does. The latency isn't there yet so there's not much money being poured in, but when a 8b model can run in microseconds you'd see a very different world of robotics.
- manx 2y agoWhat is your setup?
- aappleby 2y agoMinisforum BD790i with 96 gigs of LPDDR5 RAM running Ollama. I run Qwen 110b on the CPU and get around a token per second, which is enough for me to dash off a question and get an answer a few minutes later. Smaller models like CodeLlama that can run 100% on the GPU are 10x faster, but I've found that any model under 70b params makes stupid errors when asked more complex questions.