3 ms·
What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but f
by karencarits 1y ago
What use cases are people using local LLMs for? Have you created any practical tools that actually increase your efficiency? I've been experimenting a bit but find it hard to get inspiration for useful applications
- jsemrau 1y agoI have a signal tracer that evaluates unusual trading volumes. Given those signals, my local agent receives news items through API to make an assessment what happens. This helps me tremendously. If I would do this through a remote app, I'd have to spend a several dollars per day. So I have this on existing hardware.
- karencarits 1y agoThank you, this is a great example!
- dyauspitr 1y agoDo you want to share it?
- sudomarcma 1y agoAny companies with any type of sensitive data will love to have anything to do with LLM done locally.
- thenameless7741 1y agoA recent example: a law firm hired this person [0] to build a private AI system for document summarization and Q&A. [0] https://xcancel.com/glitchphoton/status/1927682018772672950 https://xcancel.com/glitchphoton/status/1927682018772672950
- lvturner 1y agoAlso worth it for the speed of AI autocomplete in coding tools, the round trip to my graphics card is much faster than going out over the network.
- mbac32768 1y agoAnyone actually doing this? DeepSeek-R1 32b ollama can't run on an RTX 4090 and the 17b is nowhere near as good at coding as OpenAI or Claude models.
- lvturner 1y agoI specified autocomplete, I'm not running a whole model asking it to build something and await an output. DeepSeek-coder-v2 is fine for this, I occasionally use a smaller Qwen3 (I forget exactly which at the moment... Set and forget) for some larger queries about code, given my fairly light used cases and pretty small contexts it works well enough for me
- bcoates 1y agoI use the local LLM-based autocomplete built into PyCharm and I'm pretty happy with it
- codedokode 1y agoAnyone who does not want to leak their data? I am actually surprised that people are ok with trusting their secrets to a random foreign company.
- karencarits 1y agoBut what do you do with these secrets? Like tagging emails, summarizing documents?
- lurking_swe 1y agoa document management system is an easy example. Let’s say medical, legal, and tax documents.
- karencarits 1y agoThank you, but what do you use the llm for? Writing new documents based on previous ones? Tagging/categorization/summarization/lookup? RAG? Extracting structured data from them?
- lurking_swe 1y agoMe personally, i’m using paperless-ngx to manage documents. i use ollama to generate a document title, with 8 words or less. I then go through and make any manual edits at my leisure. Saves me time which i appreciate! Paperless-ngx already does a pretty good job auto-tagging, i think it uses some built in classifiers? not 100% sure.
- nprateem 1y agoNo one cares about your 'secrets' as much as you think. They're only potentially valuable if you're doing unpatented research or they can tie them back to you as an individual. The rest is paranoia. Having said that, I'm paranoid too. But if I wasn't they'd have got me by now.
- 1y ago
- itsmevictor 1y agoI do a lot of data cleaning as part of my job, and I've found that small models could be very useful for that, particularly in the face of somewhat messy data. You can for instance use them to extract some information such as postal codes from strings, or to translate and standardize country names written in various languages (e.g. Spanish, Italian and French to English), etc. I'm sure people will have more advanced use cases, but I've found them useful for that.