4 ms·

It's great that there are many comments about being able to run models locally, but I rarely see comments about what folks are using them for. Would love to lea

It's great that there are many comments about being able to run models locally, but I rarely see comments about what folks are using them for. Would love to learn more about use cases and problems being solved.

Der_Einzige 3y ago
Let's be honest, they're using them mostly for ahem personal time.
throwup238 3y ago
It's a goldrush and no one wants to share their secrets. Here's a good example from previous HN comments: https://news.ycombinator.com/item?id=38482347 https://news.ycombinator.com/item?id=38482347
- eurekin 3y ago
  I recently made them work with meeting presentation transcript. Mistral 7b (tiny one) was enough to reliably extract quite a lot of information! - It could derive speaker names, based purely on how people called themselves in the conversation - Drew a mermaid sequence diagram that wasn't perfect, but wasn't a complete garbage either. With few back and forth corrections it was on-point. - Created a truly usable meeting notes It was a much better UX than having to hunt down relevant video section, watch it and force to focus despite a lot of filler communication. That works very well for those kinds of 30 min. upward meeting, where the real content is in the spoken words and slides just reitarate that. I was really pleasantly surprised how much I liked that. Also, some models can rate each speaker contribution :o)
- kossTKR 3y ago
  Are people actually using this to 10x themselves / create actually profitable services already? Or more like finetuning a model to have an edge on the leaderboards for a day or two then taking VC money, or integrating som "AI magic" into existing userbases? I've been following the locallama sub on reddit and the few services created seem very niche besides tons of hobby stuff.
tarruda 3y ago
I use them for coding, you'd be surprised at how much you can get done with deepseek coder 6.7b Most GPT plugins for code editors will also work for local models since you can have OpenAI API stubs running locally. Clearly it is nowhere near GPT-4 capacity, but if you ask simple boilerplate things ("write a class with the following methods", then "write unit tests for it") it will mostly work. Even if it doesn't, you can manually fix it, and it still can save you some time. Always review code generated by LLM, regardless if it comes from GPT-4!
stefs 3y ago
i recently started using them as assistants for programming. i usually don't let them write code as it's often (subtly) wrong. examples are: * how do i use library X to do task Y (excellent for quickly getting up to speed with new libraries). * actual example from a few days ago: "the most common CI/CD systems and how to identify them" - chatgpt correctly gave me the environment variable names for github actions, gitlab ci, travis, circleci, jenkins and a one or two others. theoretically it saved me having to go through the docs of 7 different systems looking for the right information, which i still did to make sure the data was legit. just confirming the info was still a lot less work as i already knew what to look for. * how do i create a certain style with css framework XYZ * is there an algorithm for solving the following problem ...? * alternative phrasings or synonyms if i can't find the right words myself. * cooking recipe suggestions ("i have ingredients a, b and c, give me a stew recipe") * pop culture questions ("why did the fremen settle on a hostile planet like arrakis in the first place?") i'm too lazy to research myself or ask on reddit * sometimes my (non-natively english speaking) coworkers produce engrish i just can't parse. asking gpt to correct or explain the sentence often yields surprisingly good results. usually i double check the results but that's still less work than doing all the work by myself from the beginning. recently i also let it write and style html forms which works quite well. so for me, they're a welcome productivity boost.

Der_Einzige 3y ago

Let's be honest, they're using them mostly for ahem personal time.

throwup238 3y ago

It's a goldrush and no one wants to share their secrets. Here's a good example from previous HN comments: https://news.ycombinator.com/item?id=38482347 https://news.ycombinator.com/item?id=38482347

eurekin 3y ago
I recently made them work with meeting presentation transcript. Mistral 7b (tiny one) was enough to reliably extract quite a lot of information! - It could derive speaker names, based purely on how people called themselves in the conversation - Drew a mermaid sequence diagram that wasn't perfect, but wasn't a complete garbage either. With few back and forth corrections it was on-point. - Created a truly usable meeting notes It was a much better UX than having to hunt down relevant video section, watch it and force to focus despite a lot of filler communication. That works very well for those kinds of 30 min. upward meeting, where the real content is in the spoken words and slides just reitarate that. I was really pleasantly surprised how much I liked that. Also, some models can rate each speaker contribution :o)
kossTKR 3y ago
Are people actually using this to 10x themselves / create actually profitable services already? Or more like finetuning a model to have an edge on the leaderboards for a day or two then taking VC money, or integrating som "AI magic" into existing userbases? I've been following the locallama sub on reddit and the few services created seem very niche besides tons of hobby stuff.

tarruda 3y ago

I use them for coding, you'd be surprised at how much you can get done with deepseek coder 6.7b Most GPT plugins for code editors will also work for local models since you can have OpenAI API stubs running locally. Clearly it is nowhere near GPT-4 capacity, but if you ask simple boilerplate things ("write a class with the following methods", then "write unit tests for it") it will mostly work. Even if it doesn't, you can manually fix it, and it still can save you some time. Always review code generated by LLM, regardless if it comes from GPT-4!

stefs 3y ago

i recently started using them as assistants for programming. i usually don't let them write code as it's often (subtly) wrong. examples are: * how do i use library X to do task Y (excellent for quickly getting up to speed with new libraries). * actual example from a few days ago: "the most common CI/CD systems and how to identify them" - chatgpt correctly gave me the environment variable names for github actions, gitlab ci, travis, circleci, jenkins and a one or two others. theoretically it saved me having to go through the docs of 7 different systems looking for the right information, which i still did to make sure the data was legit. just confirming the info was still a lot less work as i already knew what to look for. * how do i create a certain style with css framework XYZ * is there an algorithm for solving the following problem ...? * alternative phrasings or synonyms if i can't find the right words myself. * cooking recipe suggestions ("i have ingredients a, b and c, give me a stew recipe") * pop culture questions ("why did the fremen settle on a hostile planet like arrakis in the first place?") i'm too lazy to research myself or ask on reddit * sometimes my (non-natively english speaking) coworkers produce engrish i just can't parse. asking gpt to correct or explain the sentence often yields surprisingly good results. usually i double check the results but that's still less work than doing all the work by myself from the beginning. recently i also let it write and style html forms which works quite well. so for me, they're a welcome productivity boost.