4 ms·

I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who c

by andy99 3y ago

I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can't run one of the many open source local llm software? It looks in the screenshot like it's exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even speculation about the potential market segments of LLM users.

pridkett 3y ago
In most workplaces that deal with LLMs you’ve got a few classes of people: 1. People who understand LLMs and know how to run them and have access to run them on the cloud. 2. People who understand LLMs well enough but don’t have access to cloud resources - but still have a decent MacBook Pro. Or maybe access to cloud resources is done via overly tight pipelines. 3. People who are interested in LLMs but don’t have enough technical chops/time to get things going with Llama CPP. 4. People who are fans of LLMs but can’t even install stuff in their computer. This is clearly for #3 and it works well for that group of people. It could also be for #2 when they don’t want to spin up their own front end.
wiihack 3y ago
I got Mistral-7b running locally, and although it wasn't hard, it did take some time nonetheless. I just wanted to try it out and was not that interested in the technical details.
3cats-in-a-coat 3y ago
It's for people who want to discover LLMs and either don't have the skill to deploy it, or value their time, and prefer not to fool around for hours getting it to work before they can try it. The fact it has configuration is good, as long as it has some defaults.
- transcriptase 3y ago
  Exactly. People like me have been waiting for a tool like this. I'm more than capable of compiling/installing/running pretty much any software, but all I want is the ability to chat with a LLM of my choice without spending an afternoon tabbing back to a 30 step esoteric GitHub .md full of caveats, assumptions, and requiring dependencies to be installed and configured according to preferences I don't have.
- paul_n 3y ago
  Yeah, I think I fit into this category. If I see a new model announced, it’s been nice to just click and evaluate for myself if it’s useful for me. If anyone knows other tools for this kind workflow I’d love to hear about them. Right now I just keep my “test” prompts in a text file.
mannyv 3y ago
It's actually quite handy. I built all the various things by hand at one point, but had to wipe it all. Instead of following the directions again I just downloaded this. Being able to swap out models is also handy. This probably saved a couple of hours of my life, which I appreciate.
compinter 3y ago
For me it's pretty simple- LM Studio supports Apple Silicon GPU acceleration out of the box, and I like the interface better than Gradio Web UI. It saves me the headache and the tinkering of the alternatives. That said, free software that's hiring developers probably won't stay free for long, so I'm keeping my eye on other options. Tl;DR It's for Mac users
3abiton 3y ago
It looks like civitai but for LLMs

by andy99 3y ago

pridkett 3y ago

In most workplaces that deal with LLMs you’ve got a few classes of people: 1. People who understand LLMs and know how to run them and have access to run them on the cloud. 2. People who understand LLMs well enough but don’t have access to cloud resources - but still have a decent MacBook Pro. Or maybe access to cloud resources is done via overly tight pipelines. 3. People who are interested in LLMs but don’t have enough technical chops/time to get things going with Llama CPP. 4. People who are fans of LLMs but can’t even install stuff in their computer. This is clearly for #3 and it works well for that group of people. It could also be for #2 when they don’t want to spin up their own front end.

wiihack 3y ago

I got Mistral-7b running locally, and although it wasn't hard, it did take some time nonetheless. I just wanted to try it out and was not that interested in the technical details.

3cats-in-a-coat 3y ago

It's for people who want to discover LLMs and either don't have the skill to deploy it, or value their time, and prefer not to fool around for hours getting it to work before they can try it. The fact it has configuration is good, as long as it has some defaults.

transcriptase 3y ago
Exactly. People like me have been waiting for a tool like this. I'm more than capable of compiling/installing/running pretty much any software, but all I want is the ability to chat with a LLM of my choice without spending an afternoon tabbing back to a 30 step esoteric GitHub .md full of caveats, assumptions, and requiring dependencies to be installed and configured according to preferences I don't have.
paul_n 3y ago
Yeah, I think I fit into this category. If I see a new model announced, it’s been nice to just click and evaluate for myself if it’s useful for me. If anyone knows other tools for this kind workflow I’d love to hear about them. Right now I just keep my “test” prompts in a text file.

mannyv 3y ago

It's actually quite handy. I built all the various things by hand at one point, but had to wipe it all. Instead of following the directions again I just downloaded this. Being able to swap out models is also handy. This probably saved a couple of hours of my life, which I appreciate.

compinter 3y ago

For me it's pretty simple- LM Studio supports Apple Silicon GPU acceleration out of the box, and I like the interface better than Gradio Web UI. It saves me the headache and the tinkering of the alternatives. That said, free software that's hiring developers probably won't stay free for long, so I'm keeping my eye on other options. Tl;DR It's for Mac users

3abiton 3y ago

It looks like civitai but for LLMs