6 ms·

I don't get this obsession with 0-click everything. It is really annoying when you don't want to install everything to your main hard drive. I have all my model

I don't get this obsession with 0-click everything. It is really annoying when you don't want to install everything to your main hard drive. I have all my models downloaded, organized, and ready-to-go but apps won't even ask for that, instead it presumes I am an idiot and downloads it (again!) for me. At least Makeayo asks where my models are now. It's obnoxious that I have to use symlinks for comfy/automatic.... All they need to do is ask me where my stuff is on first run, and an area in the config to update that setting. Not so hard!

simonw 3y ago
Sounds like you should download the 4.45MB llamafile-server-0.1 executable from https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 and then run it against your existing gguf model files like this: ./llamafile-server-0.1 -m llama-2-13b.Q8_0.gguf See here: https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-trying-other-models https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-t...
rgbrgb 3y ago
fwiw FreeChat does this now. It prompts you to download or select a model to use (and you can add as many as you want). No copying or forced downloads.
mft_ 3y ago
If I'm understanding (and agreeing with) your gripe correctly, isn't it two solutions to the same perceived problem? My experience is that the world of Python dependency management is a mess which sometimes works, and sometimes forces you to spend hours-to-days searching for obscure error messages and trying maybe-fixes posted in Github issues for some other package, just in case it helps. This sometimes extends further - e.g. with hours-to-days spent trying to install just-the-right-version-of-CUDA on Linux... Anyway, the (somewhat annoying but understandable) solution that some developers take is to make their utility/app/whatever as self-contained as possible with a fresh install of everything from Python downwards inside a venv - which results in (for example) multiple copies of PyTorch spread around your HDD. This is great for less technical users who just need a minimal-difficulty install (as IME it works maybe 80-90% of the time), good for people who don't want to spend their time debugging incompatibilities between different library versions, but frustrating for the more technically-inclined user. This is just another approach to the same problem, which presumably also presents an even-lower level of work for the maintainers, since it avoids Python installs and packages altogether?
- pdntspa 3y ago
  I get that, my issue is when the model is coupled with the app, or the app just presumes I don't have it downloaded and doesn't ask me otherwise. This is like basic configuration stuff... What I suspect is happening is that people are cargo-culting zero-click installations. It seems rather fashionable right now.
  - deleted 3y ago
    [deleted]
  - rgbrgb 3y ago
    I don’t think making it easy to install is cargo-culting. In my case it’s an accessibility thing. I wanted a private alternative that I could give to nontechnical people in my life who had started using ChatGPT. Some don’t understand local vs cloud and definitely don’t know about ggufs or LLMs but they all install apps from the App Store.
  - whstl 3y ago
    In the README of the project (the TFA of this whole thread) there is the option to download the app without the model: "You can also also download just the llamafile software (without any weights included) from our releases page, or directly in your terminal or command prompt" There is no cargo-culting going on. Some of us do legitimately appreciate it.
    - pdntspa 3y ago
      Which has been followed, and this comment was not a response to this specific app but rather a general trend I've noticed and was mentioned at the start of this thread
      - whstl 3y ago
        I was answering to this complaint: "my issue is when the model is coupled with the app". In this specific case there is an option for you that addresses this complaint, where the model isn't coupled with the app.
joenot443 3y ago
The “obsession” isn’t for developers like you or I, it’s for non-technical but curious non-engineers who would still like to play around with an LLM. I think you’re imposing your own very specific (and solvable!) gripe onto an unrelated project and it makes for pretty clumsy commentary.
- whstl 3y ago
  Yep. But also: It might not be for a developer like you, but it is for a developer like me. I enjoy writing software, but I don't particularly enjoy futzing with building things outside my day-to-day work, and on systems I don't write myself. If it was up to me everything would be one click. Things like this are like accessibility: it benefits me even though I don't particularly need it.
  - pdntspa 3y ago
    It's the way things have been for god-knows-how-long, and it has worked really really well. I am utterly amazed and perplexed that answering ONE configuration question (which is all that is needed here) is too much. How lazy have we become? One question. That is too much for folks?! And I have no objection to the autodownloading. Just ask me where to put it first!

simonw 3y ago

Sounds like you should download the 4.45MB llamafile-server-0.1 executable from https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 https://github.com/Mozilla-Ocho/llamafile/releases/tag/0.1 and then run it against your existing gguf model files like this: ./llamafile-server-0.1 -m llama-2-13b.Q8_0.gguf See here: https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-trying-other-models https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-t...

rgbrgb 3y ago

fwiw FreeChat does this now. It prompts you to download or select a model to use (and you can add as many as you want). No copying or forced downloads.

mft_ 3y ago

If I'm understanding (and agreeing with) your gripe correctly, isn't it two solutions to the same perceived problem? My experience is that the world of Python dependency management is a mess which sometimes works, and sometimes forces you to spend hours-to-days searching for obscure error messages and trying maybe-fixes posted in Github issues for some other package, just in case it helps. This sometimes extends further - e.g. with hours-to-days spent trying to install just-the-right-version-of-CUDA on Linux... Anyway, the (somewhat annoying but understandable) solution that some developers take is to make their utility/app/whatever as self-contained as possible with a fresh install of everything from Python downwards inside a venv - which results in (for example) multiple copies of PyTorch spread around your HDD. This is great for less technical users who just need a minimal-difficulty install (as IME it works maybe 80-90% of the time), good for people who don't want to spend their time debugging incompatibilities between different library versions, but frustrating for the more technically-inclined user. This is just another approach to the same problem, which presumably also presents an even-lower level of work for the maintainers, since it avoids Python installs and packages altogether?

pdntspa 3y ago
I get that, my issue is when the model is coupled with the app, or the app just presumes I don't have it downloaded and doesn't ask me otherwise. This is like basic configuration stuff... What I suspect is happening is that people are cargo-culting zero-click installations. It seems rather fashionable right now.
- deleted 3y ago
  [deleted]
- rgbrgb 3y ago
  I don’t think making it easy to install is cargo-culting. In my case it’s an accessibility thing. I wanted a private alternative that I could give to nontechnical people in my life who had started using ChatGPT. Some don’t understand local vs cloud and definitely don’t know about ggufs or LLMs but they all install apps from the App Store.
- whstl 3y ago
  In the README of the project (the TFA of this whole thread) there is the option to download the app without the model: "You can also also download just the llamafile software (without any weights included) from our releases page, or directly in your terminal or command prompt" There is no cargo-culting going on. Some of us do legitimately appreciate it.
  - pdntspa 3y ago
    Which has been followed, and this comment was not a response to this specific app but rather a general trend I've noticed and was mentioned at the start of this thread
    - whstl 3y ago
      I was answering to this complaint: "my issue is when the model is coupled with the app". In this specific case there is an option for you that addresses this complaint, where the model isn't coupled with the app.

joenot443 3y ago

The “obsession” isn’t for developers like you or I, it’s for non-technical but curious non-engineers who would still like to play around with an LLM. I think you’re imposing your own very specific (and solvable!) gripe onto an unrelated project and it makes for pretty clumsy commentary.

whstl 3y ago
Yep. But also: It might not be for a developer like you, but it is for a developer like me. I enjoy writing software, but I don't particularly enjoy futzing with building things outside my day-to-day work, and on systems I don't write myself. If it was up to me everything would be one click. Things like this are like accessibility: it benefits me even though I don't particularly need it.
- pdntspa 3y ago
  It's the way things have been for god-knows-how-long, and it has worked really really well. I am utterly amazed and perplexed that answering ONE configuration question (which is all that is needed here) is too much. How lazy have we become? One question. That is too much for folks?! And I have no objection to the autodownloading. Just ask me where to put it first!