3 ms·

Maybe it's a personal preference, but I don't want external programs to ever touch my package manager, even with permission. Besides, this will fail loudly for

Maybe it's a personal preference, but I don't want external programs to ever touch my package manager, even with permission. Besides, this will fail loudly for systems that don't use `apt-get`. I would just ask the user to install the package, and _maybe_ show the command line to install it (but never run it).

danielhanchen 1y ago
Hopefully the solution for now is a compromise if that works? It will show the command as well, so if not accepted, typing no will error out and tell the user on how to install the package
lucideer 1y ago
I don't think this should be a personal preference, I think it should be a standard*. That said, it does at least seem like these recent changes are a large step in the right direction. --- * in terms of what the standard approach should be, we live in an imperfect world and package management has been done "wrong" in many ecosystems, but in an ideal world I think the "correct" solution here should be: (1) If it's an end user tool it should be a self contained binary or it should be a system package installed via the package manager (which will manage any ancillary dependencies for you) (2) If it's a dev tool (which, if you're cloning a cpp repo & building binaries, it is), it should not touch anything systemwide. Whatsoever. This often results in a README with manual instructions to install deps, but there are many good automated ways to approach this. E.g. for CPP this is a solved problem with Conan Profiles. However that might incur significant maintenace overhead for the Unsloth guys if it's not something the ggml guys support. A dockerised build is another potential option here, though that would still require the user to have some kind of container engine installed, so still not 100% ideal.
- danielhanchen 1y ago
  I would like to be in (1) but I'm not a packaging person so I'll need to investigate more :( (2) I might make the message on installing llama.cpp maybe more informative - ie instead of re-directing people to the docs on manual compilation ie https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-do-i-manually-save-to-gguf https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-..., I might actually print out a longer message in the Python cell entirely Yes we're working on Docker! https://hub.docker.com/r/unsloth/unsloth https://hub.docker.com/r/unsloth/unsloth
  - lucideer 1y ago
    > Yes we're working on Docker! That will be nice too, though I was more just referring to simply doing something along the lines of this in your current build: docker run conanio/gcc11-ubuntu16.04 make clean -C llama.cpp etc etc... (likely mounting & calling a sh file instead of passing individual commands) --- Although I do think getting the ggml guys to support Conan (or monkey patching your own llama conanfile in before building) might be an easier route.
    - danielhanchen 1y ago
      Oh ok I'll take a look at conanfiles as well - sorry I'm not familiar with them!
solarkraft 1y ago
I like it when software does work for me. Quietly installing stuff at runtime is shady for sure, but why not if I consent?
- danielhanchen 1y ago
  Do you think it's ok for permissioning I guess? I might also add a 30 second timer and just bail out if there's no response from the user

danielhanchen 1y ago

Hopefully the solution for now is a compromise if that works? It will show the command as well, so if not accepted, typing no will error out and tell the user on how to install the package

lucideer 1y ago

I don't think this should be a personal preference, I think it should be a standard*. That said, it does at least seem like these recent changes are a large step in the right direction. --- * in terms of what the standard approach should be, we live in an imperfect world and package management has been done "wrong" in many ecosystems, but in an ideal world I think the "correct" solution here should be: (1) If it's an end user tool it should be a self contained binary or it should be a system package installed via the package manager (which will manage any ancillary dependencies for you) (2) If it's a dev tool (which, if you're cloning a cpp repo & building binaries, it is), it should not touch anything systemwide. Whatsoever. This often results in a README with manual instructions to install deps, but there are many good automated ways to approach this. E.g. for CPP this is a solved problem with Conan Profiles. However that might incur significant maintenace overhead for the Unsloth guys if it's not something the ggml guys support. A dockerised build is another potential option here, though that would still require the user to have some kind of container engine installed, so still not 100% ideal.

danielhanchen 1y ago
I would like to be in (1) but I'm not a packaging person so I'll need to investigate more :( (2) I might make the message on installing llama.cpp maybe more informative - ie instead of re-directing people to the docs on manual compilation ie https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-do-i-manually-save-to-gguf https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-..., I might actually print out a longer message in the Python cell entirely Yes we're working on Docker! https://hub.docker.com/r/unsloth/unsloth https://hub.docker.com/r/unsloth/unsloth
- lucideer 1y ago
  > Yes we're working on Docker! That will be nice too, though I was more just referring to simply doing something along the lines of this in your current build: docker run conanio/gcc11-ubuntu16.04 make clean -C llama.cpp etc etc... (likely mounting & calling a sh file instead of passing individual commands) --- Although I do think getting the ggml guys to support Conan (or monkey patching your own llama conanfile in before building) might be an easier route.
  - danielhanchen 1y ago
    Oh ok I'll take a look at conanfiles as well - sorry I'm not familiar with them!

solarkraft 1y ago

I like it when software does work for me. Quietly installing stuff at runtime is shady for sure, but why not if I consent?

danielhanchen 1y ago
Do you think it's ok for permissioning I guess? I might also add a 30 second timer and just bail out if there's no response from the user