4 ms·

$ chmod +x llamafile-server-0.1-llava-v1.5-7b-q4 $ ./llamafile-server-0.1-llava-v1.5-7b-q4 run-detectors: unable to find an interpreter for ./llamafile

by lol768 3y ago

$ chmod +x llamafile-server-0.1-llava-v1.5-7b-q4 $ ./llamafile-server-0.1-llava-v1.5-7b-q4 run-detectors: unable to find an interpreter for ./llamafile-server-0.1-llava-v1.5-7b-q4 Hmm. Did I do something wrong? (Ubuntu 22.04 / ) Installing the portable binfmt_misc gets me further, but still: $ ./llamafile-server-0.1-llava-v1.5-7b-q4 zsh: permission denied: ./llamafile-server-0.1-llava-v1.5-7b-q4 $ sh -c ./llamafile-server-0.1-llava-v1.5-7b-q4 sh: 1: ./llamafile-server-0.1-llava-v1.5-7b-q4: Permission denied

phh 3y ago
Last thing you need is to chmod +x the interpreter: chmod +x /usr/bin/ape (it is indeed not in the README)
- lol768 3y ago
  This was it, wget creates the file and it's non-executable (and I'd already double checked the actual llamafile was executable, but had missed this) - thanks!
stavros 3y ago
I get the same error, and there's no `ape` file to make excecutable, hm.
- jart 3y ago
  You can manually download the `ape` command from https://cosmo.zip/pub/cosmos/bin/ https://cosmo.zip/pub/cosmos/bin/ Please see the Gotchas section of the README for the copy/pastable commands you can run: https://github.com/mozilla-Ocho/llamafile#gotchas https://github.com/mozilla-Ocho/llamafile#gotchas
jart 3y ago
You can solve the run-detectors issue with: sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" You can solve the zsh permission denied issue by either (1) upgrade to zsh 5.9+ (I upstreamed a fix for this bug in zsh two years ago) or (2) use the sh -c workaround you discovered. If that one doesn't work, then it likely needs to be chmod +x. If the execute bit is set, and your sh still isn't working, then please let me know, because I'm not aware of any sh that still doesn't support APE. See the Gotchas section of the README https://github.com/mozilla-Ocho/llamafile#gotchas https://github.com/mozilla-Ocho/llamafile#gotchas
- stavros 3y ago
  That worked, thanks Justine! I use fish, so I didn't get a zsh error, but I had missed the Gotchas section (and the README), so this helps!
  - jart 3y ago
    Fish is another cool shell I got to help improve two years ago by upstreaming a patch for this. So long as you're using a recent version, you should be golden (provided binfmt_misc doesn't cause any issues). Let us know what you think of llamafile!
    - stavros 3y ago
      Thank you, I really like it! It's a very clever way to get LLMs deployed, and with Cosmopolitan, I don't need to point people to different downloads for the same LLM. Excellent job.
      - jart 3y ago
        Thanks!
- pmarreck 3y ago
  Yet another jart tour-de-force. I knew I had to sponsor you on Github back when I read your magnificent technical breakdown of APE, lol. (sorry for OT!)
  - jart 3y ago
    You're awesome!

by lol768 3y ago

phh 3y ago

Last thing you need is to chmod +x the interpreter: chmod +x /usr/bin/ape (it is indeed not in the README)

lol768 3y ago
This was it, wget creates the file and it's non-executable (and I'd already double checked the actual llamafile was executable, but had missed this) - thanks!

stavros 3y ago

I get the same error, and there's no `ape` file to make excecutable, hm.

jart 3y ago
You can manually download the `ape` command from https://cosmo.zip/pub/cosmos/bin/ https://cosmo.zip/pub/cosmos/bin/ Please see the Gotchas section of the README for the copy/pastable commands you can run: https://github.com/mozilla-Ocho/llamafile#gotchas https://github.com/mozilla-Ocho/llamafile#gotchas

jart 3y ago

You can solve the run-detectors issue with: sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf sudo sh -c "echo ':APE:M::MZqFpD::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" sudo sh -c "echo ':APE-jart:M::jartsr::/usr/bin/ape:' >/proc/sys/fs/binfmt_misc/register" You can solve the zsh permission denied issue by either (1) upgrade to zsh 5.9+ (I upstreamed a fix for this bug in zsh two years ago) or (2) use the sh -c workaround you discovered. If that one doesn't work, then it likely needs to be chmod +x. If the execute bit is set, and your sh still isn't working, then please let me know, because I'm not aware of any sh that still doesn't support APE. See the Gotchas section of the README https://github.com/mozilla-Ocho/llamafile#gotchas https://github.com/mozilla-Ocho/llamafile#gotchas

stavros 3y ago
That worked, thanks Justine! I use fish, so I didn't get a zsh error, but I had missed the Gotchas section (and the README), so this helps!
- jart 3y ago
  Fish is another cool shell I got to help improve two years ago by upstreaming a patch for this. So long as you're using a recent version, you should be golden (provided binfmt_misc doesn't cause any issues). Let us know what you think of llamafile!
  - stavros 3y ago
    Thank you, I really like it! It's a very clever way to get LLMs deployed, and with Cosmopolitan, I don't need to point people to different downloads for the same LLM. Excellent job.
    - jart 3y ago
      Thanks!
pmarreck 3y ago
Yet another jart tour-de-force. I knew I had to sponsor you on Github back when I read your magnificent technical breakdown of APE, lol. (sorry for OT!)
- jart 3y ago
  You're awesome!