jart

47 ms·

1. 931.▲
   Bash One-Liners for LLMs(justine.lol)

   12 points by jart 3y ago|1 comments
2. 932.▲
   by jart 3y ago

   Thanks!
3. 933.▲
   by jart 3y ago

   It's so great. I've been this vision model to rename all the files in my Pictures folder. For example, the one-liner: llamafile --temp 0 \ --image ~/Pictures/lemurs.jpg \ -m llava-v1.5-7b-Q4_K.gguf
4. 934.▲
   by jart 3y ago

   Yes but they claim it runs on Linux when it runs on Systemd.
5. 935.▲
   by jart 3y ago

   You'd think Google would devote its resources to improve search, but instead they're paying journalists (who hate tech) to forge the very chains that bind us.
6. 936.▲
   by jart 3y ago

   Kudos for saying no. I can't believe we're living in an age where open source developers are being asked openly to leak the private conversations of their users to the public.
7. 937.▲
   by jart 3y ago

   Why? $7k for a Mac Studio isn't much if we consider the original IBM PC adjusted for inflation cost $6k.
8. 938.▲
   by jart 3y ago

   I'm not able to use this locally on Alpine Linux because ollama needs glibc.
9. 939.▲
   by jart 3y ago

   The same thing happened at Google. Company started falling apart around the time the SWEs got obsessed with Braco.
10. 940.▲
    by jart 3y ago

    > You don't need specific hardware, it will run on non-TPM devices as well. The article says that if you don't buy a new computer with TPM 2.0 then you have to migrate to "the cloud" in order to use Windows 11. What t
11. 941.▲
    by jart 3y ago

    Thanks!
12. 942.▲
    by jart 3y ago

    I thought I was. It's just no one told me about the Lobsters policy. I consistently violated the Lobsters policy for several years and no one warned me. I'm just someone who quit her job at Google Brain to build open source code a
13. 943.▲
    by jart 3y ago

    Lobsters doesn't have any contact information (I later learned they have an IRC channel) so I didn't get to hear an explanation from the Lobsters moderator until the topic came up here and ended up getting mediated in a Hacker New
14. 944.▲
    by jart 3y ago

    Looking at https://explore2.marginalia.nu/search?domain=simonwillison.n... now that's an interesting service. The web has felt isolating since it became commercialized. Bloggers are living in the Google dark ages right
15. 945.▲
    by jart 3y ago

    Check out the llamafile 0.2.1 release. Old Intel CPU support is fully fixed now on the release page and hugging face. Enjoy!
16. 946.▲
    by jart 3y ago

    It's back. Sorry about that.
17. 947.▲
    by jart 3y ago

    Floating divs killed the Dreamweaver. CSS came and broke your heart. Oh, a, a, a, oh.
18. 948.▲
    by jart 3y ago

    You're awesome!
19. 949.▲
    by jart 3y ago

    So you have a Sandybridge processor with AVX support (but not AVX2). Could you open llamafile in a debugger and find out which instructing is faulting and what its address in memory is? I haven't personally tested Sandybridge but I&#x2
20. 950.▲
    by jart 3y ago

    We have an issue here tracking this: https://github.com/Mozilla-Ocho/llamafile/issues/14 Please follow that issue for updates.
21. 951.▲
    by jart 3y ago

    I'm assuming you're on Apple Silicon? Please follow https://github.com/Mozilla-Ocho/llamafile/issues/11 which is tracking this. We've received multiple reports even though I personally haven&#x
22. 952.▲
    by jart 3y ago

    You need to upgrade to zsh 5.9+ or run `sh -c ./llamafile`. See the Gotchas section of the README.
23. 953.▲
    by jart 3y ago

    Simon Willison's blog post has a screenshot. It's worth a read. https://simonwillison.net/2023/Nov/29/llamafile/
24. 954.▲
    by jart 3y ago

    I've received a lot of good advice today on how we can potentially improve our Nvidia story so that nvcc doesn't need to be installed. With a little bit of luck, you'll have releases soon that get your GPU support working.
25. 955.▲
    by jart 3y ago

    The APE shell script needs to run /bin/mkdir in order to map the embedded ELF executable in memory. It should be possible for you to work around this on Linux by installing our binfmt_misc interpreter: sudo wget -O /usr
26. 956.▲
    by jart 3y ago

    Could you disable SIP and run `lldb -- $TMPDIR/.ape-1.8 ./llamafile-server-0.1-llava-v1.5-7b-q4` and give me (1) the name of the instruction that's illegal (or its hex value) and (2) the hex address of where that instruction
27. 957.▲
    by jart 3y ago

    Correct. Apple Silicon GPU performance should be equally fast in llamafile as it is in llama.cpp. Where llamafile is currently behind is at CPU inference (only on Apple Silicon specifically) which is currently going ~22% slower compared to
28. 958.▲
    by jart 3y ago

    That's a nice model that fits comfortably on Raspberry Pi. It's also only a few days old! I've just finished cherry-picking the StableLM support from the llama.cpp project upstream that you'll need in order to run these
29. 959.▲
    by jart 3y ago

    Fish is another cool shell I got to help improve two years ago by upstreaming a patch for this. So long as you're using a recent version, you should be golden (provided binfmt_misc doesn't cause any issues). Let us know what you t
30. 960.▲
    by jart 3y ago

    You can manually download the `ape` command from https://cosmo.zip/pub/cosmos/bin/ Please see the Gotchas section of the README for the copy/pastable commands you can run: https://github.com&#