jart

53 ms·

901.
by jart 3y ago
Author here. Thanks for sharing your concern. Mozilla is funding my work on llamafile and Emacs Copilot because Mozilla wants to help users to be able to control their own AI experiences. You can read more about the philosophy of why we&#x2
902.
by jart 3y ago
Interesting project. I can't help but ask if you've ever considered how Richard Stallman would feel about people configuring Emacs to upload code into OpenAI's cloud. It amazes me even more that you get people to pay to do it
903.
by jart 3y ago
Author here. It does make use of mmap(). I worked on adding mmap() support to llama.cpp back in March, specifically so I could build things like Emacs Copilot. See: https://github.com/ggerganov/llama.cpp/pull/
904.
by jart 3y ago
I've just added a video demo to the README. It takes several seconds the first time you do a completion on any given file, since it needs to process the initial system prompt. But it stores the prompt to a foo.cache file alongside your
905.
by jart 3y ago
llamafile has an HTTP server mode with an OpenAI API compatible completions endpoint. But Emacs Copilot doesn't use it. The issue with using the API server is it currently can't stream the output tokens as they're generated.
906.
by jart 3y ago
Yes and regulation will ensure non-commercial activities stay that way. I write open source code because I love building software tools and there aren't a whole lot of opportunities to commercialize my work. I'd love it if an oppo
907.
by jart 3y ago
It's a price increase, probably due to inflation, which Amazon is benevolently not imposing upon consumers, who like ads, or don't care about ads. I'll tell you what I am upset about though. Where's my one click buy butt
908.
by jart 3y ago
They should have just opened up a cafeteria. The root problem is that you need someone to serve you in order to eat. It's bourgeois and you get bourgeois customs as a result.
909.
by jart 3y ago
Tipping is basically a soft form of bribery, which isn't great at all. When it becomes mandatory, it's even worse.
910.
by jart 3y ago
The terminal bell is dead; history has killed it.
911.
by jart 3y ago
> https://redditmap.social/ This is the best thing I've ever seen.
912.
by jart 3y ago
In order to make digital data durable, you have to store it on cassette tape. This has the same issue as physical stuff in the sense that you have to wait for it to be recalled before you can access its data, which is why tape is normally o
913.
by jart 3y ago
That's the real issue then. Nothing would stimulate automation faster than suicidal pilots.
914.
by jart 3y ago
I honestly don't see why it's so hard. On my project we wrote our own gemm kernels from scratch so llama.cpp didn't need to depend on cublas anymore. Only took a few days and a few hundred lines of code. We had to trade away
915.
by jart 3y ago
It doesn't matter how much money USENET makes one company, what matters is what it represents. Surrendering USENET entirely to abuse means we're losing one of the last vestiges of the ARPANET. It brings great shame and dishonor th
916.
by jart 3y ago
It's true. Nothing says middle class more than toting around the shiny silver status symbol.
917.
by jart 3y ago
Sounds like the barbarians are sacking Rome. What happened to mighty Google's thirst for blood and will to dominate the Internet? It feels like they've just started accepting being helpless to stop all this spam.
918.
by jart 3y ago
I know it's arm64 but you can hack Mac Studio and put Linux on it. Putting Linux on Apple Silicon makes it go so much faster than MacOS if you're doing command line development work. With x86 there really isn't anything you c
919.
by jart 3y ago
I think it has more to do with how close to the brink you are. It takes at least a decade for a technology to mature to the point where there's a polished point and click gui for doing it. It sounds like Borg just hit that inflection p
920.
by jart 3y ago
`--temp 0` makes it deterministic. What can make output reliable is `--grammar` which the blog post discusses in detail. It's really cool. For example, the BNF expression `root ::= "yes" | "no"` forces the LLM to on
921.
by jart 3y ago
Did you try running it from cmd.exe or powershell? Could you try passing the --strace (or possibly --ftrace) flag and see what happens?
922.
by jart 3y ago
Has anyone written a shell script before that uses a local llm as a composable tool? I know there's plenty of stuff like https://github.com/ggerganov/llama.cpp/blob/master/examples/... where th
923.
by jart 3y ago
llamafile is basically just llama.cpp except you don't have to build it yourself. That means you get all the knobs and dials with minimal effort. This is especially true if you download the "server" llamafile which is the fas
924.
by jart 3y ago
Which instructions did you follow? The blog post linked here only talks about the command line interface.
925.
by jart 3y ago
llamafile doesn't use docker. It's just an executable file. You download your llamafile, chmod +x it, then you ./run it. It is possible to use Docker + llamafile though. Check out https://github.com/ajbouh&#x2
926.
by jart 3y ago
Thanks for saying that. The last part of my blog post talks about how you can run Rocket 3b on a $50 Raspberry Pi 4, in which case llamafile goes 2.28 tokens per second.
927.
by jart 3y ago
llamafile can provide an abstraction, but ultimately it boils down to how the model was trained and/or fine-tuned.
928.
by jart 3y ago
I fixed that bug in zsh two years ago. It got released in zsh 5.9. https://github.com/zsh-users/zsh/commit/326d9c203b3980c0f841... I'm going to be answering questions about it until my dying day.
929.
by jart 3y ago
Yes you need to install CUDA and MSVC for GPU. But here's some good news! We just rolled our own GEMM functions so llamafile doesn't have to depend on cuBLAS anymore. That means llamafile 0.4 (which I'm shipping today) will h
930.
by jart 3y ago
I second that. I can't count how many times Raymond Chen has helped me understand things while working on Windows polyfills for Cosmopolitan Libc. Microsoft is very fortunate to have him.

More ›