jart

39 ms·

721.
by jart 3y ago
Still trying to figure out what's wrong with the 8B tokenizer. Use https://huggingface.co/jartine/Meta-Llama-3-70B-Instruct-lla... for now, since I managed to get 70B working well. If you want to use the buggy 8B
722.
by jart 3y ago
AMD introduced BMI2 in Excavator c. 2015.
723.
by jart 3y ago
Any Macbook with 32GB should be able to run Meta-Llama-3-70B-Instruct.Q2_K.llamafile which I uploaded a few minutes ago. It's smart enough to solve math riddles, but at this level of quantization you should expect hallucinations. If yo
724.
by jart 3y ago
That's like saying Occupy Wall Street should have protested in the Bronx, so they wouldn't disturb Wall Street.
725.
by jart 3y ago
It's probably due to Chrome penalizing the connection for not being HTTPS
726.
by jart 3y ago
ahgamut is a college student. He doesn't have $15,000 workstations like us.
727.
by jart 3y ago
Software linked with Cosmopolitan Libc usually goes 2x faster than Musl. - https://justine.lol/cosmo3/ - https://twitter.com/JustineTunney/status/1726141024597324189... What exactly looked sl
728.
by jart 3y ago
Here's the code I use: https://github.com/jart/cosmopolitan/blob/master/net/http/to...
729.
by jart 3y ago
Five eyes probably kills anyone who tries.
730.
by jart 3y ago
It's so easy to crush ddos with token buckets that usually the only thing I need my cat to wake me for is when my Discord gets raided.
731.
by jart 3y ago
Just because PDF files can't hijack your domain, doesn't mean they can't spy on you. Unfortunately, there isn't really an open source tool for sanitizing PDF files.
732.
by jart 3y ago
My analysis as an outsider is that Charles Hannum joined the NetBSD dev team a month after it was founded and his contributions were almost exclusively code cleanup changes. By 1995 he was writing bug fixes and features like SA_RESETHAND an
733.
by jart 3y ago
When I was a teenager, my curiosity led me to do urban exploring, so I felt a lot more pulled in by freedomben's story than the one about cordless phones. If there were photos, it'd definitely be blog post worthy. Although that mi
734.
by jart 3y ago
If you zig zag encode the deltas you might be able to get it down to 10mb. δzd is the best. https://justine.lol/sizetricks/#dzd
735.
by jart 3y ago
> Reading code is underrated, and many software engineers don’t understand how and why they should read it. Reading code is much harder than writing code because writing code translates your thoughts into code, and reading code is the op
736.
by jart 3y ago
It works better if the people are preserved too, just like Pompeii; otherwise it's just a dollhouse. You should shelf your proposal until we have the technology for cloning and resurrection.
737.
by jart 3y ago
The code that launches my code (see ggml_compute_forward_mul_mat) comes after CLBLAST, Accelerate, and OpenBLAS. The latter take precedence. So if you're not seeing any speedup in enabling them, it's probably because tinyBLAS ha
738.
by jart 3y ago
llamafile author here. I'm downloading Mixtral 8x22b right now. I can't say for certain it'll work until I try it, but let's keep our fingers crossed! If not, we'll be shipping a release as soon as possible that get
739.
by jart 3y ago
It sounds like Torvalds has finally found a way to delay the Rust developers taking over the kernel.
740.
by jart 3y ago
Here's a complete working example for POSIX systems on how to reproduce my llamafile tinyBLAS vs. MKL benchmarks: https://gist.github.com/jart/640231a627dfbd02fb03e23e8b01e59... This new generalized kernel does ev
741.
by jart 3y ago
I was part of the Google Brain team working on TensorFlow back in 2015. Please note that doesn't mean I understand how LLMs work. Although I do know how matrix multiplication works now. At least until I apply my focus to the next area
742.
by jart 3y ago
Why are you advocating for the government to take control of open source projects? Is anyone here naïve enough to believe that, after being persuaded of the national security interests of these projects, they're just going to hand over
743.
by jart 3y ago
If anyone wants to follow along with EPYC optimizations, please subscribe to: https://github.com/ggerganov/llama.cpp/pull/6412#issuecommen...
744.
by jart 3y ago
BLIS does that in their kernels. I've tried doing that but was never able to get something better than half as good as MKL. The BLIS technique of tiling across k also requires atomics or an array of locks to write output.
745.
by jart 3y ago
The inequality between matrix multiplication implementations is enormous. It gets even more extreme on GPU where I've seen the difference between naïve and cuBLAS going as high as 1000x. Possibly 10000x. I have a lot of faith in myself
746.
by jart 3y ago
There's a simple fix for that. sudo wget -O /usr/bin/ape https://cosmo.zip/pub/cosmos/bin/ape-$(uname -m).elf sudo chmod +x /usr/bin/ape sudo sh -c "echo &#
747.
by jart 3y ago
People don't always reveal the true reason they want to destroy something.
748.
by jart 3y ago
https://layoffs.fyi/
749.
by jart 3y ago
I thought the original mantra was social mania, after an order came down from the prince of darkness.
750.
by jart 3y ago
If the open source software interacts with the data brokers as you, then how would it do that without giving up even more of your personal information? Services like Optery also lose the ability to remove your information from a data broker

More ›