adeon

11 ms·

91.
by adeon 4y ago
I think implying that GPL is not "fully open source" is a hot take. It's specifically designed to ensure you and anyone you distribute your code gets the same freedoms. Maybe you don't agree that it's a good license
92.
by adeon 4y ago
I've followed updates on this project r/machinelearning and for me the existence of projects like this is some good evidence that the OpenAI moat is not that strong. It gives some hope you are not going to need massive huge comput
93.
by adeon 4y ago
I have not, but I want to in near future because I'm really curious myself too. I've been following Rust community that now has llama.cpp port and also my OpenCL thing and one discussion item has been to run a verification and com
94.
by adeon 4y ago
I ran it on a 128 RAM machine with a Ryzen 5950X. It's not fast, 4 seconds per token. But it's just about fits without swapping. https://github.com/Noeda/rllama/
95.
by adeon 4y ago
I think the LLaMA-7B in general might not just be very good. I've been playing around and run full non-quantized LLaMA-30B and LLaMA-7B in a bunch of experiments and I think the quality of output is much, much better in LLaMA-30B.
96.
by adeon 4y ago
As far as I know, the two ggml ones are basically just llama.cpp ports that include the ggml source code so if the support is not in llama.cpp, I don't think it's in these implementations either. Although maybe that also means th
97.
by adeon 4y ago
I've counted three different Rust LLaMA implementations on r/rust subreddit this week: https://github.com/Noeda/rllama/ (pure Rust+OpenCL) https://github.com/setzer22/llama-rs/
98.
by adeon 4y ago
Yeah, it does seem like there's a fundamental limit how fast you can go even if you engineer the data juggling to perfection. My guess is that every loop through the transformer is going to have to visit every weight and if those weigh
99.
by adeon 4y ago
If you are interested in implementing LLaMA yourself or learning, I noticed that the reference code by Facebook is one of the cleaner, easier to read ML code I've seen in a while. https://github.com/facebookresearch&#x2
100.
by adeon 4y ago
Also, the game is $50 in the US Steam store. This is an oof price for a broken early access game. Full AAA games are often "just" $60.
101.
by adeon 4y ago
Has anyone had success being approved to download weights even if you are just a hobbyist with a big GPU? (I'm here about to begrudgingly fill out the Google Form) Asking in general, not just for this particular model. I've been f
102.
by adeon 4y ago
I don't know about these large models but I saw on a random HN comment earlier in a different topic where someone showed a GPT-J model on CPU only: https://github.com/ggerganov/ggml I tested it on my Linux and Mac
103.
by adeon 4y ago
+1 agreed. There are some pretty crappy bloated client-side apps but when it's done well and it is appropriate for the app in question, it's amazing. I've been playing novelai.net text generation and I think their app is most
104.
by adeon 4y ago
There's a cool technique called "negative" prompt that I didn't see mentioned. Some of the community projects have implemented it. (I use https://github.com/AUTOMATIC1111/stable-diffusion-webui loca
105.
by adeon 4y ago
Mallard Basic and QBasic is how I got into programming. I'm glad people still have nostalgia for QBasic. Even by today's standards I think it's impressive how QBasic was a somewhat complete IDE with a built-in help menu you w
106.
by adeon 4y ago
There were many things I liked about Nim: * The compiled programs run pretty fast. * Compiling is pretty fast too (compared to Rust or Haskell). * It's easy for me to bind to whatever C library I have even if nobody made nice Nim packa
107.
by adeon 4y ago
I like Go a lot. At my previous employer a lot of the systems we wrote for log processing were written in Golang. And I thought it was really nice; I can compile my code really quickly and code in general will run pretty fast. I've onl
108.
by adeon 4y ago
I think you are pretty spot on here. I write Rust and Haskell most of the time, where the choice depends on the kind of project I'm working on. Also I've written Rust professionally but Haskell has been confined to random hobby pr
109.
by adeon 4y ago
I wish there was a Rust-like programming language that was just a little bit higher level than Rust. I like Rust's wide ecosystem with high quality packages, the nice type system, traits, the idea that my code generally runs pretty fas