AMICABoard

12 ms·

31.
by AMICABoard 2y ago
Will do! Thanks :)
32.
by AMICABoard 2y ago
:)
33.
by AMICABoard 2y ago
Hmm yeah, it started as fork of karpathy's llama2.c and some experiments. So it is an abomination I agree.
34.
by AMICABoard 2y ago
Thanks, I'll read it up. Interesting.
35.
by AMICABoard 2y ago
Actually "My cat is funny" was the prompt it continued that. I got to fix some stuff to reflect meta's implementation and also fix the chat mode, then it would be usable. Will take a few days to do that.
36.
by AMICABoard 2y ago
Interesting, I heard something like that, but now I must read about it.
37.
by AMICABoard 2y ago
Lol :)
38.
by AMICABoard 2y ago
To be honest your pr and these notes are super helpful cos, otherwise I'd have been too lazy to read up the original implementation, but I can't merge it. Will make the fix soonish and credit it to you. I'll tell you a secret
39.
by AMICABoard 2y ago
Llama.cpp is the king, this is just a lowly wanna be peasant. But some day it will reach there.
40.
by AMICABoard 2y ago
I think the non English part is mostly hit and miss in this primitive version, probably cos the implementation is not correct. I got to read up a lot and fix it.
41.
by AMICABoard 2y ago
Maybe we should make "Brain Damage Factor" a official term to denote how much types of quantizations degrade output compared to unquantized.:)
42.
by AMICABoard 2y ago
Ja Amiga 500! My first computer. Still in love with her...:)
43.
by AMICABoard 2y ago
Oh thanks bro, nope it uses the simple llama 2 rope with tetha changed to 50k to match llama 3's. I'll check your python PR, have a deeper look at the meta llama 3 & 3.1 implementation and hack together something soonish. Awes
44.
by AMICABoard 2y ago
Okay but hold your horses. Still a bit buggy. Sample output: Meta's Llama 3.1 models can output multilingual text which is awesome. Here are some examples output of 8 bit quantized 8b model with 100 token output (-n 100)... Quantizatio
45.
Llama 3.1 in C(github.com)
212 points by AMICABoard 2y ago|36 comments
46.
by AMICABoard 2y ago
Okay if anyone wants to try Llama 3.1 inference on CPU, try this: https://github.com/trholding/llama2.c (L2E) It's a bit buggy but it is fun. Disclaimer: I am the author of L2E
47.
by AMICABoard 2y ago
Hold your horses, it's still very buggy! Meta's Llama 3.1 models can output multilingual text which is awesome. Here are some example of 8 bit quantized 8b model with 100 token output (-n 100)... Quantization creates some brain da
48.
Single C file Llama 3.1 Support in Llama 2 Everywhere(github.com)
1 points by AMICABoard 2y ago|1 comments
49.
by AMICABoard 3y ago
Nice article. We did a demo for booting to LLM and also as Kernel Module: https://github.com/trholding/llama2.c The whole things was funny and buggy, but since then we have been developing in stealth, even trying to ra
50.
by AMICABoard 3y ago
https://github.com/fredrikwidlund/libreactorng
51.
by AMICABoard 3y ago
link please bro
52.
by AMICABoard 3y ago
Okay here is a wish list: Chip fan out modules. A repo of all popular chips with traces to pins fanned out and variations (Chip XXXX, Fanout A, Fanout B, Fanout C, Compact Fanout D etc) Ideally the fanout modules should have required decoup
53.
by AMICABoard 3y ago
Okay here is a feature that I'd like to see. This has a VS Code extension, but I don't use VS Code, I like to use other editors like Kate. Having a LSP Server and xml for syntax would be a great feature.
54.
by AMICABoard 3y ago
Really love this idea! I will give it a try. I have seen code to circuit before but this seems like it has potential.
55.
Juma the Jaguar(en.wikipedia.org)
1 points by AMICABoard 3y ago|0 comments
56.
by AMICABoard 3y ago
We are trying to create a proper OS that boots to an LLM. A toy demo is available in the releases: https://github.com/trholding/llama2.c Can we apply?
57.
by AMICABoard 3y ago
Which is a smaller model, that gives good output and that works best with this. I am looking to run this on lower end systems. I wonder if someone has already tried https://github.com/jzhang38/TinyLlama , could save me
58.
by AMICABoard 3y ago
This puts a super great evil happy grin on my face. I am going to add it in the next version of L2E OS! Thank you jart, thank you mozilla! Love you folks!
59.
by AMICABoard 3y ago
Oops they did it again! Massive fan, and I use it!
60.
by AMICABoard 3y ago
epic!

More ›