croqaz

4 ms·

1.
Making a vintage LLM from scratch; Take #2(crlf.link)
2 points by croqaz 13d ago|1 comments
2.
by croqaz 13d ago
I built a pile of new datasets, 3 new vintage models, a benchmark, an evaluation pipeline, experiments and a very good vintage tokenizer. Demo play with the vintage-LLM: https://huggingface.co/spaces/croqaz/Piston-
3.
by croqaz 4mo ago
Thank you very much! It is humbling and motivating to see other people interested in this.
4.
by croqaz 4mo ago
Do share! I read all the blog posts where people share their experiences of building small scale LLMs "from scratch".
5.
by croqaz 4mo ago
That's a fair point TBH. I said in my post that this LLM is first of all a learning project and I skipped an important step: the training loop. But on the other hand, how many data scientists are writing their own training loops? Is i
6.
by croqaz 4mo ago
"A more granular followup would be cool too" Do you mind expanding this question? More granular in what way? what would you like to know that is missing from the post?
7.
by croqaz 4mo ago
That's exactly what I had in mind. When I started this, I was jumping back and forth between this thought: "Can this model size actually generate logical English text?" and I played with a few different models of the same siz
8.
by croqaz 4mo ago
It looks like ROT13 text to me, I hope it's not Welsh. Don't want to offend anyone if that's their actual language :)
9.
Making a vintage LLM from scratch(crlf.link)
104 points by croqaz 4mo ago|29 comments
10.
by croqaz 4mo ago
I am creating my tiny Llama 340M base model from scratch. If you're curious about the steps, challenges and cost, read on. I am still working on the instruct model.
11.
TwoFold (2f), CLI text expander/template engine(github.com)
17 points by croqaz 1y ago|1 comments
12.
by croqaz 1y ago
TwoFold is a small command line app that allows plain text files to behave like dynamic files. It is a hybrid between a text expande, a template engine and a mini programming language.
13.
by croqaz 4y ago
What are web snapshots; comparison of the most popular methods: WARC, HTML, rrWeb and a better alternative called "recorded".
14.
Blocking adware, malware, and tracking sites(crlf.link)
1 points by croqaz 5y ago|2 comments
15.
by croqaz 5y ago
DYI blocking, a few methods
16.
Harden and secure browsers in containers, with GUI(crlf.link)
58 points by croqaz 5y ago|31 comments
17.
Tomb: File Encryption on GNU/Linux(dyne.org)
6 points by croqaz 5y ago|2 comments
18.
by croqaz 5y ago
Tomb consists of a simple shell script (Zsh) using standard filesystem tools (GNU) and the cryptographic API of the Linux kernel (cryptsetup and LUKS).
19.
Things that only someone who has been programming 20 years would know(quora.com)
23 points by croqaz 5y ago|7 comments
20.
by croqaz 6y ago
The same in Romania. In the 90s, everyone I know had to learn how to write with a fountain pen. For me at least, I write horribly when I use a ballpoint pen, vs a fountain pen. Because the ballpoint slides on the paper much too easy, like o
21.
by croqaz 6y ago
I love my Plaisir too! I have a green color and of course, it's always inked with green :)
22.
by croqaz 6y ago
Wow, I had no idea my crappy post ended on Hacker news! I just collected a bunch of info and links for myself :) I guess I have to update the post with some pictures, at least...