gpjt

10 ms·

121.
The Business of the AI Labs(blog.omega-prime.co.uk)
19 points by gpjt 1y ago|3 comments
122.
by gpjt 2y ago
Presumably that is limited by the gig or so of information in our DNA, though?
123.
by gpjt 2y ago
It does depend on where you enter the country, at least in my experience (UK citizen, have visited a lot both on visa waivers and prior to that on visas, since the 80s). Every time I've flown into Austin, TX, they've been super-ni
124.
by gpjt 2y ago
My intuition is very undeveloped on this, but it makes some kind of sense to me that dropout would make convergence slower, because you're ignoring a bunch of parameters in every batch. The goal seems to be to get a better, more gener
125.
by gpjt 2y ago
OP here -- I'm new at this, but I don't think so. A zero output from a neuron still contributes to the output. Taking a silly toy case, imagine a network with one input, one neuron, one output. You're training to it match
126.
by gpjt 2y ago
OP here -- that's the one! Highly recommended.
127.
Writing an LLM from scratch, part 10 – dropout(gilesthomas.com)
90 points by gpjt 2y ago|8 comments
128.
Adding /Llms.txt(gilesthomas.com)
1 points by gpjt 2y ago|0 comments
129.
by gpjt 2y ago
I've found it very useful with organisational problems too. I had a complex issue to work through recently and tried working with ChatGPT, Claude and Grok 3 to optimise letters to some of the people involved to try to get things solved
130.
Writing an LLM from scratch, part 9 – causal attention(gilesthomas.com)
4 points by gpjt 2y ago|0 comments
131.
by gpjt 2y ago
I have 100% set myself a "no side quests" rule while going through the book so that I don't do that. I've had... patchy success with that, but I think I'm doing pretty well apart from the week I spent getting LaTeX
132.
by gpjt 2y ago
OP here: my posts are kind of reading notes for the book, so I don't normally copy the code from there -- so you wouldn't have seen the packages used. So far there's been tiktoken for tokenization (Raschka shows how to write
133.
by gpjt 2y ago
I do see your point! But at the end of the day, it depends on where you want to spend your time. "Build an LLM from scratch" is over 300 pages -- and they are very dense pages. My blog post covers fewer than 10 of them (though
134.
by gpjt 2y ago
OP here: that is an excellent point! Of the eight read-throughs, four were on Friday night, then I had dreams involving some kind of TRON-like vector spaces, and at brunch on Saturday things seemed to start gelling. The four extra read-th
135.
Writing an LLM from scratch, part 8 – trainable self-attention(gilesthomas.com)
380 points by gpjt 2y ago|31 comments
136.
by gpjt 2y ago
Oh God, I just worked out the joke. Very funny, Kendrick :P
137.
by gpjt 2y ago
Sometimes, but you have to be careful. IME Claude and (to my surprise) Grok 3 are really good at understanding your style and adapting their suggestions to match. ChatGPT, by contrast, tries to change everything into some kind of corporate
138.
by gpjt 2y ago
Proofreading is one place that AI can actually be a friend rather than a foe. If you give Claude your draft and tell it explicitly to call out misspellings and grammatical errors only, it does a really good job.
139.
by gpjt 2y ago
Is your job super-safe? If so, that's awesome :-) The whole marketing thing only becomes important if you have to get a new one, and then it can become important very quickly.
140.
by gpjt 2y ago
Me too! These last two posts blogging about blogging are unusual for me. I'm working through a book (Sebastian Raschka's "Build an LLM from scratch") and posting about that at the moment. It's likely not a coinciden
141.
by gpjt 2y ago
The jokey last paragraph was probably based on a half-memory of having read that, now you mention it...
142.
by gpjt 2y ago
Good point. "Blog as much as you can, as otherwise the training set will be Twitter!"
143.
by gpjt 2y ago
Thanks! And that's a good counter-example... which made me realise, there's also https://waitbutwhy.com/
144.
by gpjt 2y ago
That's what I was trying to cover with the "make your newly-acquired knowledge concrete" bit, and was my focus in the previous post. This time around I wanted to look into the aspects that might be impacted by AI (and why I
145.
by gpjt 2y ago
Agreed. A typical post on my blog gets no comments. The mean is probably one because the posts that do get comments normally get three or four. The biggest dopamine hit I've had in 19 years of blogging was seeing this post hit #1 on
146.
by gpjt 2y ago
I just posted a follow-up about exactly that -- why I think it's still worth blogging even if AIs will copy everything you say. Would love to know your thoughts! Here's a link: https://news.ycombinator.com/item?i
147.
by gpjt 2y ago
You're right, I should have gone into graphic detail about the medical stuff ;-)
148.
by gpjt 2y ago
TBH I think there's value in posting even if AIs are going to steal it -- I did a follow-up post trying to express why that is: https://news.ycombinator.com/item?id=43166761
149.
It’s still worth blogging in the age of AI(gilesthomas.com)
333 points by gpjt 2y ago|222 comments
150.
by gpjt 2y ago
Good point! I can never remember the steps to install the Ubiquiti management app on Arch after repaving my workstation (which I do ever other year or so). But I blogged about it in 2019, so I use that as a reminder.

More ›