gwern

10 ms·

121.
by gwern 7mo ago
> They almost certainly have never seen regular conversations in Base64 in their training set, so its weird that it 'just works'. People use Base64 to store payloads of many arbitrary things, including web pages or screenshots,
122.
by gwern 7mo ago
> I encourage everyone to RTFA and not just respond to the headline. This is an example of an article which 'buries the lede'†. It should have started with the announcement of the new zlib autoformalization (!) https:/&
123.
by gwern 7mo ago
Insider trading is misappropriating knowledge. Not really misappropriation here. Maybe you're looking for 'moral hazard'?
124.
by gwern 7mo ago
(Pangram commentary: https://x.com/max_spero_/status/2028620466995220568 https://x.com/max_spero_/status/2028689219112034436 )
125.
by gwern 7mo ago
People are making a big deal of the fact that there's multiple prompts or interactions, but so what? That's true of source code too! The final commit doesn't reflect every last keystroke or thing you ever said while working o
126.
by gwern 7mo ago
One of the downsides of using an expert LLM to write for you is that they know all that perfectly well, even if you don't, and aren't too bothered by such a chunk. It's like reading any Wikipedia article on mathematics... Thi
127.
by gwern 7mo ago
Dude, you literally start the article with > Andrej Karpathy wrote a 200-line Python script that trains and runs a GPT from scratch, with no libraries or dependencies, just pure Python . Almost immediately afterwards, you have a section
128.
by gwern 7mo ago
The review queue is not going to work, unless you have millions of dollars to spare. No one is interested in providing such extremely laborious skilled labor to be thrown away on one-off tweaks of someone's self-promotion of their AI s
129.
by gwern 7mo ago
Naturally, when hobbyists spend a lot of time in a single block to get results (as they are unable to parallelize or meaningfully coordinate over multiple invocations of themselves, due to lacking key cognitive capabilities such as embeddin
130.
by gwern 8mo ago
Oh yes, this is blatantly AI-written. Whether this is ironic and discredits the claims is up to you. But the opening at least makes a good point: AI-written code is right now something of an embezzlement or theft or fraud. Like junk food, i
131.
by gwern 8mo ago
That was probably a mistake. The religion is great (and I say that as a staunch materialist atheist) and _Fall of Hyperion_ had a lot more TechnoCore and filling out the background than _Hyperion_.
132.
by gwern 8mo ago
When you are that spectacularly wrong, it is. If you can't update correctly, you probably shouldn't.
133.
by gwern 8mo ago
Specifically, Cochrane wrote: > On reflection I have started to worry again. In 10 to 20 years nobody will read anything any more, they just will read LLM digests. So, the single most important task of a writer starting right now is to g
134.
by gwern 8mo ago
There is an interesting AI point here: the US Copyright Office recently tried to argue that images generated by a model could not be copyrighted, no matter how detailed the prompt nor how curated, because the artist did not envision the ex
135.
by gwern 8mo ago
Oh, if you meant what is the progress indicator itself, see https://gwern.net/design-graveyard#ordinal-word-counts
136.
by gwern 8mo ago
Attacks can be chained, and this can all be automated. For example, imagine pigbutchering scams... except it's there, similar to some voice-cloning scams, just to get enough data to stylometrically fingerprint you for future reference.
137.
by gwern 8mo ago
You should write all that up! (I've been around for a while and I never used Flickr except as a casual, but I did know about a lot of mashups and even spent time using FlickrLickr for Wikipedia; yet I've never heard of... any of t
138.
by gwern 8mo ago
The SMPY page is essentially an annotated bibliography: fulltext + abstract + commentary. Since the papers are of highly varied quality, it doesn't much make sense to try to put any particular confidence on it; I am convinced of some t
139.
by gwern 8mo ago
> I'd note that there are many studies showing that the usual outcomes for gifted kids are not all that great. No, there's not, and they do do great. And this goes back to Terman: there's a handful of highly selected exam
140.
by gwern 8mo ago
I enjoyed the reward-hacking one: https://pagedout.institute/download/PagedOut_008.pdf#page=59
141.
by gwern 8mo ago
K-V caches are large, but hidden states aren't necessarily that large. And if you can run a model once ridiculously fast, then you can loop it repeatedly and still be fast. So I wonder about the 'modern RNNs' like RWKV here..
142.
by gwern 8mo ago
> Ideally, I'd like to continue working on it to build something that can help clubs make minimal, inexpensive changes while maximally improving the strategic interest if the way the course plays. Yes, that's why I mentioned t
143.
by gwern 8mo ago
I'm not sure this is even measuring LLMs in the first place! They say the definition is "big data analytics and AI". Is putting Google Analytics onto your website and pulling a report 'big data analytics'...?
144.
by gwern 8mo ago
OK, so assuming you clean that up a bit and this becomes officially supported in SingleFile/SingleFileZ, what is missing compared to Gwtar? Anything important or just optional features like image recompression and PAR2?
145.
by gwern 8mo ago
Allowing more metadata might be useful. You can add anything to the manifest at build time as assets are not required to be loaded or ever used (because this is impossible to statically check). I suppose we'd have to define an official
146.
by gwern 8mo ago
If those were guaranteed inaccessible, wouldn't a web browser be within its rights to optimize those away?
147.
by gwern 8mo ago
My immediate thought is that OP is reinventing dynamic programming/RL from first principles. The final visualization looks exactly like a standard value estimate heatmap. Golf is a MDP over all the physical points on the course, with s
148.
by gwern 8mo ago
But that's a reason you should expect it to stop working soon, just like all the older tricks like "my grandmother will die". If you have a universal 'blind' prompt which can increase performance a little bit... the
149.
by gwern 8mo ago
Yep: "The tower itself isn’t just a gimmick; it’s a physical presence that creates tension and anticipation the moment it hits the table. Add the Dark Horde expansion and suddenly the game isn’t just something you play—it’s something y
150.
by gwern 8mo ago
I think there's still something missing here. This is a strange place for ChatGPT to confabulate quotes: extracting short quotes from a short text blog post is about easy as it gets these days. GPT-5.2 Pro can handle tens of thousands

More ›