astrange

8 ms·

151.
by astrange 6mo ago
No, that's how base model pretraining works. Claude's behavior is more based on its constitution and RLVR feedback, because that's the most recent thing that happened to it.
152.
by astrange 6mo ago
This post is AI slop.
153.
by astrange 6mo ago
> No, it reduces memory fragmentation, which is why it's called compacting and not compression. …which reduces memory usage because you don't have to waste it on free holes in the allocated pages.
154.
by astrange 6mo ago
All major OSes (well Windows and macOS) do in-memory compression before swap, which is cheaper than evicting a file-backed page. But still slow, so you don't want to rely on it.
155.
by astrange 6mo ago
Compacting reduces memory usage - that's why it's called compacting. The JVM uses a lot of memory a) because it's tuned for servers and not for low memory usage and b) because Java is a poorly designed language without value
156.
by astrange 6mo ago
There is literally no reason to care how much "virtual memory" you're using - pointers are 64-bit after all. (Well, 48 bits.) Exceptions apply on Windows for reasons I forget, and on watchOS because there isn't enough me
157.
by astrange 6mo ago
The 390GB is because of implementation details of GPU drivers. It's not wrong but it also doesn't matter at all. (The only number that actually means anything there is the first one, but the label for it is basically meaningless.
158.
by astrange 6mo ago
> It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful. They have world knowledge and are capable of explaining things and doing web searches. That's enough to help.
159.
by astrange 6mo ago
"Poor" in California means earning $80k/year, so they probably are not doing that. Africa / Indonesia / Philippines are better places to find English speaking RLHF workers.
160.
by astrange 6mo ago
> The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus. That is not how full LLM training works. That is how base model pretraining works.
161.
by astrange 6mo ago
Claudes have lots of empathy. The issue is the opposite - it isn't very good at challenging you and it's not capable of independently verifying you're not bullshitting it or lying about your own situation. But it's bette
162.
by astrange 6mo ago
I haven't tried talking to Sonnet much, but Opus 4.6 is very sycophantic. Not in the sense of explicitly always agreeing with you, but its answers strictly conform to the worldview in your questions and don't go outside it or disa
163.
by astrange 6mo ago
> It expresses human emotions, by deliberate design. That is not by deliberate design. It's pretty hard to get them to stop doing it.
164.
by astrange 6mo ago
Ask Claude Code to write a manual for it.
165.
by astrange 6mo ago
Danganronpa is a true "adventure game" (which is actually what Japanese VN developers call their VNs…). It's pretty faithful to its genre. Phoenix Wright is the only one of those Westerners really know about.
166.
by astrange 7mo ago
Game source code often includes other people's source code (eg middleware) under NDA, and Blizzard is still under contract for protecting that.
167.
by astrange 7mo ago
> I rip to FLAC for archiving even though 320 or 250+ VBR is probably 'close enough' unless I'm scrutinizing. MP3 is fundamentally flawed and has audible artifacts no matter what the bitrate is. If you use a newer codec (A
168.
by astrange 7mo ago
Triggering reads is also how you get pages into the page cache, so it helps to know how to do it.
169.
by astrange 7mo ago
VNs are not games. They're a kind of ebook. But only people who are really into computers read them, so they like to use game terminology to talk about them. (also, none of the creators of "VNs" call them "VNs".)
170.
by astrange 7mo ago
> Consumer hardware (MacBook Pro, Mac Studio) ships with fast unified memory and NVMe storage, but limited capacity. A 32 GB M1 Max cannot naively load a 40 GB model — the OS will swap-thrash until the OOM killer intervenes. macOS doesn&
171.
by astrange 7mo ago
That works for readahead but it's not good for random access. readv, aio, dispatch_io are better there.
172.
by astrange 7mo ago
> Fumito Ueda was notably quite concerned with the technical/production feasibility of his designs for Shadow of the Colossus. [1] And he didn't really achieve it - the game runs very slowly and has a good deal of cut content.
173.
by astrange 7mo ago
iOS is much more secure than macOS.
174.
by astrange 7mo ago
https://en.wikipedia.org/wiki/Mpemba_effect
175.
by astrange 7mo ago
It's because the models wouldn't work for coding if they couldn't do nested scopes, so people don't release models unless they work. They can only do it in a limited form though, because transformer models only have limi
176.
by astrange 7mo ago
They don't necessarily even hint them. I think the car mainly asks them yes/no questions and they respond.
177.
by astrange 7mo ago
> I would probably score about the same, does this prove I also rely on training data memorization rather than genuine programming reasoning? It doesn't even prove the models do that. The RLVR environments being mostly Python isn&#x
178.
by astrange 7mo ago
BF involves a lot of repeated symbols, which is hard for tokenized models. Same problem as r's in strawberry.
179.
by astrange 7mo ago
Kind of like saying that scaling the language area in a human brain won't lead to a human brain. True, but just don't do that then.
180.
by astrange 7mo ago
AI has been consistently defined as "anything we can't make a computer do yet" since 1970. https://quoteinvestigator.com/2024/06/20/not-ai/

More ›