gwern

37 ms·

601.
by gwern 2y ago
I don't think any of the numbers you quote are really that meaningful. Like you are quoting IDs - what is the cost of an ID ? If all the IDs were stripped, would that even make a measurable difference? Really now. Similarly for those
602.
by gwern 2y ago
Some background: https://www.lesswrong.com/posts/cB2Rtnp7DBTpDy3ii/memory-ban...
603.
by gwern 2y ago
Well, that would've been a lot more interesting if it had so much as hyperlinked the references...
604.
by gwern 2y ago
Yeah, it used to be fast, but then you add features and have pages with a ridiculous amount of stuff in them compared to most pages (every DOM element costs you), and, well... We plan to go back and optimize stuff once one last feature is i
605.
by gwern 2y ago
That would also be Gwern.net. :) https://github.com/gwern/gwern.net/blob/master/build/Inflati...
606.
by gwern 2y ago
> If you want to show progress while the reader is somewhere in the middle of a page, you could use a page progress bar. As best as I can tell, the standard name for these is 'scroll spy'. Gwern.nt mobile example: https:/
607.
by gwern 2y ago
If you have NoScript, note that the article will be invisibly cut off. It should continue with "AI-enabled tools and weapons are not just being deployed in exercises. They are also in use on a growing scale in places like Gaza and Ukra
608.
by gwern 2y ago
A regrettable name. Everyone who sees it will assume that it's something like 'a startup self-help book'.
609.
by gwern 2y ago
Think of it as a "you could have invented binpacking algorithm X too" post.
610.
by gwern 2y ago
Was briefly concerned until I saw it wasn't Ivan Sutherland: https://en.wikipedia.org/wiki/Ivan_Sutherland
611.
by gwern 2y ago
What an outrageous (and yet, predictable) response by the first systemd developer trying to WONTFIX it (not even to the extent of adding a note in the man page): https://github.com/systemd/systemd/issues/33349
612.
by gwern 2y ago
Well, that's true inasmuch as every other prediction did far worse. Saying ARC did the best is passing a low bar when your competition is people like Gary Marcus or HNers saying 'yeah but no scaled-up GPT-3 could ever write a who
613.
by gwern 2y ago
How does a mistake like that get made 3 or 4 years later when he was never the chair, and the only official source for that had been edited within days to remove that, and apparently there has never been a YC chair at all?
614.
by gwern 2y ago
From Chollet's perspective, the LLM hype started well before, with at least GPT-2 half a year before his paper, and he spent plenty of time mocking GPT-2 on Twitter before he came up with ARC as a rebuttal.
615.
by gwern 2y ago
Figure 2 ( https://arxiv.org/pdf/2406.05587#page=10 ) is not at the logit level, it's at the whole completion level (entire names classified by nationality). So you don't know how any sampling would affect that
616.
by gwern 2y ago
Cite? I don't see how either of those could deal with the fact that the logits become uninformative and 'flattened' after the tuning. How can a sampler undo the erasure of information?
617.
by gwern 2y ago
Not after RLHF tuning, due to the 'flattened logits' phenomenon (which is the logit-level version of the mode collapse OP documents at higher levels). All the temperature settings wind up yielding pretty much the same output, unti
618.
by gwern 2y ago
Also https://chipmonk.substack.com/p/spaced-repetition-for-teachi...
619.
by gwern 2y ago
Yeah, I think that one suffered from not really having a clear vision of how AI would work, so it lacked the backbone that most Chiang stories start with and couldn't figure out what to do with itself and so went on too long. I'd
620.
by gwern 2y ago
I'm not sure your math there on the birthday paradox is right (you only check the time during a single 10 minute range every day, no matter what? and you wouldn't look at it out of curiosity or for the pleasure of seeing a new quo
621.
by gwern 2y ago
13,000 seems surprisingly small, all things considered. Seems like the sort of task where you could automate it with a small LLM instance and process millions of books easily. Then extract all of the key strings and have a set of tens of th
622.
by gwern 2y ago
You are, unfortunately, not the first to be misled by my impersonator. I've done what I can to get that account blocked: I have reported him for impersonation to Twitter years ago, after his previous accounts, and I have asked my fol
623.
by gwern 2y ago
This was extensively discussed and criticized at the time, so presenting it at face-value, ignoring all of the criticism, does not make a very good submission.
624.
by gwern 2y ago
I have no idea what you are talking about, given that you are linking to a comment a decade old which has nothing to do with Conway's death just now; nor have I commented on Conway in any way today or been on HN today until just now, w
625.
by gwern 2y ago
Yes. See the related work section in the paper: there is a long history of models, recently like MAE and MaskGit, which predict pixels in basically arbitrary orders, and that is useful because it lets you train on subsets of each image, ups
626.
by gwern 2y ago
> Every phenomenon found massively in training set will eventually pop up in LLMs. That's a nontrivial and controversial claim that many on HN would hotly disagree with - that LLMs are guaranteed to do everything humans do, if you j
627.
by gwern 2y ago
Oh hm, so they are. I thought they were binary because they used a digital pen to create them, IIRC, and logistic regression is always the baseline; but checking, they technically are grayscale and people don't always binarize them. So
628.
by gwern 2y ago
How can MNIST be solved using just two binary pixels when there's 10 classes, 0-9?
629.
by gwern 2y ago
Not just being partner, but what sounds like some extraordinary generosity by Paul Graham, new in the WSJ article cited by OP: > Hydrazine bought out a portion of startup shares owned by Graham, a transaction that gave Altman stakes in s
630.
by gwern 2y ago
Keep in mind, when Swisher says she likes Sam, what she means is, to quote her Twitter: "Sam Altman is no different than most of the talented ones, which is to say, aggressive, sometimes imperious and yes, self-serving."

More ›