gwern

47 ms·

661.
by gwern 2y ago
Note that the YT recommender has changed a lot since then.
662.
by gwern 2y ago
> in a manner that caps costs to IA at a survivable level How do you know that? The last I saw, the penalty was still undisclosed.
663.
by gwern 2y ago
https://www.thenewatlantis.com/publications/shop-class-as-so...
664.
by gwern 2y ago
No, not really. OA has been reticent to publish any real details about what RLHF GPT-4 and later models go through; while some models have been much more open, those weren't used in OP. And it's unclear how easily you can interrog
665.
by gwern 2y ago
No, because you're confusing loss functions: a LLM makes a probabilistic prediction, not a hard decision. That is the optimal strategy only if you have something like a 0-1 loss function†, akin to betting on a coin flip, which is not a
666.
by gwern 2y ago
They do, via the temperature sampling.
667.
by gwern 2y ago
> A consequence of being an auto regressive model is not being able to plan token output. Generating independent simple random variables requires zero planning by definition, because they are independent. And base auto-regressive models
668.
by gwern 2y ago
> If you trained on specific generated data with real distributions It was trained on generated data from real distributions! The datasets LLMs are trained on include gigabytes of real data from real distributions, in addition to all o
669.
by gwern 2y ago
> Humans would say "Left" 100% of the time in a zero-shot scenario as well. They do not! And you should not just make up assertions like these. You don't know what humans would say. In fact, in polls, they wind up remarkab
670.
by gwern 2y ago
> Indeed this is unsurprising given how LLMs work. I mean if you ask a human to generate a random number, and then reset the universe and all state of the human and ask again, you will get the same number. It actually is surprising, an
671.
by gwern 2y ago
Also a disincentive to not buy land to build on, because now you have to pay for the demolition of the dangerous pile of rubbish on it pretending to be a house to reduce the tax burden.
672.
by gwern 2y ago
Main discussion at https://news.ycombinator.com/item?id=40199715 turns out.
673.
by gwern 2y ago
More informative link: https://rentry.org/GPT2 I played with it today with my _Paradise Lost_ test, and I got a strong vibe of it being GPT-4 but without the distinct RLHF and much more like Claude's RLAIF. Even if it&
674.
by gwern 2y ago
Made a PDF with the missing page: https://gwern.net/doc/cs/security/1949-bradley.pdf
675.
by gwern 2y ago
I'm surprised you noticed! I think you may be the first I've seen. Most people just think it's a 1 vs 2 column layout; but we didn't find that quite satisfactory... If you guys like cat optical illusions, they're su
676.
by gwern 2y ago
Direct link: https://archive.computerhistory.org/resources/access/text/20...
677.
by gwern 3y ago
He mentions Common Lisp as something he did a game in, and you can see the link he praises https://www.youtube.com/watch?v=72y2EC5fkcE as basically a very well-developed Lisp image approach with timetravel debugging etc aim
678.
by gwern 3y ago
You can support mobile with ASCII art if you render at different widths and use a HTML+CSS wrapper for media-queries! I have a whole proposal about this: https://gwern.net/utext
679.
by gwern 3y ago
Hm... I don't get it. This seems like a fairly random link to some very old college comic archives. Is there context I am missing here like a death or something?
680.
by gwern 3y ago
Probably a good thing too, because whatever you learn that rhymes, there will be a criticism that also rhymes. Seems like the cognitive bias matters mostly when some things use rhyme, but others don't.
681.
by gwern 3y ago
I don't think that makes any sense. Many of the objects make little sound (how loud is a q-tip hitting the ground, exactly?), you wouldn't watch the falling object if you were expecting to flush out prey elsewhere - nor, for t
682.
by gwern 3y ago
Vapnik: https://www.cs.princeton.edu/courses/archive/spring13/cos511... https://engineering.columbia.edu/files/engineering/vapnik.pd... https://www.learningtheory.org&#x2
683.
by gwern 3y ago
If you want some much more old-school Tao Teh Ching samples from GPT-2: https://gwern.net/gpt-2#tao-te-ching
684.
by gwern 3y ago
I thought the answer was 'no'? After all of the discovery and testimony, despite the supposed smoking gun of entire copied chips, the endless vilification of Uber and the FUD and rumors, they came up with next to nothing. Lewandow
685.
by gwern 3y ago
Yes, that sort of mirror-sampling would reduce variance. The problem is, though, you need to know all the uses of randomness in order to properly counterbalance them, and these systems are already enough of a pain to use. (For example, if y
686.
by gwern 3y ago
Or they might be doing it just to keep their fingers 'warm'. I recall during AlphaStar, there were a lot of questions about why the human pro APMs were so high when it seemed like a lot of them were null ops, and apparently the
687.
by gwern 3y ago
Interesting. I had no idea that the universe would eventually be basically just photons. I wonder if that is why Stephen Baxter had his 'photino birds' as the final form of life in his Xeeleverse?
688.
by gwern 3y ago
> Actually, you SHOULD be paranoid about important data. I learned this the really hard way - a long time ago - when I had just finished writing a 200+ A4 pages book and then lost everything because of a silly mistake and had to start al
689.
by gwern 3y ago
The potential for systematic bias here is also quite concerning. Imagine your experimental group tends to come in the morning (maybe because they're on-site) but your control group is scheduled to come in the rest of the day...
690.
by gwern 3y ago
I wonder how much appetite suppressants like GLP-1s could help here? They seem to hit appetite at a high level and curiously reduce other cravings or addictions, so they might be able to deal with the hunger here.

More ›