gwern

45 ms·

631.
by gwern 2y ago
It might be worrying but they also point out that the quality seems to go up. Perhaps people think that random web scrapes are way better than they really are, and so expect ChatGPT output to worsen corpuses on average rather than improve t
632.
by gwern 2y ago
There's also serious coordination problems internally. How easy is it to get access? Did you make buddies with the little emperors in charge of the historical archives? Is the archive in your company large enough? Are the chances of ge
633.
by gwern 2y ago
Wow - this explains a lot about why Chinese LLMs and AI struggle so much to get data, despite the supposed near-infinite databases: a lot of it is just gone , far more than anyone had ever proposed might be the case, because of neglect, ce
634.
by gwern 2y ago
> Maybe I missed it, but it didn’t seem to me like the authors presented any reasoning about what makes this capability particular to transformers… especially on the relatively small HMMs, it’s not clear to me that we wouldn’t see the sa
635.
by gwern 2y ago
You're looking at it the wrong way: Altman really wants this deal. For a long time, accelerated by the OA coup, he has been searching for ways out of the MS trap, where OA becomes an appendix to MS, dependent on MS infrastructure compl
636.
by gwern 2y ago
> let me put it to you this way: what is the LLM doing differently when it generates tokens that are "wrong" compared to when the tokens are "right"? It is conditioning on latents about truth, falsity, reliability, an
637.
by gwern 2y ago
pg was already on the record about Altman being fired back in December: https://www.wsj.com/tech/ai/sam-altman-openai-protected-by-s... > Graham said it was his wife’s doing. “If anyone ‘fired’ Sam, it was Jess
638.
by gwern 2y ago
I remember working a part-time job as a dishwasher where my manager demanded I stay until 1-2AM to get it right according to her and indefinitely 'fully focus' on my dishwashing duties instead of my other obligations, like to &#x2
639.
by gwern 2y ago
You can do this with dynamic evaluation ( https://gwern.net/doc/ai/nn/dynamic-evaluation/index ): literally just train on the data as you are fed new data to predict, in the most obvious possible way. It
640.
by gwern 2y ago
Which should have been a bluff (board members typically have "director's insurance" to cover exactly this scenario - trying to control a company by threatening to pierce the corporate veil and sue board members individually
641.
by gwern 2y ago
No, it would control for a lot of those factors. Things like impulsivity, sexuality, sensation-seeking etc. All very heritable or family-level. If you ever find any correlations reported for tattoos between family members, it's gonna
642.
by gwern 2y ago
http://karpathy.github.io/2019/04/25/recipe/ > leave it training. I’ve often seen people tempted to stop the model training when the validation loss seems to be leveling off. In my experience networks
643.
by gwern 2y ago
I'm not sure they've written that anywhere else (which is a cautionary lesson for anyone trying to understand how research happens, BTW), but as further evidence besides just 'Ethan told me so on EAI Discord [IIRC]', y
644.
by gwern 2y ago
Yes. This has already been demonstrated by "Teaching Arithmetic to Small Transformers" https://arxiv.org/abs/2307.03381 , I'm not sure what OP adds except demonstrating that you can do that via the embed
645.
by gwern 2y ago
This is a Swedish study, so what might be possible is using the population registry to contact siblings of the cancer patients to ask about traits like tattooing and then their health data would already be in Swedish system and linkable. Th
646.
by gwern 2y ago
I feel like OP unfortunately mostly misses the point of Gosset's work and the t-test, in the usual way of people taught the contemporary bastardization of Gosset/Fisher/Pearson/Neyman as NHST. The important thing isn&#x2
647.
by gwern 2y ago
> For a case that doesn't involve AI at all, consider situations where a voice actor in a cartoon is replaced (sometimes while still alive) by someone who can perform a voice that sounds the same. Decisively not illegal. Because the
648.
by gwern 2y ago
No one said it was identical (and OA is rebutting a strawman when they expostulate on how they trained on a real human voice actress, which is not what Johansson's statement accused them of to begin with). The point is here is someone
649.
by gwern 2y ago
> I tried reverse image searching a few of the prints, only to find they were one-of-a-kind. I started googling “morris artwork” and found images that, while beautiful, seemed to lack the finesse and colour range of my new prints. Yeah,
650.
by gwern 2y ago
> and somehow trained another voice within the few weeks after she said no. Er, that is totally possible? You act like it's not a machine learning system. You train new stuff in hours or days easily, especially if you have good tool
651.
by gwern 2y ago
> Exactly, everyone claiming so hasn't actually listened to it Completely false. Even the journalists at the launch of "Sky" last year were singling it out of the batch of voices, and specifically asking OA about how it so
652.
by gwern 2y ago
> it's still hard to believe they didn't intend it to sound like the voice in her though: Especially when you have ex-OAers, who had been working there at the time on 'J.A.R.V.I.S.', tweeting things like https:/
653.
by gwern 2y ago
Note that when Bill Gates praises GPT-4o, he is at least partially "talking his book": he retains large amounts of equity in Microsoft, and has been intimately involved in MS's big bet on OA - in 2019, when Kevin Scott was lo
654.
by gwern 2y ago
That's interesting. So this would have been 2017, 2018? How long did the NDA + paired gag order about the NDA, and the non-disparagement order last? The OA one is apparently a lifetime one, including the 'non-disparagement' o
655.
by gwern 2y ago
GPT-4o still uses BPEs.
656.
by gwern 2y ago
I'm surprised to hear that. I think it looks like a straightforward inversion of the light-mode, which you aren't complaining about, and generally, people seem to like it & I see lots of screenshots of it on social media. Are
657.
by gwern 2y ago
Word Golf: https://www.word.golf/
658.
by gwern 2y ago
These aren't real stock, they are "profit participation units" or PPUs; in addition, the fact that there is a NDA and a NDA about the NDA, means no one can warn you before you sign your employment papers about the implication
659.
by gwern 2y ago
And so the goalposts move again, from "of course AIs cannot learn deception, it is wild sensationalist reporting to suggest it is even possible given how many decades we are from AGI" to "of course they would not learn that e
660.
by gwern 2y ago
Lots of reasons, but big ones were they were stung by the political extremism claims so now it tries to steer you away from anything extreme.

More ›