gwern

65 ms·

691.
by gwern 3y ago
The article says Walmart said it didn't know when they told Walmart.
692.
by gwern 3y ago
The original Chinchilla authors have now identified the original bug, apparently: https://twitter.com/borgeaud_s/status/1780988694163321250
693.
by gwern 3y ago
I didn't even bother checking Libgen, because if it wasn't in Libgen, it'd be in some other dataset.
694.
by gwern 3y ago
> FRCOphth Part 2 questions were sourced from a textbook for doctors preparing to take the examination [17]. This textbook is not freely available on the internet, making the possibility of its content being included in LLMs’ training da
695.
by gwern 3y ago
Are you trying to imply that Wikipedia articles by definition have no agenda and no one quoting a Wikipedia article can have an agenda either?
696.
by gwern 3y ago
Most RL agents are 'jumpy' or jitter a lot, because it makes no difference to the reward, and where it does make a difference, that tends to only matter close to convergence where slight speedups are all that's left. If you w
697.
by gwern 3y ago
> One of the tricks OpenAI uses when fine-tuning models is to use the unadjusted foundation model as a coherency model alongside the data or reward model to be fine-tuned on. Loss is calculated as the sum[0] of both the fine-tuning and
698.
by gwern 3y ago
https://en.wikipedia.org/wiki/Garden-path_sentence
699.
by gwern 3y ago
Look at the XSLT stylesheet this XML uses - it is doing parsing and indenting of Python in XSLT! I know XSLT is Turing-complete, but that is still highly unusual these days. Who knew that would even work in contemporary browsers...
700.
by gwern 3y ago
I am looking at the source of this '.xml' URL and... that's not something you see every day.
701.
by gwern 3y ago
I was expecting crime labs or maybe biology like genetics. But music copyright clearance problems (copyright is why we can't have nice things) is good too.
702.
by gwern 3y ago
> However, Wilson died in 1935 and Watson seems to have lost interest in the project in the late 1930s.[3] After Watson's death in 1965, J. M. Whittaker examined Watson's papers (which were in disarray, due to be incinerated in
703.
by gwern 3y ago
I was thinking to myself, "he's trying to invent the _Akira_ bike!" and then I get to the end: https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg...
704.
by gwern 3y ago
Now that's a blast from the past - T5! I always thought it was underused, but it's also from so long ago now, and even Google has moved past it to UL2 etc AFAIK. What's the use-case here for reproducing it? Aren't there
705.
by gwern 3y ago
That's the ChatGPT style. It is extremely stereotypical when it comes to writing lyrics or poetry: 4 lines, rhyming, with metronome regularity. Both Suno & Udio seem to use ChatGPT for lyrics, and it's a terrible choice, whi
706.
by gwern 3y ago
Yes.
707.
by gwern 3y ago
Suno doesn't do that either. I think it may be an anti-spam/leech mechanism - outsource sybil prevention to Google/Discord.
708.
by gwern 3y ago
Yeah, that's the problem. Buying it was so bad an idea and so overpriced that he had to load the blue bird down with debt. He can't just quit Twitter and toodle off and say 'well, I stopped wokeism, and that justifies destroy
709.
by gwern 3y ago
Aside from being a long-standing obsession of Musk's, the thing about the 'everything app' is that it's the Hail Mary move which could make the Twitter thing anything but a dumpster fire immolating $20b+, Musk's rep
710.
by gwern 3y ago
Paid by the standards of Kenya, perhaps. Not America. And how many of those people were paid anything like a living wage across all of the years of training? And how many orders of magnitude smaller is that number than '8 billion'
711.
by gwern 3y ago
"If they are willing to pay you in particular, how many other people like you are they also willing to pay for, and is that number much greater than or much less than the current number paid to do it?" There are not many Usain Bol
712.
by gwern 3y ago
> It's foreshadowing AI helping experts become better. The humans are still way worse than the Go programs. People are still willing to pay them to play a game as entertainment. Are lots of people willing to pay you to do whatever i
713.
by gwern 3y ago
I think if you go check the n-gram (Markov chain) models which were trained by IBM and Google and others (eg. 'unreasonable effectiveness of data'), you'll find them remarkably chonky sample-wise. (This was possible because
714.
by gwern 3y ago
Specialist AI doesn't scale. When you train separate models, each one starts from scratch, so you get no transfer / informative priors. So your specialist models wind up collectively scaling much worse than the power law of the ge
715.
by gwern 3y ago
There are many reasons it might not be profitable even if the ML part were working well.
716.
by gwern 3y ago
This is a bad submission because Boing Boing heavily editorializes, and is at least 2 levels removed from the actual reporting at this point - the editorialized Boing Boing excerpts, the editorialized Gizmodo article, The Information articl
717.
by gwern 3y ago
I got an alert on this one too, but it was an odd one. Usually, the Google DeepMind patents are pretty obviously based on specific papers, and often quite old ones, but... seq2seq with tree search, filed 2022? Didn't ring any bells on
718.
by gwern 3y ago
If they were "commonly known", then you exploiting them seems even more, not less, likely to have removed them from stockpiles, no? Because the North Koreans were clearly not going to fix them on their own anytime soon (because th
719.
by gwern 3y ago
OEIS is fun but the bare homepage probably isn't too helpful. I highly recommend Sloane's memoir about the history of OEIS: http://neilsloane.com/doc/HIS50.pdf "_A Handbook of Integer Sequences_ 50 Years
720.
by gwern 3y ago
No details but it's related to Pascal triangles? So it involves pricing financial instruments, seems like a safe guess...

More ›