5 ms·
Would love to find out they're overfitting for pelican drawings.
by stkai 8mo ago
Would love to find out they're overfitting for pelican drawings.
- andy_ppp 8mo agoYes, Racoon on a unicycle? Magpie on a pedalo?
- deleted 8mo ago[deleted]
- throw310822 8mo agoCorrect horse battery staple: https://claude.ai/public/artifacts/14a23d7f-8a10-4cde-89fe-05f108da236d https://claude.ai/public/artifacts/14a23d7f-8a10-4cde-89fe-0...
- ta988 8mo agono staple?
- iwontberude 8mo agoit looks like a bodge wire
- Schlagbohrer 8mo agoThat is the nastiest, ugliest horse ever
- HappMacDonald 8mo agowait, how do you know my pw?
- _kb 8mo agoPlatypus on a penny farthing.
- fragmede 8mo agoThe estimation I did 4 months ago: > there are approximately 200k common nouns in English, and then we square that, we get 40 billion combinations. At one second per, that's ~1200 years, but then if we parallelize it on a supercomputer that can do 100,000 per second that would only take 3 days. Given that ChatGPT was trained on all of the Internet and every book written, I'm not sure that still seems infeasible. https://news.ycombinator.com/item?id=45455786 https://news.ycombinator.com/item?id=45455786
- eli 8mo agoHow would you generate a picture of Noun + Noun in the first place in order to train the LLM with what it would look like? What's happening during that 1 estimated second?
- Terretta 8mo agoThis is why everyone trains their LLM on another LLM. It's all about the pelicans.
- metalliqaz 8mo agoits pelicans all the way down
- fragmede 8mo agoUse any of the image generation models (eg Nanobanana, Midjourney, or ChatGPT) to generate a picture of a noun on a noun. Simonw's test is to have a Language (text) model generate a Scalar Vector Graphic, which the language model has to do by writing curves and colors, like draw a spline from point 150,100 to 200,300 of type cubic, using width 20, color orange. In that hypothetical second is freaking fascinating. It's a denoising algorithm, and then a bunch of linear algebra, and out pops a picture of a pelican on a bicycle. Stable diffusion does this quite handily. https://stablediffusionweb.com/image/6520628-pelican-bicycle-ride https://stablediffusionweb.com/image/6520628-pelican-bicycle...
- AnimalMuppet 8mo agoBut you need to also include the number of prepositions. "A pelican on a bicycle" is not at all the same as "a pelican inside a bicycle". There are estimated to be 100 or so prepositions in English. That gets you to 4 trillion combinations.
- theanonymousone 8mo agoEven if not intentionally, it is probably leaking into training sets.
- fdeage 8mo agoOpenAI claims not to: https://x.com/aidan_mclau/status/1986255202132042164 https://x.com/aidan_mclau/status/1986255202132042164
- mattacular 8mo agoThat settles it