4 ms·
Did no one read beyond the first paragraph? This is an article about the effectiveness of large language models, which have learned to do tasks beyond regurgit
by VyseofArcadia 4y ago
Did no one read beyond the first paragraph?
This is an article about the effectiveness of large language models, which have learned to do tasks beyond regurgitating text despite only having been trained to regurgitate text.
As long as you are able to describe the task as a text stream and are ok with having a text stream as output, GPT-3 might actually be able to do the task with just a couple of examples as context. I saw a talk at Black Hat about using GPT-3 as a spam filter. Because GPT-3 has trained on so much spam, it knows what spam looks like. It just needs a couple of examples, and off it goes.
> The most complicated reasoning programs in the world can be defined as a textual I/O stream to a leviathan living on some technology company’s servers.
The references to the UNIX philosophy feel a bit tacked on, honestly. Probably just there to grab attention.
- zwieback 4y agoThe answer is "no, nobody read the article".
- stinkytaco 4y agoIt briefly discusses DALL-E and imagine models near the end, which seems to paper over a big part research in AI right now, but that also wasn't the point of the article, which is fair. What I find interesting is that this reminds me how AI is being trained on a very narrow set of human inputs. Images, text, possibly some video, but there's a huge amount of data governing human behavior that is none of those things. Facial expressions, body language, tone, diet, environment, etc. etc. etc. I'm sure it's coming, but it feels a long way off.
- verisimilitudes 4y ago
- comfypotato 4y agoI lost interest in the article itself, but I'm currently down a rabbit hole of one of its links [1]. It's the webpage of some random (random to me that is) person, but it has quite a bit of content (and links to other content). I consider this find a win! [1] http://www.catb.org/~esr/ http://www.catb.org/~esr/
- foooobaba 4y agoWhen I was 12 I followed ESR’s guide on “how to become a hacker”. I wanted to learn hacking but didn’t know where to start or have anyone to ask. Started with html then c++ and using linux. For me, it’s probably one of the most important things I read, because it ultimately altered the trajectory of my whole life. http://www.catb.org/~esr/faqs/hacker-howto.html http://www.catb.org/~esr/faqs/hacker-howto.html
- 9wzYQbTYsAIc 4y agoSimilar story for me, was encouraged on IRC by ESR (was randomly inquiring about a GPS software project that he turned out to be supporting - go figure). His advice was that given ten years almost anybody can become a professional software developer.
- odnes_lex 4y agoLike the OP, I owe much in my life trajectory to esr's How to Become a Hacker. He's one of the people I've never met/barely interacted with who are dear to me. He must be over 70 now, and it's been a few years he's neglected his- in past times- lively blog. He used to be in financial straits, tried to launch a cultural initiative to draw support for Load Bearing Internet People like himself: http://www.catb.org/esr/loadsharers/ http://www.catb.org/esr/loadsharers/ He's still got a subscriberstar if you're looking to lighten your pocket.
- 0x445442 4y agoESR is random to you? Oh youngling, much to learn you have.
- VyseofArcadia 4y agoEric S. Raymond is a fairly prolific open source contributor, but he's probably most famous as an author and essayist, specifically of _The Cathedral and the Bazaar_, which arguably did more to popularize the notion of open source than Stallman ever did for free software [0]. I particularly like his series of posts about his modern rewrite of Crowther and Woods' Adventure, starting here: http://esr.ibiblio.org/?p=7536 http://esr.ibiblio.org/?p=7536 [0] It helps that ESR's arguments where largely practical in nature rather than ideological. In particular, he coined the aphorism, "with enough eyes, all bugs are shallow".
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- P5fRxh5kUvp2th 4y ago> Because GPT-3 has trained on so much spam, it knows what spam looks like. It just needs a couple of examples, and off it goes. it's seen so much spam it only needs a few examples of spam to work?
- johnmorrison 4y agoNo, he's not contradicting himself. He's saying: it has seen so much spam (without it necessarily having been labeled as such) already, that it only needs to see a couple examples labeled as spam in order to recognize the pattern in all of the spam it has seen before.
- P5fRxh5kUvp2th 4y agooh, gotcha.
- mellavora 4y agowhat is this "article" of which you speak?
- wodenokoto 4y agoCan I use it to pivot my csv files?