5 ms·
GPT-3 is self-driving car hype all over again. ML can generate all kinds of signals across a large variety of domains [1], but it lacks intelligence for what to
by echelon 6y ago
GPT-3 is self-driving car hype all over again. ML can generate all kinds of signals across a large variety of domains [1], but it lacks intelligence for what to do with any of the signal it spews. Humans are still very much required at all steps.
If anything this is just a tool to aid human authors. A photoshop brush for the mind, if you will.
[1] I authored https://vo.codes https://vo.codes, so I'm at least a little knowledgeable.
- jamesjyu 6y agoAgreed that GPT-3 is a tool, and it isn't going to write entire stories for you, at least, not good ones. But it can be a valuable part of a writer's toolkit, similar to Google or other forms of research. The comparison to Photoshop is right: GPT-3 is in some ways like content aware fill. I believe in a few years, Word will have something like this integrated, albeit, probably set at a low temperature or for specific types of boilerplate writing. However, I don't think GPT-3 it's reached the level of hype as self-driving cars.
- 2bitencryption 6y agoI think you're bringing the "Will GPT-3 change life as we know it?" debate to a place where it's entirely unnecessary. An author made a cool sci-fi story with a cool AI model's help. That's exactly "photoshop brush for the mind", like you say.
- Swizec 6y agoWhat’s more, AI talking in character as itself sounds like just about the most perfect use of GPT3 imaginable. Don’t try to write like AI would, let AI do it.
- 0-_-0 6y agoThat sounds good in principle, but in reality GPT-3 is pretending to be a human pretending to be an AI, rather than write like "itself", since there's no self involved. Rather, it simply comes up with the most probable continuation.
- skybrian 6y agoYes, you want to be aware of what your function approximator is doing when it approximates something. Fortunately in the case of fiction, making stuff up is kind of the point. A human coauthor participates in turning randomness into meaning. Compare with interpreting vague song lyrics. The problems are in areas where it's important to tell the truth. In non-fiction, quoting people accurately is important, but GPT-3 will make up quotes and citations, as well as getting them correct sometimes too. (It might be interesting to see if there is training that would encourage it to always quote accurately.)
- jamesjyu 6y agoSourcing is definitely an issue. With GPT-3 there is no "source," as it is still essentially a bag of word vectors. I've also seen it take existing quotes and misattribute them to the wrong (but similar) person.
- visarga 6y agoNLP used to be based on bag of words. GPT-3 takes into consideration all interactions and the ordering, thus, not a simple bag of vectors. Quite the opposite, it's an implicit graph neural net.
- minimaxir 6y ago> If anything this is just a tool to aid human authors That's an extremely nontrivial use case.
- thorum 6y agoI think the true importance of GPT-3 is its general learning ability across multiple domains. David Chalmers wrote about this recently: > More remarkably, GPT-3 is showing hints of general intelligence. Previous AI systems have performed well in specialized domains such as game-playing, but cross-domain general intelligence has seemed far off. GPT-3 shows impressive abilities across many domains. It can learn to perform tasks on the fly from a few examples, when nothing was explicitly programmed in. It can play chess and Go, albeit not especially well. Significantly, it can write its own computer programs given a few informal instructions. It can even design machine learning models. Thankfully they are not as powerful as GPT-3 itself (the singularity is not here yet). source: http://dailynous.com/2020/07/30/philosophers-gpt-3/#chalmers http://dailynous.com/2020/07/30/philosophers-gpt-3/#chalmers
- echelon 6y agoHTML is a declarative grammar, so it entirely makes sense that GPT-3 can generate signal in this domain. It's just another shape. A sine wave signal. Show it a pattern of waves and it'll duplicate the ripples. Pretty neat stuff, but nothing to get existentially worried about. I'll fall off my chair when a model can write fizzbuzz. I think we have a ways to go.
- TeMPOraL 6y agoI hope you have your mattress ready by your chair, because I just put this prompt in AI dungeon: $ You are a fresh CS grad. You're preparing for a job interview. You ask on StackOverflow: "how to write a FizzBuzz program?" And, repeatedly pressing <Enter>, I got the following: > The first response is from an experienced coder who has been programming for 20 years and knows the answer. He writes: "FizzBuzz is a simple algorithm that can be written in one line of code." > He goes on to explain how you should implement it, but he doesn't bother to give any examples or explanations. It's all very basic and obvious. > Then comes another coder with 10 years experience. This guy has a lot more knowledge about programming languages than the first one did, so he gives you some pointers. Although his advice is useful, it's still fairly basic. > Then, someone with only 2 years experience answers. They give you some simplistic examples. At this point, I add: $ The first example looks like: And the game immediately wrote the following: int fizzBuzz ( int number ) { if ( number % 3 == 0 && number % 5 == 0 ) printf ( "FizzBuzz" ); if ( number % 3 == 0 ) printf ( "Fizz" ); if ( number % 5 == 0 ) printf ( "Buzz" ); if ( number % 3 ! = 0 && number % 5 ! = 0 ) printf ( "%d" , number ); printf ( " " ); return 0 ; } (Yeah, I know it hasn't really learned to code, but it apparently does have it encoded in its model.)