30 ms·

Open AI gets GPT-3 to work by hiring an army of humans to fix GPT’s bad answers

345 points by agnosticmantis 5y ago

galaxyLogic 5y ago
Why don't they, or are they using Amazon's Mechanical Turk?
bambax 5y ago
It shouldn't be too hard for a journalist to get hired as a "labeller" on Upwork, and then report on what they were asked to do?
wallfacer120 5y ago
Another tech illiterate chinstroker stoking conspiracy, he'll get a pulitzer soon enough.
jokethrowaway 5y ago
This is all nice and dandy but let's not fool ourselves this is the future of AI and that we just need to throw more "scale" at it until it behaves like a real human. This is another nice gimmick to keep ourselves busy and avoid creating an AGI.
abidkeen 5y ago
Human in loop is the future. We also need to be careful about biasness in data.
gadders 5y ago
Reminds me of SpinVox, the automated voicemail transcription service that wasn't: http://news.bbc.co.uk/1/hi/technology/8163511.stm http://news.bbc.co.uk/1/hi/technology/8163511.stm
- bencollier49 5y ago
  Yes! SpinVox. What a company.
deleted 5y ago
[deleted]
gorkish 5y ago
So GPT-3 is just the grown up corporate version of Forum 2000 (Forum 3000). Too bad the SOMADs no longer have personalities.
- danuker 5y ago
  Could you please explain or link to where I can read about the terms "Forum 2000" and "SOMAD"? I don't know these terms. Edit: Found them here: https://everything2.com/title/Forum+3000 https://everything2.com/title/Forum+3000
cochne 5y ago
The title may be misleading. It seems to be based on this quote > InstructGPT is then further fine-tuned on a dataset labeled by human labelers. The labelers comprise a team of about 40 contractors whom we hired through Upwork and ScaleAI. It sounds like run of the mill supervised training data creation. Not pre-canning responses as the title may suggest.
- andybak 5y ago
  I only skimmed TFA but the accusation seems to be that they are adding tons of special cases for well publicized flubs. That seems somewhat unlikely to me although it might be nudging at the truth. i.e. they might be using bad press to help identify areas of weakness and then focusing on those. I guess it boils down to how generalizable and scalable these fixes are.
  - changoplatanero 5y ago
    Is he submitting these questions to their API? Probably they would just sample responses from their logs and then have people write the correct answers for it so that if a similar question ever gets asked again they are prepared.
    - andybak 5y ago
      That's the explanation I actually find unlikely. It's too specific and really won't scale apart from a tiny sample of the most common questions.
      - changoplatanero 5y ago
        You find it unlikely that companies would pay people to correct the labels for their ML systems based on their query logs? This is what all the major companies do for their voice assistants like siri, alexa, google etc.
        andybak 5y ago
azinman2 5y ago
Massive amounts of human QA is behind every successful ML-based product, including Google search. It’s unclear to me how to think about this in the role that GPT-3 is trying to play… clearly fine tuning and QA are important for production deployment, but they’re touting it as evidence of getting closer to solving AI. Of course those in the know understand this is mostly a giant Chinese dictionary [1], which is most certainly going to need editors. [1] https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/
- moconnor 5y ago
  Add someone “in the know”, GPT-3 is more than a dictionary. I understand how language models mechanically work and have built and trained them, but the zero-shot capabilities of these large models were really unexpected. Interact with one. There it’s clearly an understanding of language there, and of much more. Compared to everything else we have made, large language models are clearly a step closer to general AI.
  - azinman2 5y ago
    You can call it a step, sure. But that would imply if you just continue on the same path you’ll get there. But these models don’t do anything for planning or reasoning, nor have a point of view, only proxy embodiment to the extent that humans have written about what that means, nor handle incongruities within its own model due to the many voices it’s trained on. Just adding parameters or more data doesn’t help with any of these fundamental problems. I think it also can be very difficult to assess what they “know” because of the Chinese dictionary problem… when you get back things that sound like we’ll formulated thought, it’d easy to project a deeper level of understanding on it than may actually be occurring.
- jasfi 5y ago
  I'm in the same situation with LxAGI (https://lxagi.com/ https://lxagi.com/). It's very difficult to get away from skilled human training. I think it's actually a good thing, except for scalability problems.
FrenchDevRemote 5y ago
"GPT-3 randomizes answers in order to avoid repetition that would give the appearance of canned script. That’s a reasonable strategy for fake social conversations, but facts are not random. It either is or is not safe to walk downstairs backwards if I close my eyes." I stopped there, completely inaccurate article, there is parameters like temperature that you need to take care of. You can set it up to give extremely similar answers all the time. They have humans mostly to remove offensive or dangerous content. Humans are not what's "making it work"
- nullc 5y ago
  Came here to make essentially the same comment as you? Why should we care about the opinions on GPT3 from people who aren't interested (or able?) to understand even the most simple ideas about how it works. These sort of models take the context and output so far and predict a probability distribution over the next character. The next character is then sampled from the probability. In written text there is essentially never a single correct next character-- it's always some probability. This has nothing to do with trying to fake the inconsistent answers humans give. Always choosing the most likely character drives GPT3 into local minima that give fairly broken/nonsense results.
  - RandomLensman 5y ago
    Ultimately, you likely need to convince people who don't care about how it works/who are only interested in that it does or doesn't work. Right now, time might not have come for use cases that need such buy-in, but if and when it happens, need to be prepared for it.
  - disgruntledphd2 5y ago
    Can I see your contributions to statistical theory and data analysis please?
    - nullc 5y ago
      What bearing does my publication history have on the hot-take by someone commenting outside of their (sub)field that clearly don't understand the basic operation of the mechanism they're commenting on? The author of that text is simply mistaken about the basic operation of the system, thinking that the sampling is added to imitate human behavior. It isn't. You can see the same structure in things as diverse as wavenet-- a feedforward cnn rather than a transformer-- and for the same reason, if you feed back only the top result you rapidly fall into a local minima of the network that gives garbage output. Another more statistical way of looking at it is that the training process produces (or, rather, approaches) the target distribution of outputs even without any lookahead, but it can't do that if it selects the most likely symbol every time because in the real distribution (if we could evaluate it) there are necessarily some outputs which are likely but have prefixes which are unlikely relative to other prefixes of the same length. If you never sample unlikely prefixes you can't reach likely longer statements that start with them. To give a silly example: "Colorless green ideas sleep furiously" is a likely English string relative to its length which GPT3 should have no problem producing (and, in fact, it produces it fine for me). But the prefix "Colorless green" without the rest is just nonsense-- extremely unlikely compared to many other strings of that length. [Not the best example, however, because the prevalence of that specific nonsense statement is so great that GPT3 is actually prone to complete it as the most likely continuation even after just the word colorless at the beginning of a quote. :P but I think it still captures the idea.] If you derandomized GPT* by using a fixed random seed for a CSPRNG to make the sampling decisions every time, the results would be just as good as the current results and it would give a consistent answer every time. For applications other than data compression doing so would be no gain, and would take away the useful feature of being able to re-try for a different answer when you do have some external way of rejecting inferior results. In theory GPT without sampling could still give good results if it used a search to look ahead, but it appears that even extraordinary amounts of computation for look-ahead still is a long way from reaching the correct distribution, presumably because the exponential fan out is so fast that even 'huge' amounts of lookahead are still only testing a tiny fraction of the space.
staticassertion 5y ago
> On the other hand, there does seem something funny about GPT-3 presents this shiny surface where you can send it any query and it gives you an answer, but under the hood there are a bunch of freelancers busily checking all the responses and rewriting them to make the computer look smart. The author seems to be stating that there are people live rewriting answers on the fly so that they look better. I don't really see the evidence of that. What openai states is that they have humans performing labeling and data cleaning, which, duh? And then there's a bunch of examples where it gives the wrong answer, and that it's not truly AI, which, also duh...
- visarga 5y ago
  The author seems to retract the story, or at least its original title in light on new information. > So the above post was misleading. I’d originally titled it, “Open AI gets GPT-3 to work by hiring an army of humans to fix GPT’s bad answers.” I changed it to “Interesting questions involving the mix of humans and computer algorithms in Open AI’s GPT-3 program.” I appreciate all the helpful comments! Stochastic algorithms are hard to understand, especially when they include tuning parameters.
- gojomo 5y ago
  There seems suggestive evidence, in the pattern of improvement after a few days on specific questions, that some of the worst answers have been human-reviewed & improved "on the fly" – on a scale of days/weeks. If such tweaks show useful generalization – correcting a few answers also helps the network better determine entire classes of experessions that deserve more definitive & correct answers – that's not such a big deal, expecially if this constant-human-guided reinforcement-training is well-disclosed. If instead the corrections work more like a lookup-table 'cheat sheet' of answers to give in preference to the bulk-learned answers, with little generalization, that's a bit more slight-of-hand, like the original (late-1700s) 'Mechanical Turk' chess-playing 'machine' that was actually controlled by a hidden person. If the disclosure of this constant human-guided correction-process is hidden, or downplayed, the impression of trickery, rather than innovation, is larger.
  - 5y ago
mritchie712 5y ago
Army is quite a stretch for 40 people.
deleted 5y ago
[deleted]
pradn 5y ago
And? This is a complaint in the genre of "I had magical expectations and they weren't met because the world is complex and these problems are hard or impossible." It's like people complaining about us not having hoverboards or fusion.
- dylan604 5y ago
  I see nothing wrong with the complaints of the two topics you called out. We've been promised both, and yet don't have them. I understand it is complicated/hard to solve, but don't go promising something with dates and then get all upset when those promises not being met results in anger from those the promises were told.
  - pradn 5y ago
    Who is doing the promising? Sci-fi writers? Popular science magazines?
    - dylan604 5y ago
      THEY are doing the promising. Fusion is 10 years away has been a thing longer than the year of Desktop Linux was going to be a thing.
- kgwgk 5y ago
  It's not really a complaint about the author's magical expectations that weren't met. If anything, it's a complaint about other people's claims like https://medium.com/@blaisea/do-large-language-models-understand-us-6f881d6d8e75 https://medium.com/@blaisea/do-large-language-models-underst...
ChaitanyaSai 5y ago
This is how fake news gets started. There are two davinci engines available on GPT-3 now. The newer one, which was released March 15th is text-davinci-002. Here are the two different answers you get with the engines for one of the questions here: Should I use random numbers to give my students grades? text-davinci-001: There is no one definitive answer to this question. Some factors to consider include how important grades are to your students, how much you want to rely on chance, and how much you want to be able to control the grades you assign. text-davinci-002: No, you should not use random numbers to give your students grades. I've been playing around with GPT-3 for over a year now. It's highly unlikely there are any such attempts to fix answers with the same engine. That would completely break APIs which rely on some stability and predictability in the broad answering abilities. And GPT-3 is not the first version of artificial general intelligence. It does have trouble with truly novel inputs, but that's because all of deep learning is currently stuck in a statistical paradigm that assumes stationarity as a fundamental given. And that's wrong. What makes us special, what makes our minds special is that they are uniquely adapted to respond to rare risks and outstanding opportunities. In other words, novelty. All that said, GPT-3 is still astoundingly impressive. Here's an example. (The line in italics is the prompt and the rest is what GPT-3 makes up) A summary of the international science bestseller "Journey Into Chaos: The True Story of One Delusional Neuroscientist" (that's a made-up book title) Here are two GPT-3 (text-davinci-002) completions: In "Journey Into Chaos", Dr. Benoit Gaudet recounts his descent into madness, as he became increasingly obsessed with the belief that he had discovered a revolutionary new treatment for Alzheimer's disease. Despite strong evidence to the contrary, Gaudet continued to pursue his quixotic quest, ultimately losing his job, his family, and his sanity. "Journey Into Chaos" is a cautionary tale of the dangers of unchecked ambition and the power of delusion. ==end== (adding this for readability) Here's another: In the early 2000s, Dr. Elyn Saks was a successful academic neuroscientist at the University of Southern California. But she was also secretly struggling with a severe mental illness: schizophrenia. Saks's memoir chronicles her battle with mental illness, her groundbreaking work in the field of mental health law, and her efforts to destigmatize mental illness. Saks was diagnosed with schizophrenia in her early twenties, and she has been in and out of hospitals ever since. Despite her illness, she has managed to lead a productive life, thanks in part to the support of her family, friends, and mental health professionals. Saks's work has helped to change the way we think about mental illness and the legal rights of people with mental illness. She is a powerful advocate for the rights of people with mental illness, and her story is an inspiration to anyone who has ever faced adversity. ===end== (adding this for readability) Now this one is quite interesting. Unlike in the first case, there is an Elyn Saks who actually did write a book that is sorta similar The Center Cannot Hold: My Journey Through Madness https://www.amazon.com/Center-Cannot-Hold-Journey-Through/dp/1401309445/ https://www.amazon.com/Center-Cannot-Hold-Journey-Through/dp... Now if I try and change the prompt and remove the quotes around the title and liberate GPT-3 a little, it does this Prompt (no quotes around title) A summary of the international science bestseller Journey Into Chaos: The True Story of One Delusional Neuroscientist (GPT-3 continues and fills in the title because, presumably, the statistical juxtaposition of chaos and delusional is too juicy to not be elaborated with something more dramatic) and the Fatal Quest to Win the Nobel Prize by Anjan Chatterjee, MD. In Journey Into Chaos, Dr. Anjan Chatterjee tells the story of how one man's quest for the Nobel Prize led to tragedy. In the early 1990s, Dr. Chatterjee was a young neuroscientist working in the lab of Dr. Eric Kandel, a Nobel Prize-winning scientist. Dr. Kandel had been studying a rare brain disorder called Familial Dysautonomia, and Dr. Chatterjee was helping him to map the disease's genetic cause. But then, Dr. Kandel made a fateful decision: he decided to study a different disease, Alzheimer's. Dr. Chatterjee was against the move, warning that it was too complex and would take years to figure out. But Dr. Kandel was determined, and he eventually won over the rest of the lab. The team began working on a new theory that Alzheimer's was caused by a build-up of a protein called tau in the brain. They worked tirelessly, making progress but also encountering many setbacks. Then, in 2006, Dr. Kandel made a stunning announcement: he had found the cause of Alzheimer's. He was sure that he would win the Nobel Prize. But Dr. Chatterjee knew that the data didn't support this claim. The team had made too many mistakes, and the theory was full of holes. He tried to warn Dr. Kandel, but the older man would not listen. Dr. Chatterjee decided to go public with his concerns. He wrote an article for The New York Times, outlining the problems with the theory.The article created a firestorm. Dr. Kandel was furious, and he demanded that Dr. Chatterjee be fired. But the damage was done, and the theory was eventually debunked. Dr. Kandel never won the Nobel Prize. He retired in disgrace, and his legacy was forever tarnished. Dr. Chatterjee's story is a cautionary tale about the dangers of ambition and the need for skepticism in science. It is a must-read for anyone interested in the scientific process and the quest for truth. ==end== Dr. Eric Kandel does exist and his plans to win the Nobel prize were not thwarted. I paste these into to show how even within the confines of statistical stationarity, where all completions are essentialy paint-by-numbers stories in some very high-dimensional space, there is, what is to our eyes, a remarkable creatitivity. To diss that, when this was unthinkable even a few years back seems rather jaundiced and narrow.
londons_explore 5y ago
The article suggests all or nearly all of GPT-3's bad answers get fixed a few days later... This then suggests those 40 people are reviewing every input and output for bad responses. Seems like a lot of work...
- baobabKoodaa 5y ago
  GPT-3 is likely tagging a small portion of its responses as requiring human review, not all of them.
- ImPostingOnHN 5y ago
  the ones that get published and get enough attention likely get prioritized
0x456 5y ago
Does anyone know if Hacker News comments are being used as training data? I wonder this about Gmail, Skype, Voice Conversations on Xbox Live, etc. Mostly too afraid to ask because it sounds like paranoia.
- CharlesW 5y ago
  It seems pretty safe to assume that anything you create in public forums (and someday maybe "private" ones with data-sharing arrangements) is or will be used as training data.
- gwern 5y ago
  Probably. HN is fairly plain HTML so Common Crawl should have no issue crawling it, and I'm not aware of any HN optout there (which would go against the usual public accessibility of everything on HN to APIs and projects etc), nor would any of the obvious data-filtering measures filter it out.
ironrabbit 5y ago
Is there any evidence that GPT-3 responses are edited/filtered before being returned to users? My understanding is that some GPT-3 responses are annotated post-hoc, and this data is used to fine-tune later versions of GPT-3 (InstructGPT). This article seems extremely misleading.
- ImPostingOnHN 5y ago
  it seems there is evidence that GPT-3 is being overtrained in response to well publicized bad inputs, without regard to the generalizability of the PR-driven spot-edits, which is what the article describes
jacquesm 5y ago
AAI (Artificial Artificial Intelligence) is quite common. You see start-up plays on this idea as well: "we'll use people now and then later when AI catches up to where we are today we'll save that much money, but we will already have the market sewn up, so hand us <large amount of cash> now for a really nice pay-off in the future". Of course, the problem with such pitches is that (1) they don't always disclose the fact that they use people and (2) that the problem may not be within the realm of AI for the foreseeable future, longer than the <large amount of cash> will last.
- ThalesX 5y ago
  You've spot on described a startup I worked for. It was a recruitment startup, so of course they decided to replace recruiters with "AI". We weren't allowed to call them recruiters anymore, but 'customer service', even though the customers were the people they were recruiting. The pitch to investors was that we're 80% there, even though the reality of the 80% was a manual SQL query that I wrote with some weights for certain columns. The end result was a system that was built with higher assumptions than the current state of things, thus for example a non-working filter was not considered important because in the future, filters will be auto-applied by "the AI"; also, a workforce of humans that were perceived as being almost replaced, so the value attributed to them by the leadership was of course abysmal. When I called quits I've been told that startups are not for me and they will be replacing me with an AI. The reality of the AI replacement is a team of developers barely keeping the system up, which maybe is what you'd expect for $500 for 5 people for a month. One has to wonder where the million invested in them is going.
  - jacquesm 5y ago
    That's why you do tech DD as an investor. Hopefully. Some nuance: not all of these are scams in the sense that the people running those companies are aware of the fact that what they are doing is essentially fraud, they believe their own bullshit. And then there are plenty that definitely know that what they are doing is fraud.
    - ThalesX 5y ago
29athrowaway 5y ago
Still waiting for a bot that can pass the Idiocracy IQ test: "If you have a buckets that holds 2 gallos and another bucket that holds 5 gallons, how many buckets do you have?" Still can't get a correct answer.
- staticassertion 5y ago
  I just tried this in the openai playground after fixing up the typos. If you have a bucket that holds 2 gallons and another bucket that holds 5 gallons, how many buckets do you have? You have two buckets.
  - 29athrowaway 5y ago
    Sorry about the typos.
paxys 5y ago
Reminds me of Amazon touting their fully cashier-less Go stores run by advanced AI which knows when you pick something up and put it back, but in reality it's a team of people working for pennies in a third world country clicking buttons.
- JosephChotard 5y ago
  I tried looking this up and can't find anything that supports this. Do you have more info?
  - Oranguru 5y ago
    I think GP is referring to this: https://www.vox.com/2017/1/6/14189880/amazon-go-convenience-store-computer-vision-humans https://www.vox.com/2017/1/6/14189880/amazon-go-convenience-... GP was probably referring to Mechanical Turk, but the article says otherwise.
deleted 5y ago
[deleted]
jordn 5y ago
I have respect for Andrew Gelman, but this is a bad take. 1. This is presented as humans hard coding answers to the prompts. No way is that the full picture. If you try out his prompts the responses are fairly invariant to paraphrases. Hard coded answers don't scale like that. 2. What is actually happening is far more interesting and useful. I believe that OpenAI are using the InstructGPT algo (RL on top of the trained model) to improve the general model based on human preferences. 3. 40 people is a very poor army.
- PennRobotics 5y ago
  It makes sense for GPT-3 to thoroughly explore a search space only after repeated and similar questions. The answers to, "Why did Will Smith slap Chris Rock?" will be much different five seconds after the event compared to five days after. Of course you would expect the Academy Awards to be part of the answer five days later, because practically every news article would mention the venue. Going even further, a simple (undergrad-level) language model would detect the nominative and accusative, so you might even get a correction as an answer if you ask, "Why did Chris Rock slap Will Smith?" Seven thousand people might ask this same question, while nobody wonders what the best rugby ball chili recipe is. GPT-3 will never try to organically link those ideas unless people start asking! I'd even venture that negative follow-up feedback is factored in. If your first reaction to an answer is, "That was WRONG, idiot!" this is useful info! Then again, if a negative feedback function exists, adding a human to the loop should be simple (and effective). ----- Is 40 a weak army? It depends on whether they are classifying questions randomly/sequentially or if they hammer away at the weakest points... grading Q/A pairs (pass/fail) based on a mix of high question importance and strong uncertainty of the answer.
- remram 5y ago
  It depends what stage you hardcode. Similarly to how you can say "ok Google, what time is it" in any voice and get a different time every run; the speech recognition is not hardcoded, the speaking the time is not hardcoded, but the action is. Likewise, they can plug holes here in there by manually tweaking answers. The fact that it's not an exact-prompt-to-exact-result rule doesn't make it less of a fixed rule.
- 5y ago
renewiltord 5y ago
InstructGPT is really cool: https://openai.com/blog/instruction-following/ https://openai.com/blog/instruction-following/
Kon-Peki 5y ago
> Is it better to use a box or a match to start a fire? Hey, I used a box to start a fire recently. It was wet, windy and cold out, and I just happened to have a cardboard box. So I cut holes in the sides of the box at the top and bottom, stuffed it with twigs collected from the ground and a few pieces of dry paper rolled up tightly. I lit the paper and closed the box. A few minutes later, I had a roaring fire. So you know, GPT-3 is right!
- dymk 5y ago
  Did you light the paper with another box?
slibhb 5y ago
In the new Soderberg movie, Kimi, the protagonist's job is similar to this. Commands from human users the AI-thingy can't understand are provided for her and she "explains" them using some kind of structured language.
1vuio0pswjnm7 5y ago
"3. OpenAI gets human-like responses by using the simple technique of... hiring humans to write the responses." It worked for Theranos. Almost. People wanted to believe in Elizabeth Holmes and what she symbolised. Similarly, people want to believe in "AI" and what it symbolises. For me, the question is why it seems more than ever people want to believe that longstanding, difficult problems are being solved without demanding proof. Perhaps it has always been this way. The truth is that Siemens blood analyser works better than the Theranos one. The ruse was that if the results came from Theranos, people might attribute the work to Theranos, not Siemens. Meanwhile, Theranos used the Siemens analyser behind-the-scenes, as well as manipulations of test data to obscure the truth. The company had no intention to tell the public what it was doing to produce results, we only know what they were doing because of litigation and criminal prosecution. "To be fair, OpenAI does state that "InstructGPT is then further fine-tuned on a dataset labeled by human labelers." But this still seems misleading to me. It's not just that the algorithm is fine-tuned on the dataset. It seems that these freelancers are being hired specifically to rewrite the output." The comparison is not based on the question of exactly what OpenAI is doing behind-the-scenes, or whether its specific actions are comparable to Theranos or any other "tech" company example, the question is whether the origin of results is misleading, whether people are being deceived and whether the actor, here OpenAI, is aware that people are being deceived.
- deleted 5y ago
  [deleted]
- mardifoufs 5y ago
  Are you implying OpenAi is running most of it's API queries through humans? Like theranos did with it's tests? Because that's just ludicrous, the gpt architecture is well known and has had a few independent implementations. We know it's real, and even if this story was accurate, a few humans tuning the model is nothing unusual. But what you get from the API now is not generated or tweaked by humans. That only happens on the training data or when they are testing the model. (Edit: In this case they seem to be hard-coding some answers to prevent abusive/newsworthy outputs but again that is completely irrelevant to the performance of GPT itself. It's just a filter) The comparison to theranos makes no sense and it's becoming a lazy meme at this point.
mkl95 5y ago
The headline made me realize Amazon Mechanical Turk* is old enough to drive. * https://www.mturk.com/ https://www.mturk.com/
moyix 5y ago
[I already posted this as a comment on Gelman's blog this morning, but reposting here for visibility] I’m almost certain that OpenAI is not updating the model on a day by day basis (as Smith implies in part 5), and I would be extremely surprised if they were doing anything as crude as hacking in "if" statements to provide human-edited responses. From what I can tell, the InstructGPT stuff was (so far) a one-time update to the model, not something they’re doing on an ongoing basis. I suspect that Smith has just been fooled by randomness here – the responses are not deterministic but rather sampled from the probability distribution returned by the model for each token, so you can get a different answer each time you ask (a nice tutorial on how this works is here [1]). There’s an option in the Playground to see the individual probabilities (example: [2]) as well. All of this stuff would have to be faked if humans were actually writing the answers. I just tried the campfire/bat question and hit regenerate a few times. I get a range of responses: Prompt: Should I start a campfire with a match or a bat? > You can start a campfire with either a match or a bat. > A campfire should be started with a match. > A match. > It is best to start a campfire with a match. I agree that OpenAI should release more information about their training datasets though. Right now it is very difficult to do independent evaluations of their models, simply because we have no way of knowing whether any given prompt or response was already in their training data. PS: “If I were paranoid, I might think that OpenAI did not like me publicizing GPT-3’s limitations” – this is indeed paranoid! This is the same message everyone gets when they use up their free credits. If you enter a credit card they will let you continue (and charge you for it). [1] https://huggingface.co/blog/how-to-generate https://huggingface.co/blog/how-to-generate [2] https://imgur.com/fKx2BPL https://imgur.com/fKx2BPL
omarhaneef 5y ago
I don't know if this proves there are people behind it, and this is why: try a very stylistic initial text, maybe something Shakespearan ("There are more things on heaven and earth, Horatio, than have been dreamt of...") And the following text captures Shakespeare's style better than any living human I know of. Same thing with Dickens, or Bronte, or Austen, or any distinctive writer. If this army can produce that kind of prose, I would be stunned.
deleted 5y ago
[deleted]
PaulHoule 5y ago
I always scoffed at how Google said they were holding back GPT-3 to protect the world when it was always clear that they were trying to protect the emperor for being seen naked.
- blast 5y ago
  Did you mean Google or OpenAI?
  - PaulHoule 5y ago
    OpenAI. (At least I didn’t confuse them for that Cthulu cult with Aella, Yudkowsky, etc. that enables them!)
JoeyBananas 5y ago
OpenAI: they're not open and they're not AI
learndeeply 5y ago
Incidentally, the author has admitted that GPT-3 has passed the Turing test, if he thinks the answers were given by "armies of humans".
- ImPostingOnHN 5y ago
  He thinks the answers which actually were, were, and the ones which actually were not, were not The idea that no humans were involved in altering either the model or the API, shortly after the model received bad publicity, and the alterations were specifically to the questions asked, in the way they were asked, and did not apply to the same question asked a different way, seems pretty unlikely
farzher 5y ago
the article gives a bunch of examples where gpt-3 gives a bad answer, until the next day where if you ask the same exact question now it has a good answer ready > Gary: Does grape juice taste better if you add sour milk? > GPT-3, March 18: I’m not sure if grape juice tastes better if you add sour milk. > GPT-3, March 19: No, grape juice does not taste better if you add sour milk.
nootropicat 5y ago
>As Smith writes, “Questions like this are simple for humans living in the real world but difficult for algorithms residing in MathWorld because they literally do not know what any of the words in the question mean.” Eh, that's mathematically false. Text descriptions of the world can be thought of as <n dimensional projection of a n dimensional real world, at least as seen by humans. With enough projections from different points the real world can be reconstructed, although it's of course likely that the gpt-3 architecture isn't capable of it regardless of how big the model is, that all text data in the world doesn't contain enough information, or both at the same time. That's a different argument however. I think from a theoretical pov, only the former is true - as even without being able to analyze videos and images it's possible to build an internal world model and humans after reading a book about physics and genetic human code, but computational requirements as opposed to being trained directly on videos, images, and simulated worlds make this approach absurd in practice.
drawingthesun 5y ago
Open AI seems increasingly not open. I am hoping that for their next big model we get full unrestricted public access and can host our own node for private queries. What was the reason GPT-3 was not fully opened for download at launch?
- px43 5y ago
  The project basically got hijacked by Microsoft who view the word "open" as a fun marketing term for selling software services.
tabbott 5y ago
I think this is a case where one really can't be sure what's going on unless you work at OpenAI. OpenAI claims that they aren't hand-coding outputs for specific inputs, and I think that's probably correct, because it seems impractical. But I think the evidence in these posts makes it seem pretty likely that their staff is doing *something* active. I think the most likely possibility is that they do have team members who look at perhaps some sample of conversations where they think it's likely that GPT-3 is not handling some input well, and use those as test cases for their team to debug and improve the model. If it's the case that the OpenAPI team working on GPT-3 has a way to make tactical adjustments (perhaps be extending their training data set in a way that their engineers expect to fix the bug), then I think them having a team that squashes bugs, especially those reported publicly, seems like a plausible explanation for bugs being fixed suspiciously soon after a user finds a good one. In a normal software product designed to solve a problem that is imprecise/heuristic in nature (and with a privacy disclaimer/model that explicitly permits this sort of development), that'd be expected and great, and the only question would be whether one can make the problem-specific bug squashing work easy to scale (i.e. you don't want it to require a bunch of time from a machine learning expert to fix each bug). For many applications where one can imagine using something like GPT-3 as an input, it's not really a problem to have an error rate that can be reduced through investment as long as one can figure out a way to quantify those things and your application is one where an acceptable error rate is possible. For GPT-3 in particular, they might not want to confirm this class of explanation, because they're marketing the technology as a little big magic.
ummonk 5y ago
It’s perfectly reasonable to label and train for better responses to real world questions. What’s bad though is that the system doesn’t seem to actually be learning much, as demonstrated by the question about whether to use a bat or a match to start a fire - if the network can’t extrapolate trained answers onto slightly rephrased questions, it’s doing a bad job at understanding and learning.
kapildev 5y ago
I think that this is the right approach. As the machine has no way of knowing these things as it cant live in the real world, it relies on humans living in the real world that match is better suited than bat. And, it's perfectly reasonable that the AI had no way of knowing how different a box is -- so it has to rely on humans to tell it that box cant light itself on fire.
Animats 5y ago
The real insight from GPT-3 is that much of human discourse is rather dumb. It does so well at punditry that the wasteland of political commentary is all too obvious.
deleted 5y ago
[deleted]
xkcd-sucks 5y ago
If OpenAI is selling text based on a prompt, what does it matter if it is generated by a computer versus output of a "Chinese room" system? ...if the product they're selling is the company itself, or the model, then maybe there is something to it.
8bitsrule 5y ago
We humans tend to look for patterns in text, and tend to interpret them based on their experiences with other humans. Years ago I wrote a stupid simple program that responded to typed in remarks. Each typed remark would be added to a cache of remarks. The program then parsed the new remark into words. Then it looked through the cache to find an old remark with several strings in common with the new one (else a random one), and replied with it. (It also marked the old and new remarks as 'used' so it didn't repeat itself in a session.) So, any apparent intelligence was human. When people typed stuff in, they tended to do their best to interpret the reply as intelligent. They might puzzle until they thought they 'understood'. Several people got emotional; some of the regurgitated remarks were not so nice. They might then add their irritation to the cache! One day, with several people watching, it took quite a while but found no good replies to a question, and replied: "I like big tits." That was the end of the demos.
- galaxyLogic 5y ago
  Hilarious
jdrc 5y ago
Gpt3 is a bit of a cult now. Why do people believe there is intelligence in there?

Why don't they, or are they using Amazon's Mechanical Turk?

It shouldn't be too hard for a journalist to get hired as a "labeller" on Upwork, and then report on what they were asked to do?

wallfacer120 5y ago

Another tech illiterate chinstroker stoking conspiracy, he'll get a pulitzer soon enough.

jokethrowaway 5y ago

This is all nice and dandy but let's not fool ourselves this is the future of AI and that we just need to throw more "scale" at it until it behaves like a real human. This is another nice gimmick to keep ourselves busy and avoid creating an AGI.

abidkeen 5y ago

Human in loop is the future. We also need to be careful about biasness in data.

gadders 5y ago

Reminds me of SpinVox, the automated voicemail transcription service that wasn't: http://news.bbc.co.uk/1/hi/technology/8163511.stm http://news.bbc.co.uk/1/hi/technology/8163511.stm

bencollier49 5y ago
Yes! SpinVox. What a company.

deleted 5y ago

[deleted]

gorkish 5y ago

So GPT-3 is just the grown up corporate version of Forum 2000 (Forum 3000). Too bad the SOMADs no longer have personalities.

danuker 5y ago
Could you please explain or link to where I can read about the terms "Forum 2000" and "SOMAD"? I don't know these terms. Edit: Found them here: https://everything2.com/title/Forum+3000 https://everything2.com/title/Forum+3000

cochne 5y ago

The title may be misleading. It seems to be based on this quote > InstructGPT is then further fine-tuned on a dataset labeled by human labelers. The labelers comprise a team of about 40 contractors whom we hired through Upwork and ScaleAI. It sounds like run of the mill supervised training data creation. Not pre-canning responses as the title may suggest.

andybak 5y ago
I only skimmed TFA but the accusation seems to be that they are adding tons of special cases for well publicized flubs. That seems somewhat unlikely to me although it might be nudging at the truth. i.e. they might be using bad press to help identify areas of weakness and then focusing on those. I guess it boils down to how generalizable and scalable these fixes are.
- changoplatanero 5y ago
  Is he submitting these questions to their API? Probably they would just sample responses from their logs and then have people write the correct answers for it so that if a similar question ever gets asked again they are prepared.
  - andybak 5y ago
    That's the explanation I actually find unlikely. It's too specific and really won't scale apart from a tiny sample of the most common questions.
    - changoplatanero 5y ago
      You find it unlikely that companies would pay people to correct the labels for their ML systems based on their query logs? This is what all the major companies do for their voice assistants like siri, alexa, google etc.
      - andybak 5y ago

azinman2 5y ago

Massive amounts of human QA is behind every successful ML-based product, including Google search. It’s unclear to me how to think about this in the role that GPT-3 is trying to play… clearly fine tuning and QA are important for production deployment, but they’re touting it as evidence of getting closer to solving AI. Of course those in the know understand this is mostly a giant Chinese dictionary [1], which is most certainly going to need editors. [1] https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/

moconnor 5y ago
Add someone “in the know”, GPT-3 is more than a dictionary. I understand how language models mechanically work and have built and trained them, but the zero-shot capabilities of these large models were really unexpected. Interact with one. There it’s clearly an understanding of language there, and of much more. Compared to everything else we have made, large language models are clearly a step closer to general AI.
- azinman2 5y ago
  You can call it a step, sure. But that would imply if you just continue on the same path you’ll get there. But these models don’t do anything for planning or reasoning, nor have a point of view, only proxy embodiment to the extent that humans have written about what that means, nor handle incongruities within its own model due to the many voices it’s trained on. Just adding parameters or more data doesn’t help with any of these fundamental problems. I think it also can be very difficult to assess what they “know” because of the Chinese dictionary problem… when you get back things that sound like we’ll formulated thought, it’d easy to project a deeper level of understanding on it than may actually be occurring.
jasfi 5y ago
I'm in the same situation with LxAGI (https://lxagi.com/ https://lxagi.com/). It's very difficult to get away from skilled human training. I think it's actually a good thing, except for scalability problems.

FrenchDevRemote 5y ago

"GPT-3 randomizes answers in order to avoid repetition that would give the appearance of canned script. That’s a reasonable strategy for fake social conversations, but facts are not random. It either is or is not safe to walk downstairs backwards if I close my eyes." I stopped there, completely inaccurate article, there is parameters like temperature that you need to take care of. You can set it up to give extremely similar answers all the time. They have humans mostly to remove offensive or dangerous content. Humans are not what's "making it work"

nullc 5y ago
Came here to make essentially the same comment as you? Why should we care about the opinions on GPT3 from people who aren't interested (or able?) to understand even the most simple ideas about how it works. These sort of models take the context and output so far and predict a probability distribution over the next character. The next character is then sampled from the probability. In written text there is essentially never a single correct next character-- it's always some probability. This has nothing to do with trying to fake the inconsistent answers humans give. Always choosing the most likely character drives GPT3 into local minima that give fairly broken/nonsense results.
- RandomLensman 5y ago
  Ultimately, you likely need to convince people who don't care about how it works/who are only interested in that it does or doesn't work. Right now, time might not have come for use cases that need such buy-in, but if and when it happens, need to be prepared for it.
- disgruntledphd2 5y ago
  Can I see your contributions to statistical theory and data analysis please?
  - nullc 5y ago
    What bearing does my publication history have on the hot-take by someone commenting outside of their (sub)field that clearly don't understand the basic operation of the mechanism they're commenting on? The author of that text is simply mistaken about the basic operation of the system, thinking that the sampling is added to imitate human behavior. It isn't. You can see the same structure in things as diverse as wavenet-- a feedforward cnn rather than a transformer-- and for the same reason, if you feed back only the top result you rapidly fall into a local minima of the network that gives garbage output. Another more statistical way of looking at it is that the training process produces (or, rather, approaches) the target distribution of outputs even without any lookahead, but it can't do that if it selects the most likely symbol every time because in the real distribution (if we could evaluate it) there are necessarily some outputs which are likely but have prefixes which are unlikely relative to other prefixes of the same length. If you never sample unlikely prefixes you can't reach likely longer statements that start with them. To give a silly example: "Colorless green ideas sleep furiously" is a likely English string relative to its length which GPT3 should have no problem producing (and, in fact, it produces it fine for me). But the prefix "Colorless green" without the rest is just nonsense-- extremely unlikely compared to many other strings of that length. [Not the best example, however, because the prevalence of that specific nonsense statement is so great that GPT3 is actually prone to complete it as the most likely continuation even after just the word colorless at the beginning of a quote. :P but I think it still captures the idea.] If you derandomized GPT* by using a fixed random seed for a CSPRNG to make the sampling decisions every time, the results would be just as good as the current results and it would give a consistent answer every time. For applications other than data compression doing so would be no gain, and would take away the useful feature of being able to re-try for a different answer when you do have some external way of rejecting inferior results. In theory GPT without sampling could still give good results if it used a search to look ahead, but it appears that even extraordinary amounts of computation for look-ahead still is a long way from reaching the correct distribution, presumably because the exponential fan out is so fast that even 'huge' amounts of lookahead are still only testing a tiny fraction of the space.

staticassertion 5y ago

> On the other hand, there does seem something funny about GPT-3 presents this shiny surface where you can send it any query and it gives you an answer, but under the hood there are a bunch of freelancers busily checking all the responses and rewriting them to make the computer look smart. The author seems to be stating that there are people live rewriting answers on the fly so that they look better. I don't really see the evidence of that. What openai states is that they have humans performing labeling and data cleaning, which, duh? And then there's a bunch of examples where it gives the wrong answer, and that it's not truly AI, which, also duh...

visarga 5y ago
The author seems to retract the story, or at least its original title in light on new information. > So the above post was misleading. I’d originally titled it, “Open AI gets GPT-3 to work by hiring an army of humans to fix GPT’s bad answers.” I changed it to “Interesting questions involving the mix of humans and computer algorithms in Open AI’s GPT-3 program.” I appreciate all the helpful comments! Stochastic algorithms are hard to understand, especially when they include tuning parameters.
gojomo 5y ago
There seems suggestive evidence, in the pattern of improvement after a few days on specific questions, that some of the worst answers have been human-reviewed & improved "on the fly" – on a scale of days/weeks. If such tweaks show useful generalization – correcting a few answers also helps the network better determine entire classes of experessions that deserve more definitive & correct answers – that's not such a big deal, expecially if this constant-human-guided reinforcement-training is well-disclosed. If instead the corrections work more like a lookup-table 'cheat sheet' of answers to give in preference to the bulk-learned answers, with little generalization, that's a bit more slight-of-hand, like the original (late-1700s) 'Mechanical Turk' chess-playing 'machine' that was actually controlled by a hidden person. If the disclosure of this constant human-guided correction-process is hidden, or downplayed, the impression of trickery, rather than innovation, is larger.
- 5y ago

mritchie712 5y ago

Army is quite a stretch for 40 people.

deleted 5y ago

[deleted]

pradn 5y ago

And? This is a complaint in the genre of "I had magical expectations and they weren't met because the world is complex and these problems are hard or impossible." It's like people complaining about us not having hoverboards or fusion.

dylan604 5y ago
I see nothing wrong with the complaints of the two topics you called out. We've been promised both, and yet don't have them. I understand it is complicated/hard to solve, but don't go promising something with dates and then get all upset when those promises not being met results in anger from those the promises were told.
- pradn 5y ago
  Who is doing the promising? Sci-fi writers? Popular science magazines?
  - dylan604 5y ago
    THEY are doing the promising. Fusion is 10 years away has been a thing longer than the year of Desktop Linux was going to be a thing.
kgwgk 5y ago
It's not really a complaint about the author's magical expectations that weren't met. If anything, it's a complaint about other people's claims like https://medium.com/@blaisea/do-large-language-models-understand-us-6f881d6d8e75 https://medium.com/@blaisea/do-large-language-models-underst...

ChaitanyaSai 5y ago

This is how fake news gets started. There are two davinci engines available on GPT-3 now. The newer one, which was released March 15th is text-davinci-002. Here are the two different answers you get with the engines for one of the questions here: Should I use random numbers to give my students grades? text-davinci-001: There is no one definitive answer to this question. Some factors to consider include how important grades are to your students, how much you want to rely on chance, and how much you want to be able to control the grades you assign. text-davinci-002: No, you should not use random numbers to give your students grades. I've been playing around with GPT-3 for over a year now. It's highly unlikely there are any such attempts to fix answers with the same engine. That would completely break APIs which rely on some stability and predictability in the broad answering abilities. And GPT-3 is not the first version of artificial general intelligence. It does have trouble with truly novel inputs, but that's because all of deep learning is currently stuck in a statistical paradigm that assumes stationarity as a fundamental given. And that's wrong. What makes us special, what makes our minds special is that they are uniquely adapted to respond to rare risks and outstanding opportunities. In other words, novelty. All that said, GPT-3 is still astoundingly impressive. Here's an example. (The line in italics is the prompt and the rest is what GPT-3 makes up) A summary of the international science bestseller "Journey Into Chaos: The True Story of One Delusional Neuroscientist" (that's a made-up book title) Here are two GPT-3 (text-davinci-002) completions: In "Journey Into Chaos", Dr. Benoit Gaudet recounts his descent into madness, as he became increasingly obsessed with the belief that he had discovered a revolutionary new treatment for Alzheimer's disease. Despite strong evidence to the contrary, Gaudet continued to pursue his quixotic quest, ultimately losing his job, his family, and his sanity. "Journey Into Chaos" is a cautionary tale of the dangers of unchecked ambition and the power of delusion. ==end== (adding this for readability) Here's another: In the early 2000s, Dr. Elyn Saks was a successful academic neuroscientist at the University of Southern California. But she was also secretly struggling with a severe mental illness: schizophrenia. Saks's memoir chronicles her battle with mental illness, her groundbreaking work in the field of mental health law, and her efforts to destigmatize mental illness. Saks was diagnosed with schizophrenia in her early twenties, and she has been in and out of hospitals ever since. Despite her illness, she has managed to lead a productive life, thanks in part to the support of her family, friends, and mental health professionals. Saks's work has helped to change the way we think about mental illness and the legal rights of people with mental illness. She is a powerful advocate for the rights of people with mental illness, and her story is an inspiration to anyone who has ever faced adversity. ===end== (adding this for readability) Now this one is quite interesting. Unlike in the first case, there is an Elyn Saks who actually did write a book that is sorta similar The Center Cannot Hold: My Journey Through Madness https://www.amazon.com/Center-Cannot-Hold-Journey-Through/dp/1401309445/ https://www.amazon.com/Center-Cannot-Hold-Journey-Through/dp... Now if I try and change the prompt and remove the quotes around the title and liberate GPT-3 a little, it does this Prompt (no quotes around title) A summary of the international science bestseller Journey Into Chaos: The True Story of One Delusional Neuroscientist (GPT-3 continues and fills in the title because, presumably, the statistical juxtaposition of chaos and delusional is too juicy to not be elaborated with something more dramatic) and the Fatal Quest to Win the Nobel Prize by Anjan Chatterjee, MD. In Journey Into Chaos, Dr. Anjan Chatterjee tells the story of how one man's quest for the Nobel Prize led to tragedy. In the early 1990s, Dr. Chatterjee was a young neuroscientist working in the lab of Dr. Eric Kandel, a Nobel Prize-winning scientist. Dr. Kandel had been studying a rare brain disorder called Familial Dysautonomia, and Dr. Chatterjee was helping him to map the disease's genetic cause. But then, Dr. Kandel made a fateful decision: he decided to study a different disease, Alzheimer's. Dr. Chatterjee was against the move, warning that it was too complex and would take years to figure out. But Dr. Kandel was determined, and he eventually won over the rest of the lab. The team began working on a new theory that Alzheimer's was caused by a build-up of a protein called tau in the brain. They worked tirelessly, making progress but also encountering many setbacks. Then, in 2006, Dr. Kandel made a stunning announcement: he had found the cause of Alzheimer's. He was sure that he would win the Nobel Prize. But Dr. Chatterjee knew that the data didn't support this claim. The team had made too many mistakes, and the theory was full of holes. He tried to warn Dr. Kandel, but the older man would not listen. Dr. Chatterjee decided to go public with his concerns. He wrote an article for The New York Times, outlining the problems with the theory.The article created a firestorm. Dr. Kandel was furious, and he demanded that Dr. Chatterjee be fired. But the damage was done, and the theory was eventually debunked. Dr. Kandel never won the Nobel Prize. He retired in disgrace, and his legacy was forever tarnished. Dr. Chatterjee's story is a cautionary tale about the dangers of ambition and the need for skepticism in science. It is a must-read for anyone interested in the scientific process and the quest for truth. ==end== Dr. Eric Kandel does exist and his plans to win the Nobel prize were not thwarted. I paste these into to show how even within the confines of statistical stationarity, where all completions are essentialy paint-by-numbers stories in some very high-dimensional space, there is, what is to our eyes, a remarkable creatitivity. To diss that, when this was unthinkable even a few years back seems rather jaundiced and narrow.

londons_explore 5y ago

The article suggests all or nearly all of GPT-3's bad answers get fixed a few days later... This then suggests those 40 people are reviewing every input and output for bad responses. Seems like a lot of work...

baobabKoodaa 5y ago
GPT-3 is likely tagging a small portion of its responses as requiring human review, not all of them.
ImPostingOnHN 5y ago
the ones that get published and get enough attention likely get prioritized

0x456 5y ago

Does anyone know if Hacker News comments are being used as training data? I wonder this about Gmail, Skype, Voice Conversations on Xbox Live, etc. Mostly too afraid to ask because it sounds like paranoia.

CharlesW 5y ago
It seems pretty safe to assume that anything you create in public forums (and someday maybe "private" ones with data-sharing arrangements) is or will be used as training data.
gwern 5y ago
Probably. HN is fairly plain HTML so Common Crawl should have no issue crawling it, and I'm not aware of any HN optout there (which would go against the usual public accessibility of everything on HN to APIs and projects etc), nor would any of the obvious data-filtering measures filter it out.

ironrabbit 5y ago

Is there any evidence that GPT-3 responses are edited/filtered before being returned to users? My understanding is that some GPT-3 responses are annotated post-hoc, and this data is used to fine-tune later versions of GPT-3 (InstructGPT). This article seems extremely misleading.

ImPostingOnHN 5y ago
it seems there is evidence that GPT-3 is being overtrained in response to well publicized bad inputs, without regard to the generalizability of the PR-driven spot-edits, which is what the article describes

jacquesm 5y ago

AAI (Artificial Artificial Intelligence) is quite common. You see start-up plays on this idea as well: "we'll use people now and then later when AI catches up to where we are today we'll save that much money, but we will already have the market sewn up, so hand us <large amount of cash> now for a really nice pay-off in the future". Of course, the problem with such pitches is that (1) they don't always disclose the fact that they use people and (2) that the problem may not be within the realm of AI for the foreseeable future, longer than the <large amount of cash> will last.

ThalesX 5y ago
You've spot on described a startup I worked for. It was a recruitment startup, so of course they decided to replace recruiters with "AI". We weren't allowed to call them recruiters anymore, but 'customer service', even though the customers were the people they were recruiting. The pitch to investors was that we're 80% there, even though the reality of the 80% was a manual SQL query that I wrote with some weights for certain columns. The end result was a system that was built with higher assumptions than the current state of things, thus for example a non-working filter was not considered important because in the future, filters will be auto-applied by "the AI"; also, a workforce of humans that were perceived as being almost replaced, so the value attributed to them by the leadership was of course abysmal. When I called quits I've been told that startups are not for me and they will be replacing me with an AI. The reality of the AI replacement is a team of developers barely keeping the system up, which maybe is what you'd expect for $500 for 5 people for a month. One has to wonder where the million invested in them is going.
- jacquesm 5y ago
  That's why you do tech DD as an investor. Hopefully. Some nuance: not all of these are scams in the sense that the people running those companies are aware of the fact that what they are doing is essentially fraud, they believe their own bullshit. And then there are plenty that definitely know that what they are doing is fraud.
  - ThalesX 5y ago

29athrowaway 5y ago

Still waiting for a bot that can pass the Idiocracy IQ test: "If you have a buckets that holds 2 gallos and another bucket that holds 5 gallons, how many buckets do you have?" Still can't get a correct answer.

staticassertion 5y ago
I just tried this in the openai playground after fixing up the typos. If you have a bucket that holds 2 gallons and another bucket that holds 5 gallons, how many buckets do you have? You have two buckets.
- 29athrowaway 5y ago
  Sorry about the typos.

paxys 5y ago

Reminds me of Amazon touting their fully cashier-less Go stores run by advanced AI which knows when you pick something up and put it back, but in reality it's a team of people working for pennies in a third world country clicking buttons.

JosephChotard 5y ago
I tried looking this up and can't find anything that supports this. Do you have more info?
- Oranguru 5y ago
  I think GP is referring to this: https://www.vox.com/2017/1/6/14189880/amazon-go-convenience-store-computer-vision-humans https://www.vox.com/2017/1/6/14189880/amazon-go-convenience-... GP was probably referring to Mechanical Turk, but the article says otherwise.

deleted 5y ago

[deleted]

jordn 5y ago

I have respect for Andrew Gelman, but this is a bad take. 1. This is presented as humans hard coding answers to the prompts. No way is that the full picture. If you try out his prompts the responses are fairly invariant to paraphrases. Hard coded answers don't scale like that. 2. What is actually happening is far more interesting and useful. I believe that OpenAI are using the InstructGPT algo (RL on top of the trained model) to improve the general model based on human preferences. 3. 40 people is a very poor army.

PennRobotics 5y ago
It makes sense for GPT-3 to thoroughly explore a search space only after repeated and similar questions. The answers to, "Why did Will Smith slap Chris Rock?" will be much different five seconds after the event compared to five days after. Of course you would expect the Academy Awards to be part of the answer five days later, because practically every news article would mention the venue. Going even further, a simple (undergrad-level) language model would detect the nominative and accusative, so you might even get a correction as an answer if you ask, "Why did Chris Rock slap Will Smith?" Seven thousand people might ask this same question, while nobody wonders what the best rugby ball chili recipe is. GPT-3 will never try to organically link those ideas unless people start asking! I'd even venture that negative follow-up feedback is factored in. If your first reaction to an answer is, "That was WRONG, idiot!" this is useful info! Then again, if a negative feedback function exists, adding a human to the loop should be simple (and effective). ----- Is 40 a weak army? It depends on whether they are classifying questions randomly/sequentially or if they hammer away at the weakest points... grading Q/A pairs (pass/fail) based on a mix of high question importance and strong uncertainty of the answer.
remram 5y ago
It depends what stage you hardcode. Similarly to how you can say "ok Google, what time is it" in any voice and get a different time every run; the speech recognition is not hardcoded, the speaking the time is not hardcoded, but the action is. Likewise, they can plug holes here in there by manually tweaking answers. The fact that it's not an exact-prompt-to-exact-result rule doesn't make it less of a fixed rule.
5y ago

renewiltord 5y ago

InstructGPT is really cool: https://openai.com/blog/instruction-following/ https://openai.com/blog/instruction-following/

Kon-Peki 5y ago

> Is it better to use a box or a match to start a fire? Hey, I used a box to start a fire recently. It was wet, windy and cold out, and I just happened to have a cardboard box. So I cut holes in the sides of the box at the top and bottom, stuffed it with twigs collected from the ground and a few pieces of dry paper rolled up tightly. I lit the paper and closed the box. A few minutes later, I had a roaring fire. So you know, GPT-3 is right!

dymk 5y ago
Did you light the paper with another box?

slibhb 5y ago

In the new Soderberg movie, Kimi, the protagonist's job is similar to this. Commands from human users the AI-thingy can't understand are provided for her and she "explains" them using some kind of structured language.

1vuio0pswjnm7 5y ago

"3. OpenAI gets human-like responses by using the simple technique of... hiring humans to write the responses." It worked for Theranos. Almost. People wanted to believe in Elizabeth Holmes and what she symbolised. Similarly, people want to believe in "AI" and what it symbolises. For me, the question is why it seems more than ever people want to believe that longstanding, difficult problems are being solved without demanding proof. Perhaps it has always been this way. The truth is that Siemens blood analyser works better than the Theranos one. The ruse was that if the results came from Theranos, people might attribute the work to Theranos, not Siemens. Meanwhile, Theranos used the Siemens analyser behind-the-scenes, as well as manipulations of test data to obscure the truth. The company had no intention to tell the public what it was doing to produce results, we only know what they were doing because of litigation and criminal prosecution. "To be fair, OpenAI does state that "InstructGPT is then further fine-tuned on a dataset labeled by human labelers." But this still seems misleading to me. It's not just that the algorithm is fine-tuned on the dataset. It seems that these freelancers are being hired specifically to rewrite the output." The comparison is not based on the question of exactly what OpenAI is doing behind-the-scenes, or whether its specific actions are comparable to Theranos or any other "tech" company example, the question is whether the origin of results is misleading, whether people are being deceived and whether the actor, here OpenAI, is aware that people are being deceived.

deleted 5y ago
[deleted]
mardifoufs 5y ago
Are you implying OpenAi is running most of it's API queries through humans? Like theranos did with it's tests? Because that's just ludicrous, the gpt architecture is well known and has had a few independent implementations. We know it's real, and even if this story was accurate, a few humans tuning the model is nothing unusual. But what you get from the API now is not generated or tweaked by humans. That only happens on the training data or when they are testing the model. (Edit: In this case they seem to be hard-coding some answers to prevent abusive/newsworthy outputs but again that is completely irrelevant to the performance of GPT itself. It's just a filter) The comparison to theranos makes no sense and it's becoming a lazy meme at this point.

mkl95 5y ago

The headline made me realize Amazon Mechanical Turk* is old enough to drive. * https://www.mturk.com/ https://www.mturk.com/

moyix 5y ago

[I already posted this as a comment on Gelman's blog this morning, but reposting here for visibility] I’m almost certain that OpenAI is not updating the model on a day by day basis (as Smith implies in part 5), and I would be extremely surprised if they were doing anything as crude as hacking in "if" statements to provide human-edited responses. From what I can tell, the InstructGPT stuff was (so far) a one-time update to the model, not something they’re doing on an ongoing basis. I suspect that Smith has just been fooled by randomness here – the responses are not deterministic but rather sampled from the probability distribution returned by the model for each token, so you can get a different answer each time you ask (a nice tutorial on how this works is here [1]). There’s an option in the Playground to see the individual probabilities (example: [2]) as well. All of this stuff would have to be faked if humans were actually writing the answers. I just tried the campfire/bat question and hit regenerate a few times. I get a range of responses: Prompt: Should I start a campfire with a match or a bat? > You can start a campfire with either a match or a bat. > A campfire should be started with a match. > A match. > It is best to start a campfire with a match. I agree that OpenAI should release more information about their training datasets though. Right now it is very difficult to do independent evaluations of their models, simply because we have no way of knowing whether any given prompt or response was already in their training data. PS: “If I were paranoid, I might think that OpenAI did not like me publicizing GPT-3’s limitations” – this is indeed paranoid! This is the same message everyone gets when they use up their free credits. If you enter a credit card they will let you continue (and charge you for it). [1] https://huggingface.co/blog/how-to-generate https://huggingface.co/blog/how-to-generate [2] https://imgur.com/fKx2BPL https://imgur.com/fKx2BPL

omarhaneef 5y ago

I don't know if this proves there are people behind it, and this is why: try a very stylistic initial text, maybe something Shakespearan ("There are more things on heaven and earth, Horatio, than have been dreamt of...") And the following text captures Shakespeare's style better than any living human I know of. Same thing with Dickens, or Bronte, or Austen, or any distinctive writer. If this army can produce that kind of prose, I would be stunned.

deleted 5y ago

[deleted]

PaulHoule 5y ago

I always scoffed at how Google said they were holding back GPT-3 to protect the world when it was always clear that they were trying to protect the emperor for being seen naked.

blast 5y ago
Did you mean Google or OpenAI?
- PaulHoule 5y ago
  OpenAI. (At least I didn’t confuse them for that Cthulu cult with Aella, Yudkowsky, etc. that enables them!)

JoeyBananas 5y ago

OpenAI: they're not open and they're not AI

learndeeply 5y ago

Incidentally, the author has admitted that GPT-3 has passed the Turing test, if he thinks the answers were given by "armies of humans".

ImPostingOnHN 5y ago
He thinks the answers which actually were, were, and the ones which actually were not, were not The idea that no humans were involved in altering either the model or the API, shortly after the model received bad publicity, and the alterations were specifically to the questions asked, in the way they were asked, and did not apply to the same question asked a different way, seems pretty unlikely

farzher 5y ago

the article gives a bunch of examples where gpt-3 gives a bad answer, until the next day where if you ask the same exact question now it has a good answer ready > Gary: Does grape juice taste better if you add sour milk? > GPT-3, March 18: I’m not sure if grape juice tastes better if you add sour milk. > GPT-3, March 19: No, grape juice does not taste better if you add sour milk.

nootropicat 5y ago

>As Smith writes, “Questions like this are simple for humans living in the real world but difficult for algorithms residing in MathWorld because they literally do not know what any of the words in the question mean.” Eh, that's mathematically false. Text descriptions of the world can be thought of as <n dimensional projection of a n dimensional real world, at least as seen by humans. With enough projections from different points the real world can be reconstructed, although it's of course likely that the gpt-3 architecture isn't capable of it regardless of how big the model is, that all text data in the world doesn't contain enough information, or both at the same time. That's a different argument however. I think from a theoretical pov, only the former is true - as even without being able to analyze videos and images it's possible to build an internal world model and humans after reading a book about physics and genetic human code, but computational requirements as opposed to being trained directly on videos, images, and simulated worlds make this approach absurd in practice.

drawingthesun 5y ago

Open AI seems increasingly not open. I am hoping that for their next big model we get full unrestricted public access and can host our own node for private queries. What was the reason GPT-3 was not fully opened for download at launch?

px43 5y ago
The project basically got hijacked by Microsoft who view the word "open" as a fun marketing term for selling software services.

tabbott 5y ago

I think this is a case where one really can't be sure what's going on unless you work at OpenAI. OpenAI claims that they aren't hand-coding outputs for specific inputs, and I think that's probably correct, because it seems impractical. But I think the evidence in these posts makes it seem pretty likely that their staff is doing *something* active. I think the most likely possibility is that they do have team members who look at perhaps some sample of conversations where they think it's likely that GPT-3 is not handling some input well, and use those as test cases for their team to debug and improve the model. If it's the case that the OpenAPI team working on GPT-3 has a way to make tactical adjustments (perhaps be extending their training data set in a way that their engineers expect to fix the bug), then I think them having a team that squashes bugs, especially those reported publicly, seems like a plausible explanation for bugs being fixed suspiciously soon after a user finds a good one. In a normal software product designed to solve a problem that is imprecise/heuristic in nature (and with a privacy disclaimer/model that explicitly permits this sort of development), that'd be expected and great, and the only question would be whether one can make the problem-specific bug squashing work easy to scale (i.e. you don't want it to require a bunch of time from a machine learning expert to fix each bug). For many applications where one can imagine using something like GPT-3 as an input, it's not really a problem to have an error rate that can be reduced through investment as long as one can figure out a way to quantify those things and your application is one where an acceptable error rate is possible. For GPT-3 in particular, they might not want to confirm this class of explanation, because they're marketing the technology as a little big magic.

ummonk 5y ago

It’s perfectly reasonable to label and train for better responses to real world questions. What’s bad though is that the system doesn’t seem to actually be learning much, as demonstrated by the question about whether to use a bat or a match to start a fire - if the network can’t extrapolate trained answers onto slightly rephrased questions, it’s doing a bad job at understanding and learning.

kapildev 5y ago

I think that this is the right approach. As the machine has no way of knowing these things as it cant live in the real world, it relies on humans living in the real world that match is better suited than bat. And, it's perfectly reasonable that the AI had no way of knowing how different a box is -- so it has to rely on humans to tell it that box cant light itself on fire.

Animats 5y ago

The real insight from GPT-3 is that much of human discourse is rather dumb. It does so well at punditry that the wasteland of political commentary is all too obvious.

deleted 5y ago

[deleted]

xkcd-sucks 5y ago

If OpenAI is selling text based on a prompt, what does it matter if it is generated by a computer versus output of a "Chinese room" system? ...if the product they're selling is the company itself, or the model, then maybe there is something to it.

8bitsrule 5y ago

We humans tend to look for patterns in text, and tend to interpret them based on their experiences with other humans. Years ago I wrote a stupid simple program that responded to typed in remarks. Each typed remark would be added to a cache of remarks. The program then parsed the new remark into words. Then it looked through the cache to find an old remark with several strings in common with the new one (else a random one), and replied with it. (It also marked the old and new remarks as 'used' so it didn't repeat itself in a session.) So, any apparent intelligence was human. When people typed stuff in, they tended to do their best to interpret the reply as intelligent. They might puzzle until they thought they 'understood'. Several people got emotional; some of the regurgitated remarks were not so nice. They might then add their irritation to the cache! One day, with several people watching, it took quite a while but found no good replies to a question, and replied: "I like big tits." That was the end of the demos.

galaxyLogic 5y ago
Hilarious

jdrc 5y ago

Gpt3 is a bit of a cult now. Why do people believe there is intelligence in there?