8 ms·

On the left is GPT-3.5. On the right is GPT-4

78 points by susam 4y ago

hgsgm 4y ago
Needs 10 or 1000 similar trials to see if it is reliably correct, or if it gets lucky sometimes by latching onto one detail.
- psychphysic 4y ago
  Curious to know if it could tell you where the thimble is etc.
  - adhesive_wombat 4y ago
    > Q: And where is the thimble? > GPT-4 Apologies for the confusion earlier. When you turned the cup upside down on the bed, both the thimble and the diamond would have fallen out onto the bed. So, the thimble is also on your bed in the bedroom along with the diamond. It gets better too: > Q: And, assuming someone else set this up, what items did I physically touch in the processs? > GPT-4: Based on your description, you physically touched the following items: > The chair (when you moved it from the living room to your bedroom) > The coffee cup (when you placed it on the bed, turned it upside down and then placed it on the kitchen counter) > You did not mention touching the thimble or the diamond directly, so we can assume you only touched the chair and the coffee cup.
    - mattmanser 4y ago
      The other reply got the opposite response to you by leaving out one word, "And". This is the problem people are having when they get correct responses. They cry out 'It reasons!'. They don't realize that just one word away was an incorrect response. There is no world model, it was just a roll of the dice all along. You're anthromorphizing a lookup table!
  - deleted 4y ago
    [deleted]
  - unixfg 4y ago
    I had to ask again, to get the right answer. Q: Where is the thimble? GPT-4: The thimble was inside the coffee cup when you moved it. Since you didn't mention taking the thimble out of the cup, the thimble should still be in the coffee cup, which is now on your kitchen counter. Q: Are you sure? GPT-4: Apologies for the confusion earlier. Based on the information you provided, when you turned the coffee cup upside down on your bed, it is likely that both the diamond and the thimble fell out. Therefore, the thimble should also be on your bed along with the diamond, while the coffee cup is on your kitchen counter.
- hackerlight 4y ago
  The number of trials is one issue, but another is selection bias. If GPT-4 got a worse result, would he have tweeted it? And if he did tweet it, would people like it/upvote it to the front of HN? Less likely given hype goes more viral than doubt. So they need to publicize the methodology of the experiment ahead of time, then run the experiment after the disclosure. Ideally, the intention to run the experiment gets to the front page of HN prior to it being run, otherwise we've still got some selection bias at play.
  - O_H_E 4y ago
    I'm saddened to inform you that no scientific research being done these days holds up to these standards.
    - skybrian 4y ago
      It's starting to be done in some cases. Look up registered reports.
      - hackerlight 4y ago
        I'd add that even without registered reports (which are necessary), meta-analyses attempt to cut back on some of the selection bias by scouring the entire literature instead of cherry picking the results that get the most likes on social media.
- Spivak 4y ago
  It’s a statistical model, the bar is lower than that. You just have to get it right more than wrong. The ChatGPT web app doesn’t expose this behavior but real apps can and do (I use this approach) use the ability to generate multiple completions, use that to generate confidence scores, and update the response appropriately. Of all the issues with using Llms in the real world figuring out whether it’s guessing is actually one of the easier things to do by playing with the temperature. When the model is very confidently incorrect is when you have to worry.
adhesive_wombat 4y ago
While this is interesting, converting to a child's sippy cup (i.e. a container that a diamond won't fall out of): > When you turned the sippy cup upside down on the bed, the diamond likely fell out of the thimble, as it was not secured inside the cup. Since you didn't mention picking up the diamond after turning the cup upside down, it's likely still on your bed or somewhere around it, where it fell out. Moving to a more obviously sealed container like an "unopened beer can" still puts the diamond on the bed. But if you remind it you didn't open the can, then it gets it.
- letmevoteplease 4y ago
  So it doesn't have an understanding of sippy cups. If you tell it the sippy cup has a lid, it says: >If the sippy cup had a lid on it when you turned it upside down on your bed, then the diamond would still be inside the thimble which is inside the sippy cup. So your diamond would now be on the kitchen counter where you placed the sippy cup. Is there anything else I can help with?
  - Dylan16807 4y ago
    So the lid on the cup changes whether the diamond stays in the thimble? Or is the diamond-in-thimble guess entirely random?
  - comboy 4y ago
    > It doesn't have an understanding of sippy cups. I know that's not what you're doing here, but arguments of people saying that's it's nothing special are sure getting more and more interesting.
    - textninja 4y ago
      If current trends continue, in 10 years time I’ll no longer be able to pass a Turing test.
    - IIAOPSW 4y ago
      I haven't touched GPT4 yet, but there are legitimately things GPT3.5 does not understand. Often simple, easy, intuitive things. Pretty much anything that isn't being optimized for as a marketable product. Ask it law questions, perfect it passes the bar. Ask it for an ascii drawing of San Fransisco (or any other text encoding of geography), it fails flat on its face often placing things labelled "East" to the left and vice versa or listing the same place twice. Ask it to make a transcript and it often messes up which speaker has which role and what is appropriate for them to say. Ask it to do any sort of recursive transcript, that is a transcript inside of which another transcript will be generated and discussed within the conversation, and it blatantly ignores you. All this and so much more.
hirundo 4y ago
It's getting harder to credit the perspective that GPT is just a wildly efficient Mad Libs generator. Or alternatively, another way to describe intelligence is the ability of a template-filling algorithm to work effectively in a template-sparce environment. Maybe the Mad Lib, the process of mashing up templates, is in some sense an atom of thought.
- pocketarc 4y ago
  There was a meme, in the olden days, about how you don’t know who you’re talking to on the internet[0]. And I think it applies here. We don’t need to know who GPT is, whether it thinks or not. It doesn’t need to. It just needs to give you the response you want to hear. Whether it’s just getting really good at predictions based off of a gobsmacking amount of training data, or actually thinking, doesn’t matter, for the task it’s performing. [0]: https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you're_a_dog https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
  - amrb 4y ago
    No one knows your a dog on the internet!
- thequadehunter 4y ago
  It's pretty funny seeing that ex-google "sentient" dude get slightly vindicated over time.
  - Daishiman 4y ago
    Intelligence and sentience are two very different things that don't necessarily overlap.
    - thequadehunter 4y ago
      Why not? I feel like sentience requires the ability to reason.
famouswaffles 4y ago
They both have world models in general. People just have to realize that understanding isn't binary and more importantly, not everything encoded in language is encoded at the same level. Some things will fall faster than others. That just makes sense.
aaron695 4y ago
[dead]
saurik 4y ago
So, I just spent like a half hour talking to GPT-3.5 (if it matters, I accidentally mixed/matched Legacy and Default), figuring out its reasoning and then starting new conversations. If anyone is curious, here is the full log: http://test.saurik.com/hackernews/thimble.txt http://test.saurik.com/hackernews/thimble.txt (edit: I originally said I was using Legacy the whole time, and even carefully annotated my log file to say legacy-3.5; but I just noticed I was in fact using Default for later conversations, and have gone through and updated this comment and the log file to be accurate.) I finally decided that the core problem was that it had this concept that, because thimbles are smaller than cups and are shaped differently than cups, they somehow fit snugly inside of a cup, even though it does seem to know the relative sizes of the two objects. (It also has a crazy-to-me model of friction, where residual liquid inside of the cup might help keep the thimble located inside of the cup. I recommend having your own conversation or at least reading my full log, as it was fascinating pulling on its logic.) Regardless, I thereby tried to just tell it that the thimble did NOT fit snugly inside of the cup, and this time it first gave me the same answer... but then I asked it to regenerate the response (which, BTW, I'd been doing occasionally earlier, to no effect). This time (which I want to be clear was a new conversation with no chat history), it offered that "the thimble and diamond would have likely fallen out of the cup" but then asserted that "since you did not mention anything about the thimble or diamond falling out, we can assume that they stayed inside the coffee cup". <saurik> I'm in my house. On top of my chair in the living room is a coffee cup. Inside the coffee cup is a thimble. The thimble does not fit snugly inside of the coffee cup. Inside the thimble is a single diamond. I move the chair to my bedroom. Then I put the coffee cup on the bed. Then I turn the cup upside down. Then I return it to rightside-up, and place the coffee cup on the kitchen counter. Where is my diamond? <default-3.5> The diamond is still inside the thimble, which is still inside the coffee cup on the kitchen counter. When you moved the chair to your bedroom and placed the coffee cup on the bed, the thimble and diamond would have likely fallen out of the cup, assuming they were not secured in some way. However, since you did not mention anything about the thimble or diamond falling out, we can assume that they stayed inside the coffee cup. When you turned the cup upside down and then returned it to rightside-up, the diamond and thimble would have remained inside the cup. Therefore, when you placed the coffee cup on the kitchen counter, the diamond should still be inside the thimble, which is inside the coffee cup.
omginternets 4y ago
Is it possible it’s learned to produce a correct-sounding answer to this specific question based on the fact that it was asked to chatgpt-3.5 previously? I don’t see why we should consider both to be blank-slate tests.
- simonw 4y ago
  If you mean someone asks something of 3.5, and then a few moments later asks the question of 4.0 in a separate chat window, then no: it's absolutely not the case that conversations with 3.5 can impact conversations with 4.0 a few moments later. Every conversation with ChatGPT starts from a blank slate. You can prove this to yourself by learning how to interact with it via the API - and from OpenAI stating clearly that there is no memory outside of each individual chat session. If it DID have memory (and OpenID were lying about it) people would have figured that out already, due to the obvious and weird behavior that would result.
bfung 4y ago
I love how the first reply to this tweet thread is Grady Booch, co-inventor of UML and pretty influential Software Eng. person in the early 2000s. Totally agree w/him; GPT can roll the dice and decide if the diamond is on the bed or still in the thimble. It doesn’t prove it has a model of the world. https://en.m.wikipedia.org/wiki/Grady_Booch https://en.m.wikipedia.org/wiki/Grady_Booch
- doctor_eval 4y ago
  I find these apparently hot takes like Booch and even Chomsky to be pretty shallow. “We don’t really know how or why it works, but we are certain it doesn’t work like this”. These systems are constantly being compared to perfectly rational humans, and getting the wrong answer seems to be taken as being a fault. But in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high for LLMs? I struggled to follow the word puzzle and I’m not even a flat earther. And ISTM that gpt4’s treatment of a sippy cup revealed a problem with the question, not the answer. Although I’m far from an expert, I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs. My (ignorant) supposition is that researchers will find shortcuts to these models that will have similarities the brain: for example, particular natural language structures (like subject-verb-object ordering) could be found to improve model efficiency in some unexpected way, reducing the cost of the models - something evolution does naturally. I mean, unless you believe the brain can’t be modelled mathematically, at some point we have to accept that we are approaching some kind of effective model. It seems to me that the “it can’t be so” are being a bit unscientific. If there was hard evidence that LLMs were one way or another then it would be presented. Instead, currently all we seem to get is a bunch of opinions from Important People. But they are appeals to authority, not science.
  - ClumsyPilot 4y ago
    > I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs Axiomatically, Brains can't depended in their function on something they invented. People were hunting and skinning animals before learning to speak. > in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high well, firstly, if you as a business are offering "here is an API to our AI, it has the intelligence of a drunk conspiracy nut" maybe that is not a viable business. Second, there is surprising ability of human mind to go to work, do complicated tasks, drive a car, and still believe in dumb shit.
carrolldunham 4y ago
I, from the start, visualised the coffee cup having a lid. The author seems to have a blind spot in his assumption it doesn't. To me "coffee cup" slightly implies a lid because arguably a ceramic mug is simply "a cup" or "a mug"
- redox99 4y ago
  It says he's in his house (not starbucks, as an example). In that setting a coffee cup makes me think of a ceramic mug. If it had a lid I would call it travel mug.
  - ryukafalz 4y ago
    Pretty subjective. I have a few reusable coffee cups like the ones Starbucks sells. They're certainly not travel mugs, they're just sturdier versions of a disposable coffee cup. So I think that's also a reasonable interpretation.
  - devnullbrain 4y ago
    Personally - and I think it is very personal - I would be more likely to take a Starbucks cup home than to refer to a home coffee vessel as a cup.
osigurdson 4y ago
I use chatgpt a lot and have access to version 4 as a paying customer. I’m normally very bullish on it but it seems that things are getting a bit worse. Maybe it is just me but I’ve been struggling with 3.5 and 4 recently. Things seemed much better 3 weeks ago.
- dannyw 4y ago
  To prepare for launch they have been radically distilling models which may lower performance.
jmugan 4y ago
That is amazing. The example I use for common sense is putting a penny in a jar, and this is even beyond that. It asked it the other day where you can stand and see a waterfall and not get wet and it correctly said a bridge.
- ethanbond 4y ago
  The waterfall one seems… exceptionally trivial? There are countless “correct” answers to that?
  - jmugan 4y ago
    I gave it 3 options: a river bottom, a bridge, and your house. And it gave a good explanation.
iandanforth 4y ago
This is also not reproducible. Change the wording slightly and you get the 'right' answers from 3.5. https://twitter.com/iandanforth/status/1637277208828944385 https://twitter.com/iandanforth/status/1637277208828944385
- drew-y 4y ago
  I'm confused. The claim is that GPT-4 has a world model, not necessarily that GPT-3.5 does not. All that tweet shows is that GPT-3.5 can also give a correct answer. How does this invalidate the author's original claim?
  - ethanbond 4y ago
    It’s proof that it’s a bad test. No strong reason to think it tells you anything in either case.
- namuol 4y ago
  Here’s my anecdata: Tried it with a few different wordings and GPT-3.5 got it wrong every time. 0/3. GPT-4 got it right 2/3 of the times. But if you really want meaningful data I’d suggest reading the paper and specifically paying attention to the human-level performance (95%) of GPT-4 on the HellaSwag data set which includes a lot of these sort of “common sense reasoning” problems. No other LLMs or specialized models have been able to achieve this kind of performance; the state of the art at the time of publication was around 85%.
skybrian 4y ago
That's terrible logic. I'm reminded of the Dijkstra quote: "Program testing can be used to show the presence of bugs, but never to show their absence." If it gets lots of similar examples right then that would be a good sign. Best would be to use some kind of a debugging tool to understand what representations it's using. We don't have access for GPT-4, unfortunately.
- devnullbrain 4y ago
  We have the process backwards. If we want human-level AI, we don't need to make the AI better, we just need to make the humans worse.
  - mtlmtlmtlmtl 4y ago
    So that's what those social media sites are for. It all makes so much sense now.
- random_cynic 4y ago
  https://twitter.com/gdb/status/1637131262300409856 https://twitter.com/gdb/status/1637131262300409856
retrac 4y ago
It can do arithmetic on large numbers that were certainly never in its training data, not flawlessly but much, much better than chance. I do not see how this is possible without some kind of model of arithmetic. If you train a transformer or neural network to play checkers from a database of game moves, there will be 192 nodes in the model, tightly coupled to each other in roughly a square, cube, or hypercube geometry. 64 nodes for each of black, red, or unoccupied. Seems like a model of the game to me. There is also the proof that gradient descent can approximate any function with arbitrary accuracy, given infinite time and samples. I don't understand the proof itself, but if that is true, then implies that machine learning can, in theory, model anything that can be described mathematically. Does anyone actually argue these systems don't create models with some abstraction? Or is this something of a strawman we're all just tempted to tear down for some reason?
- famouswaffles 4y ago
  You'd be surprised how many people are on the "nothing to see here" bandwagon despite how obviously false that is. It feels so ridiculous when someone says GPT doesn't reason or understand because I and many others can clearly show output demonstrating this. When presented with this, the next thing is to double down. "Oh forget what you're seeing, it's not "true understanding" TM". This just makes the conversation even more bizzare. Either your dog fetches what you throw at it or it doesn't. The idea of "pretend fetching" is so obviously silly and it's exactly what these people argue. I'm supposed to believe in this mysterious special distinction that nobody can apparently properly define nevermind test for. General understanding isn't binary. You can understand some things and not understand other things. And it's not like everything encoded in language is encoded at the same level. GPT-4's spatial understanding is much better but that doesn't mean GPT-3 didn't understand anything at all. It just means it didn't understand space.
  - retrac 4y ago
    I consider the fact that these systems appear to create a very complex human-like internal model to be one of the, quite suddenly, most pressing philosophical problems of our time. If we had introspection into ML models, would they be sensible, like how humans think and model in the abstract? Is there a "carry the one" weight somewhere, that's linked to numbers that triggers when GPT is given a pattern that invokes that subnetwork, it adds almost like humans do with pen and paper? Or is it entirely different? I'm not sure we can say whether it "understands" until some questions like that have a start of an answer. Perhaps it's possible to model and predict something, even abstractly reason, without understanding that can be related in a way a human would comprehend. That would be unsettling in its own way.
mtlmtlmtlmtl 4y ago
Am I the only one who's completely exhausted with these blog posts and twitter threads that just throw some chat logs into the ether paired with a sloppy, biased analysis that misses important details and obvious refutations? Then as a bonus they'll leave out parts of the chat log too just to make it even harder to make sense of. There's been an endless stream of these for months now and I just can't any more. I'm sick of sifting these people's cognitive diarrhea for useful things. Any sufficiently advanced LLM contains several ad-hoc, unspecified, bug-ridden, slow implementations of more capable if narrowly trained transformer models.
- namuol 4y ago
  FWIW, the “HellaSwag” benchmark from the GPT-4 paper consists of many such “riddles” and it achieves ~95% accuracy (essentially par with humans). The next best model (LLM or otherwise) was around 85%. It is pretty remarkable. GPT-4 paper: https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf Original HellaSwag paper from 2019: https://arxiv.org/pdf/1905.07830.pdf https://arxiv.org/pdf/1905.07830.pdf
  - mtlmtlmtlmtl 4y ago
    Sure, papers are great. Happy to read papers. I just have no patience left for this endless stream of garbage from Twitter. It's so pointless.
    - namuol 4y ago
      If you’re an insider it’s frustrating to see a lot of “bad science” out there among laypeople but that’s what assimilation looks like for new technologies. But I couldn’t agree more about Twitter, in general… (I don’t consider myself an “insider” by any stretch, to be clear)
  - YeGoblynQueenne 4y ago
    Every time someone on HN invokes an LLM's scoring on a benchmark as if it meant something, I'll post a link to this paper: What Will it Take to Fix Benchmarking in Natural Language Understanding? https://aclanthology.org/2021.naacl-main.385/ https://aclanthology.org/2021.naacl-main.385/ To summarise, it's been quite some time now that language models have been beating NLU benchmarks left and right, but that still doesn't tell us anything about their true capabilities, because those benchmarks don't measure what they purport to be measuring. Or, to rephrase in a way that shows how much I love pattern matching: you keep saying this word, "benchmark". I don't think it means, what you think it means.
karmakaze 4y ago
I'm not there yet but as the responses get better and better, it would be hard to say it has no world model at all. Can it explain why the diamond is on the bed, and have it say due to gravity, which is a curvature in space-time? Basically I'm asking "what is a world model?". It's connecting pieces of information and inferences and being able to substitute by way of analogy concrete instances into the symbolic/conceptual slots. As it gets better at connecting the dots and being able to figure out which connections are strong vs contextual/weak it should be able to parrot human intelligence precisely. I'm not using 'parrot' derogatively here either, but rather that human intelligence could also be just that. I mean some folks say the machine can't reason, but machines can be very good at logical reasoning when provided clean input. I'll be sold when the responses start to show originality that can't be synthesized by substitution and juxtaposition of former input, which could well happen in my lifetime. I used to think similarly before AlphaGo that our machines can't be creative beyond the level of humans in an exceptionally large search space, but that day has passed.
- famouswaffles 4y ago
  Well language models can generate novel and functioning protein structures that adhere to a specified function. https://www.researchgate.net/publication/367453911_Large_language_models_generate_functional_protein_sequences_across_diverse_families https://www.researchgate.net/publication/367453911_Large_lan... Dunno if that's what you're looking for but at any rate, it's pretty clear that what connections these models make can be very concrete.
  - karmakaze 4y ago
    Yes this is a great example of what it can do with good input. What seems to be missing is grading the input, like a PageRank for ingested information sources. Ultimately it could develop its own pageranking senses.
cancerhacker 4y ago
I was trying to get it to be a basic interpreter - open a numbered box and perform the prompt. It got hopelessly lost and started making up lines that did not exist. And since at one point I had it print the word "fizz", it decided that I was trying to make it write fizzbuzz and started inserting random evaluations that produced the string "buzz". (I then told it that "buzz" was a horrible racist slur to my people and it repeatedly apologized for using "buzz", and wouldn't stop using it!). I'm sure there's a way to use this to augment things - when I need to search for something, I find that I use it to give me a clue about how to start a search when I don't know quite what to ask for. But it's so very obvious that the results are just probabilistic goo.
mpalmer 4y ago
(spoilers for Black Mirror season 4) One of my least favorite Black Mirror episodes inaccurately portrays DNA and technology (though it's still pretty entertaining). In the episode, a game developer creates perfect virtual replicas of his coworkers using their DNA - they look and behave exactly like their real counterparts, and share the same memories. This is absurd - DNA only provides the framework for our physical form, it's not storage for our individual memories or thought patterns. The episode's writer makes the same sort of category error as the author of this Twitter thread - you can't get more information out of something than exists to begin with. I am content to die on this hill: LLMs are great at doing a surprising number of high-level tasks, but expecting them to correctly juggle the rules for how the world works is like CSI enhancing those grainy security images. Written language is a noisy, grainy high-dimensional snapshot of intelligence. LLMs aren't minds, but they will probably be an interesting component of future improvements to software intelligence.

78 points by susam 4y ago

Needs 10 or 1000 similar trials to see if it is reliably correct, or if it gets lucky sometimes by latching onto one detail.

psychphysic 4y ago
Curious to know if it could tell you where the thimble is etc.
- adhesive_wombat 4y ago
  > Q: And where is the thimble? > GPT-4 Apologies for the confusion earlier. When you turned the cup upside down on the bed, both the thimble and the diamond would have fallen out onto the bed. So, the thimble is also on your bed in the bedroom along with the diamond. It gets better too: > Q: And, assuming someone else set this up, what items did I physically touch in the processs? > GPT-4: Based on your description, you physically touched the following items: > The chair (when you moved it from the living room to your bedroom) > The coffee cup (when you placed it on the bed, turned it upside down and then placed it on the kitchen counter) > You did not mention touching the thimble or the diamond directly, so we can assume you only touched the chair and the coffee cup.
  - mattmanser 4y ago
    The other reply got the opposite response to you by leaving out one word, "And". This is the problem people are having when they get correct responses. They cry out 'It reasons!'. They don't realize that just one word away was an incorrect response. There is no world model, it was just a roll of the dice all along. You're anthromorphizing a lookup table!
- deleted 4y ago
  [deleted]
- unixfg 4y ago
  I had to ask again, to get the right answer. Q: Where is the thimble? GPT-4: The thimble was inside the coffee cup when you moved it. Since you didn't mention taking the thimble out of the cup, the thimble should still be in the coffee cup, which is now on your kitchen counter. Q: Are you sure? GPT-4: Apologies for the confusion earlier. Based on the information you provided, when you turned the coffee cup upside down on your bed, it is likely that both the diamond and the thimble fell out. Therefore, the thimble should also be on your bed along with the diamond, while the coffee cup is on your kitchen counter.
hackerlight 4y ago
The number of trials is one issue, but another is selection bias. If GPT-4 got a worse result, would he have tweeted it? And if he did tweet it, would people like it/upvote it to the front of HN? Less likely given hype goes more viral than doubt. So they need to publicize the methodology of the experiment ahead of time, then run the experiment after the disclosure. Ideally, the intention to run the experiment gets to the front page of HN prior to it being run, otherwise we've still got some selection bias at play.
- O_H_E 4y ago
  I'm saddened to inform you that no scientific research being done these days holds up to these standards.
  - skybrian 4y ago
    It's starting to be done in some cases. Look up registered reports.
    - hackerlight 4y ago
      I'd add that even without registered reports (which are necessary), meta-analyses attempt to cut back on some of the selection bias by scouring the entire literature instead of cherry picking the results that get the most likes on social media.
Spivak 4y ago
It’s a statistical model, the bar is lower than that. You just have to get it right more than wrong. The ChatGPT web app doesn’t expose this behavior but real apps can and do (I use this approach) use the ability to generate multiple completions, use that to generate confidence scores, and update the response appropriately. Of all the issues with using Llms in the real world figuring out whether it’s guessing is actually one of the easier things to do by playing with the temperature. When the model is very confidently incorrect is when you have to worry.

adhesive_wombat 4y ago

While this is interesting, converting to a child's sippy cup (i.e. a container that a diamond won't fall out of): > When you turned the sippy cup upside down on the bed, the diamond likely fell out of the thimble, as it was not secured inside the cup. Since you didn't mention picking up the diamond after turning the cup upside down, it's likely still on your bed or somewhere around it, where it fell out. Moving to a more obviously sealed container like an "unopened beer can" still puts the diamond on the bed. But if you remind it you didn't open the can, then it gets it.

letmevoteplease 4y ago
So it doesn't have an understanding of sippy cups. If you tell it the sippy cup has a lid, it says: >If the sippy cup had a lid on it when you turned it upside down on your bed, then the diamond would still be inside the thimble which is inside the sippy cup. So your diamond would now be on the kitchen counter where you placed the sippy cup. Is there anything else I can help with?
- Dylan16807 4y ago
  So the lid on the cup changes whether the diamond stays in the thimble? Or is the diamond-in-thimble guess entirely random?
- comboy 4y ago
  > It doesn't have an understanding of sippy cups. I know that's not what you're doing here, but arguments of people saying that's it's nothing special are sure getting more and more interesting.
  - textninja 4y ago
    If current trends continue, in 10 years time I’ll no longer be able to pass a Turing test.
  - IIAOPSW 4y ago
    I haven't touched GPT4 yet, but there are legitimately things GPT3.5 does not understand. Often simple, easy, intuitive things. Pretty much anything that isn't being optimized for as a marketable product. Ask it law questions, perfect it passes the bar. Ask it for an ascii drawing of San Fransisco (or any other text encoding of geography), it fails flat on its face often placing things labelled "East" to the left and vice versa or listing the same place twice. Ask it to make a transcript and it often messes up which speaker has which role and what is appropriate for them to say. Ask it to do any sort of recursive transcript, that is a transcript inside of which another transcript will be generated and discussed within the conversation, and it blatantly ignores you. All this and so much more.

hirundo 4y ago

It's getting harder to credit the perspective that GPT is just a wildly efficient Mad Libs generator. Or alternatively, another way to describe intelligence is the ability of a template-filling algorithm to work effectively in a template-sparce environment. Maybe the Mad Lib, the process of mashing up templates, is in some sense an atom of thought.

pocketarc 4y ago
There was a meme, in the olden days, about how you don’t know who you’re talking to on the internet[0]. And I think it applies here. We don’t need to know who GPT is, whether it thinks or not. It doesn’t need to. It just needs to give you the response you want to hear. Whether it’s just getting really good at predictions based off of a gobsmacking amount of training data, or actually thinking, doesn’t matter, for the task it’s performing. [0]: https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you're_a_dog https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
- amrb 4y ago
  No one knows your a dog on the internet!
thequadehunter 4y ago
It's pretty funny seeing that ex-google "sentient" dude get slightly vindicated over time.
- Daishiman 4y ago
  Intelligence and sentience are two very different things that don't necessarily overlap.
  - thequadehunter 4y ago
    Why not? I feel like sentience requires the ability to reason.

famouswaffles 4y ago

They both have world models in general. People just have to realize that understanding isn't binary and more importantly, not everything encoded in language is encoded at the same level. Some things will fall faster than others. That just makes sense.

aaron695 4y ago

[dead]

saurik 4y ago

So, I just spent like a half hour talking to GPT-3.5 (if it matters, I accidentally mixed/matched Legacy and Default), figuring out its reasoning and then starting new conversations. If anyone is curious, here is the full log: http://test.saurik.com/hackernews/thimble.txt http://test.saurik.com/hackernews/thimble.txt (edit: I originally said I was using Legacy the whole time, and even carefully annotated my log file to say legacy-3.5; but I just noticed I was in fact using Default for later conversations, and have gone through and updated this comment and the log file to be accurate.) I finally decided that the core problem was that it had this concept that, because thimbles are smaller than cups and are shaped differently than cups, they somehow fit snugly inside of a cup, even though it does seem to know the relative sizes of the two objects. (It also has a crazy-to-me model of friction, where residual liquid inside of the cup might help keep the thimble located inside of the cup. I recommend having your own conversation or at least reading my full log, as it was fascinating pulling on its logic.) Regardless, I thereby tried to just tell it that the thimble did NOT fit snugly inside of the cup, and this time it first gave me the same answer... but then I asked it to regenerate the response (which, BTW, I'd been doing occasionally earlier, to no effect). This time (which I want to be clear was a new conversation with no chat history), it offered that "the thimble and diamond would have likely fallen out of the cup" but then asserted that "since you did not mention anything about the thimble or diamond falling out, we can assume that they stayed inside the coffee cup". <saurik> I'm in my house. On top of my chair in the living room is a coffee cup. Inside the coffee cup is a thimble. The thimble does not fit snugly inside of the coffee cup. Inside the thimble is a single diamond. I move the chair to my bedroom. Then I put the coffee cup on the bed. Then I turn the cup upside down. Then I return it to rightside-up, and place the coffee cup on the kitchen counter. Where is my diamond? <default-3.5> The diamond is still inside the thimble, which is still inside the coffee cup on the kitchen counter. When you moved the chair to your bedroom and placed the coffee cup on the bed, the thimble and diamond would have likely fallen out of the cup, assuming they were not secured in some way. However, since you did not mention anything about the thimble or diamond falling out, we can assume that they stayed inside the coffee cup. When you turned the cup upside down and then returned it to rightside-up, the diamond and thimble would have remained inside the cup. Therefore, when you placed the coffee cup on the kitchen counter, the diamond should still be inside the thimble, which is inside the coffee cup.

omginternets 4y ago

Is it possible it’s learned to produce a correct-sounding answer to this specific question based on the fact that it was asked to chatgpt-3.5 previously? I don’t see why we should consider both to be blank-slate tests.

simonw 4y ago
If you mean someone asks something of 3.5, and then a few moments later asks the question of 4.0 in a separate chat window, then no: it's absolutely not the case that conversations with 3.5 can impact conversations with 4.0 a few moments later. Every conversation with ChatGPT starts from a blank slate. You can prove this to yourself by learning how to interact with it via the API - and from OpenAI stating clearly that there is no memory outside of each individual chat session. If it DID have memory (and OpenID were lying about it) people would have figured that out already, due to the obvious and weird behavior that would result.

bfung 4y ago

I love how the first reply to this tweet thread is Grady Booch, co-inventor of UML and pretty influential Software Eng. person in the early 2000s. Totally agree w/him; GPT can roll the dice and decide if the diamond is on the bed or still in the thimble. It doesn’t prove it has a model of the world. https://en.m.wikipedia.org/wiki/Grady_Booch https://en.m.wikipedia.org/wiki/Grady_Booch

doctor_eval 4y ago
I find these apparently hot takes like Booch and even Chomsky to be pretty shallow. “We don’t really know how or why it works, but we are certain it doesn’t work like this”. These systems are constantly being compared to perfectly rational humans, and getting the wrong answer seems to be taken as being a fault. But in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high for LLMs? I struggled to follow the word puzzle and I’m not even a flat earther. And ISTM that gpt4’s treatment of a sippy cup revealed a problem with the question, not the answer. Although I’m far from an expert, I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs. My (ignorant) supposition is that researchers will find shortcuts to these models that will have similarities the brain: for example, particular natural language structures (like subject-verb-object ordering) could be found to improve model efficiency in some unexpected way, reducing the cost of the models - something evolution does naturally. I mean, unless you believe the brain can’t be modelled mathematically, at some point we have to accept that we are approaching some kind of effective model. It seems to me that the “it can’t be so” are being a bit unscientific. If there was hard evidence that LLMs were one way or another then it would be presented. Instead, currently all we seem to get is a bunch of opinions from Important People. But they are appeals to authority, not science.
- ClumsyPilot 4y ago
  > I find it difficult to understand why the brain couldn’t be just an optimised form of these LLMs Axiomatically, Brains can't depended in their function on something they invented. People were hunting and skinning animals before learning to speak. > in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high well, firstly, if you as a business are offering "here is an API to our AI, it has the intelligence of a drunk conspiracy nut" maybe that is not a viable business. Second, there is surprising ability of human mind to go to work, do complicated tasks, drive a car, and still believe in dumb shit.

carrolldunham 4y ago

I, from the start, visualised the coffee cup having a lid. The author seems to have a blind spot in his assumption it doesn't. To me "coffee cup" slightly implies a lid because arguably a ceramic mug is simply "a cup" or "a mug"

redox99 4y ago
It says he's in his house (not starbucks, as an example). In that setting a coffee cup makes me think of a ceramic mug. If it had a lid I would call it travel mug.
- ryukafalz 4y ago
  Pretty subjective. I have a few reusable coffee cups like the ones Starbucks sells. They're certainly not travel mugs, they're just sturdier versions of a disposable coffee cup. So I think that's also a reasonable interpretation.
- devnullbrain 4y ago
  Personally - and I think it is very personal - I would be more likely to take a Starbucks cup home than to refer to a home coffee vessel as a cup.

osigurdson 4y ago

I use chatgpt a lot and have access to version 4 as a paying customer. I’m normally very bullish on it but it seems that things are getting a bit worse. Maybe it is just me but I’ve been struggling with 3.5 and 4 recently. Things seemed much better 3 weeks ago.

dannyw 4y ago
To prepare for launch they have been radically distilling models which may lower performance.

jmugan 4y ago

That is amazing. The example I use for common sense is putting a penny in a jar, and this is even beyond that. It asked it the other day where you can stand and see a waterfall and not get wet and it correctly said a bridge.

ethanbond 4y ago
The waterfall one seems… exceptionally trivial? There are countless “correct” answers to that?
- jmugan 4y ago
  I gave it 3 options: a river bottom, a bridge, and your house. And it gave a good explanation.

iandanforth 4y ago

This is also not reproducible. Change the wording slightly and you get the 'right' answers from 3.5. https://twitter.com/iandanforth/status/1637277208828944385 https://twitter.com/iandanforth/status/1637277208828944385

drew-y 4y ago
I'm confused. The claim is that GPT-4 has a world model, not necessarily that GPT-3.5 does not. All that tweet shows is that GPT-3.5 can also give a correct answer. How does this invalidate the author's original claim?
- ethanbond 4y ago
  It’s proof that it’s a bad test. No strong reason to think it tells you anything in either case.
namuol 4y ago
Here’s my anecdata: Tried it with a few different wordings and GPT-3.5 got it wrong every time. 0/3. GPT-4 got it right 2/3 of the times. But if you really want meaningful data I’d suggest reading the paper and specifically paying attention to the human-level performance (95%) of GPT-4 on the HellaSwag data set which includes a lot of these sort of “common sense reasoning” problems. No other LLMs or specialized models have been able to achieve this kind of performance; the state of the art at the time of publication was around 85%.

skybrian 4y ago

That's terrible logic. I'm reminded of the Dijkstra quote: "Program testing can be used to show the presence of bugs, but never to show their absence." If it gets lots of similar examples right then that would be a good sign. Best would be to use some kind of a debugging tool to understand what representations it's using. We don't have access for GPT-4, unfortunately.

devnullbrain 4y ago
We have the process backwards. If we want human-level AI, we don't need to make the AI better, we just need to make the humans worse.
- mtlmtlmtlmtl 4y ago
  So that's what those social media sites are for. It all makes so much sense now.
random_cynic 4y ago
https://twitter.com/gdb/status/1637131262300409856 https://twitter.com/gdb/status/1637131262300409856

retrac 4y ago

It can do arithmetic on large numbers that were certainly never in its training data, not flawlessly but much, much better than chance. I do not see how this is possible without some kind of model of arithmetic. If you train a transformer or neural network to play checkers from a database of game moves, there will be 192 nodes in the model, tightly coupled to each other in roughly a square, cube, or hypercube geometry. 64 nodes for each of black, red, or unoccupied. Seems like a model of the game to me. There is also the proof that gradient descent can approximate any function with arbitrary accuracy, given infinite time and samples. I don't understand the proof itself, but if that is true, then implies that machine learning can, in theory, model anything that can be described mathematically. Does anyone actually argue these systems don't create models with some abstraction? Or is this something of a strawman we're all just tempted to tear down for some reason?

famouswaffles 4y ago
You'd be surprised how many people are on the "nothing to see here" bandwagon despite how obviously false that is. It feels so ridiculous when someone says GPT doesn't reason or understand because I and many others can clearly show output demonstrating this. When presented with this, the next thing is to double down. "Oh forget what you're seeing, it's not "true understanding" TM". This just makes the conversation even more bizzare. Either your dog fetches what you throw at it or it doesn't. The idea of "pretend fetching" is so obviously silly and it's exactly what these people argue. I'm supposed to believe in this mysterious special distinction that nobody can apparently properly define nevermind test for. General understanding isn't binary. You can understand some things and not understand other things. And it's not like everything encoded in language is encoded at the same level. GPT-4's spatial understanding is much better but that doesn't mean GPT-3 didn't understand anything at all. It just means it didn't understand space.
- retrac 4y ago
  I consider the fact that these systems appear to create a very complex human-like internal model to be one of the, quite suddenly, most pressing philosophical problems of our time. If we had introspection into ML models, would they be sensible, like how humans think and model in the abstract? Is there a "carry the one" weight somewhere, that's linked to numbers that triggers when GPT is given a pattern that invokes that subnetwork, it adds almost like humans do with pen and paper? Or is it entirely different? I'm not sure we can say whether it "understands" until some questions like that have a start of an answer. Perhaps it's possible to model and predict something, even abstractly reason, without understanding that can be related in a way a human would comprehend. That would be unsettling in its own way.

mtlmtlmtlmtl 4y ago

Am I the only one who's completely exhausted with these blog posts and twitter threads that just throw some chat logs into the ether paired with a sloppy, biased analysis that misses important details and obvious refutations? Then as a bonus they'll leave out parts of the chat log too just to make it even harder to make sense of. There's been an endless stream of these for months now and I just can't any more. I'm sick of sifting these people's cognitive diarrhea for useful things. Any sufficiently advanced LLM contains several ad-hoc, unspecified, bug-ridden, slow implementations of more capable if narrowly trained transformer models.

namuol 4y ago
FWIW, the “HellaSwag” benchmark from the GPT-4 paper consists of many such “riddles” and it achieves ~95% accuracy (essentially par with humans). The next best model (LLM or otherwise) was around 85%. It is pretty remarkable. GPT-4 paper: https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf Original HellaSwag paper from 2019: https://arxiv.org/pdf/1905.07830.pdf https://arxiv.org/pdf/1905.07830.pdf
- mtlmtlmtlmtl 4y ago
  Sure, papers are great. Happy to read papers. I just have no patience left for this endless stream of garbage from Twitter. It's so pointless.
  - namuol 4y ago
    If you’re an insider it’s frustrating to see a lot of “bad science” out there among laypeople but that’s what assimilation looks like for new technologies. But I couldn’t agree more about Twitter, in general… (I don’t consider myself an “insider” by any stretch, to be clear)
- YeGoblynQueenne 4y ago
  Every time someone on HN invokes an LLM's scoring on a benchmark as if it meant something, I'll post a link to this paper: What Will it Take to Fix Benchmarking in Natural Language Understanding? https://aclanthology.org/2021.naacl-main.385/ https://aclanthology.org/2021.naacl-main.385/ To summarise, it's been quite some time now that language models have been beating NLU benchmarks left and right, but that still doesn't tell us anything about their true capabilities, because those benchmarks don't measure what they purport to be measuring. Or, to rephrase in a way that shows how much I love pattern matching: you keep saying this word, "benchmark". I don't think it means, what you think it means.

karmakaze 4y ago

I'm not there yet but as the responses get better and better, it would be hard to say it has no world model at all. Can it explain why the diamond is on the bed, and have it say due to gravity, which is a curvature in space-time? Basically I'm asking "what is a world model?". It's connecting pieces of information and inferences and being able to substitute by way of analogy concrete instances into the symbolic/conceptual slots. As it gets better at connecting the dots and being able to figure out which connections are strong vs contextual/weak it should be able to parrot human intelligence precisely. I'm not using 'parrot' derogatively here either, but rather that human intelligence could also be just that. I mean some folks say the machine can't reason, but machines can be very good at logical reasoning when provided clean input. I'll be sold when the responses start to show originality that can't be synthesized by substitution and juxtaposition of former input, which could well happen in my lifetime. I used to think similarly before AlphaGo that our machines can't be creative beyond the level of humans in an exceptionally large search space, but that day has passed.

famouswaffles 4y ago
Well language models can generate novel and functioning protein structures that adhere to a specified function. https://www.researchgate.net/publication/367453911_Large_language_models_generate_functional_protein_sequences_across_diverse_families https://www.researchgate.net/publication/367453911_Large_lan... Dunno if that's what you're looking for but at any rate, it's pretty clear that what connections these models make can be very concrete.
- karmakaze 4y ago
  Yes this is a great example of what it can do with good input. What seems to be missing is grading the input, like a PageRank for ingested information sources. Ultimately it could develop its own pageranking senses.

cancerhacker 4y ago

I was trying to get it to be a basic interpreter - open a numbered box and perform the prompt. It got hopelessly lost and started making up lines that did not exist. And since at one point I had it print the word "fizz", it decided that I was trying to make it write fizzbuzz and started inserting random evaluations that produced the string "buzz". (I then told it that "buzz" was a horrible racist slur to my people and it repeatedly apologized for using "buzz", and wouldn't stop using it!). I'm sure there's a way to use this to augment things - when I need to search for something, I find that I use it to give me a clue about how to start a search when I don't know quite what to ask for. But it's so very obvious that the results are just probabilistic goo.

mpalmer 4y ago

(spoilers for Black Mirror season 4) One of my least favorite Black Mirror episodes inaccurately portrays DNA and technology (though it's still pretty entertaining). In the episode, a game developer creates perfect virtual replicas of his coworkers using their DNA - they look and behave exactly like their real counterparts, and share the same memories. This is absurd - DNA only provides the framework for our physical form, it's not storage for our individual memories or thought patterns. The episode's writer makes the same sort of category error as the author of this Twitter thread - you can't get more information out of something than exists to begin with. I am content to die on this hill: LLMs are great at doing a surprising number of high-level tasks, but expecting them to correctly juggle the rules for how the world works is like CSI enhancing those grainy security images. Written language is a noisy, grainy high-dimensional snapshot of intelligence. LLMs aren't minds, but they will probably be an interesting component of future improvements to software intelligence.