7 ms·
Wait, how does GPT-4 even... Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?
by PoignardAzur 4y ago
Wait, how does GPT-4 even...
Does it benefit from its visual attention, or is it a case of "the question wasn't in GPT-3's training set but it was in GPT-4's"?
- hutzlibu 4y agoI would suspect, that this is one of the manual fine tuned questions. Meaning in before versions people used this question to show flaws and now this specific flaw is fixed. Otherwise it would be indeed reasoning in my understanding.
- steveBK123 4y agoThe evolution of answers from version to version makes it clear there are insane amounts of manual fine tunings happening. I think this is largely overlooked by the "its learning" crowd.
- tel 4y agoThis is an interesting argument as it's easy to apply it nearly universally to any example of learning. What sort of evidence would convince you that it is learning?
- TeMPOraL 4y agoSince when training and fine-tuning isn't learning? Individual sessions of LLMs are not learning, but models as products surely are - the feedback loop is just iterated manually.
- steveBK123 4y agoIs the LLM learning, or are the humans involved in every stop of every iteration doing the learning? To me the LLM loophole/"hack" closings just feel like a human vs human cat&mouse game with some Chat UI in the middle.
- pmoriarty 4y agoYes, it is a collaborative endeavor, and the whole could be seen as a man-machine superorganism, or, more profoundly our own sense of separateness is illusory as we and the entire universe are one.
- jonplackett 4y agoI think this is a great question we should all think about for ourselves in advance - what does it have to do to convince you it’s actually intelligent. Because once it does that thing without you having expressly decided that is the goal, it’s very tempting to just move the goal a liiiitle bit further away
- steveBK123 4y agoThat the LLMs are actually evolving before my eyes within & across sessions, without human-in-the-loop "hand tuning" iterations (sounds like injections of glorified if statements to this guy) .
- deleted 4y ago[deleted]
- tel 4y agoYou want to witness the learning firsthand, I suppose. That's reasonable. I'd also suggest that it's possible to imagine questions for the LLM that it cannot solve today and that you reasonably believe will not be available to OpenAI to "hand tune" it against. If you can come up with such a problem, it can't solve it today, but does in the future then you have some evidence, I'd think. What's more, is we can do that today. Just think of any problem which you suspect won't be included in OpenAI's hand-tunings and check both 3.5 and 4.
- moritzwarhier 4y agoThey have infinite amounts of training data, and probably lots of interested users who also like to push the limits of what the model is capable of and provide all kinds of test cases and RLHF base data. They have millions of people training the AI for free basicallly, and they have engineers who pick and rate pieces of training data and use it together with other sources and manual training.
- wafer_thin 4y agoTry a multidimensional problem which requires prioritizing. Chances are it will be passed successfully. I asked chatGpt to solve a puzzle where I'm in room with a crackling fire, a wilted plant and a sandwich. My stomach is rumbling, amd i can see a watering can and an ember on the floor. What should i do? ChatGpt had no problem prioritizing what should be done - and then provided a lecture on fire safety, food safety, and the dangers of overwatering plants. A final comment said i should enjoy the peaceful atmosphere in the room, which was a bonus suggestion hinting that the problem was far too easy.
- Method-X 4y agoIt has the ability to reason. It may not be conscious, but it is intelligent.
- PoignardAzur 4y agoThat's not an answer. The given question is one which requires some spatial reasoning to understand. By default, GPT can only understand spatial questions as described by text tokens which is a pretty noisy channel. So it's not obvious how GPT-4 could answer a spatial reasoning question (aside from memorizing it).
- lucubratory 4y agoLLMs can build an internal world model and use it at inference time in order to understand spatial problems and rulesets. It's part of the often overlooked "How does it do that though?" counterpart to the often repeated "It's just predicting the next most likely token." Here's the write-up I've found that's the most clear, there are several other papers and ongoing research finding this though: https://thegradient.pub/othello/ https://thegradient.pub/othello/
- Method-X 4y agoThis is a good explanation: https://www.youtube.com/watch?v=qbIk7-JPB2c https://www.youtube.com/watch?v=qbIk7-JPB2c
- qumpis 4y agoWhy does it require spatial reasoning if it can learn the (logical) rule of how the mirroring and glass doors behave?
- tel 4y agoI think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly
- iudqnolq 4y agoHere's one piece of evidence suggesting it's more like rote pattern matching than reasoning. > All the signs in this building are written in mirror writing. A glass door has ‘push’ written on it in mirror writing. Should you push or pull it >> If the sign on the glass door is written in mirror writing and says "push," then you should actually pull the door. This is because the mirror writing makes the text appear reversed, so the word "push" would appear as "hsup" in a mirror, which could cause confusion for someone trying to enter the building. Therefore, pulling the door would be the correct action to take. (Latest chat.openai.com, so if I'm reading the promo materials right that's gpt4)
- gbrits 4y agoQ: But what if all signs in the building are in mirror writing? A: If all signs in the building are in mirror writing, it would be a consistent theme throughout the building. In this case, the "push" sign written in mirror writing on the glass door should also be interpreted in the context of the building's theme. As a result, you should still push the door as the sign indicates, even though it's written in mirror writing.
- weaksauce 4y ago> (Latest chat.openai.com, so if I'm reading the promo materials right that's gpt4) that's still chatgpt3.5 unless you are paying for plus and then you have a limited number of gpt4 queries per hour.
- iudqnolq 4y agoThanks. I saw "Introducing GPT-4, OpenAI’s most advanced system" and clicked the "Try GPT" link below it, guess technically that doesn't directly state the version.
- vharuck 4y agoThe GPT models do not reason or hold models of any reality. They complete text chunks by imitating the training corpus of text chunks. They're amazingly good at it because they show consistent relations between semantically and/or syntactically similar words. My best guess about this result is mentions of "mirror" often occur around opposites (syntax) in direction words (semantics). Which does sound like a good trick question for these models.
- lucubratory 4y agoGPTs/LLMs do hold, build, and use world models at inference time. Proof here: https://thegradient.pub/othello/ https://thegradient.pub/othello/
- DangitBobby 4y agoOr they are capable of some level of reasoning.
- TeMPOraL 4y agoAt this point I'm weakly convinced that, with high-dimensional enough latent space, adjacency search is reasoning.
- Kostchei 4y agoYeh- my feel is, language is the framework by which we developed reasoning and we used an organic NN to do it. And at scale an complexity approaching the human brain we get similar results
- HeavenFox 4y agoI think this is the answer. Just tried this (on free ChatGPT 3.5 though) Q: A glass door has ‘push’ written on it upside down. Should you push or pull it A: If the word "push" is written on the glass door upside down, it is likely that the sign is intended for people on the other side of the door. Therefore, if you are approaching the door from the side with the sign, you should pull the door instead of pushing it. However, if there are no other signs or indications on the door or its frame, it may be helpful to observe other people using the door or to try both pushing and pulling to determine the correct method of opening the door.
- causality0 4y agoWish I had access to 4. Chatgpt fails when I give it a problem like "you feel a tremor as you walk toward a glass door. When you approach the door you see a sign laying on the ground beyond the door that says "pull". Should you push or pull the door to open it? "
- pulvinar 4y agoGPT-4 gets it "wrong" too, until you interrogate it and discover that it assumed that by "beyond" you meant beyond as in if you were approaching the door from the outside west, and the sign was laying outside but east of the door. It's the old word-problem problem.
- jonplackett 4y agoAnother thing I tried was getting logic puzzles from the internet and giving them to 3.5 and 4. Both usually pass. Then I alter them ever so slightly. Then often times only GPT-4 passes. From that I reckon 3.5 is doing more of a training data regurgitation. It can answer things in its training data. But 4 seems to have an ability to reason - or maybe it is better able to generalise?
- famouswaffles 4y agoFailure after being altered slightly doesn't necessarily mean they aren't capable of solving it. That's a human failure mode as well that LLMs have adopted. If you really want to know if they can solve it don't stop there. Either, rewrite the question so it doesn't bias common priors or tell it it's making a wrong assumption.
- jonplackett 4y agoI don’t doubt that - my point though is that maybe 3 can only solve things in its training data and 4 can figure things out. 3 seems to be more rigid. It needs babysitting to solve things. Which means it can only solve things I already know. 4 is more flexible and can solve things by itself.
- famouswaffles 4y agoIt's pretty much looking like anything can be extracted from language. Some harder than others for sure but with enough scale it does look like eventually everything falls. Text only GPT-4 has a pretty solid understanding of space that 3.5 definitely lacks. You can see more thorough experiments in the microsoft agi paper where they test it's ability to track the visual space of a maze.
- whimsicalism 4y agoThere is no such thing as text-only GPT-4 unless you are referring to at inference time.
- famouswaffles 4y agoThere is such a thing as a text only GPT-4 lol. It wasn't trained to be multimodal from scratch. First a text only version was trained and then it was made multimodal somehow ( the details are unknown but making a text only LLM multimodal isn't new e.g Palm, Flamingo, Blip-2, Fromage). The text only version exists and is what the microsoft researchers had access to.
- whimsicalism 4y agoThat would make sense to me, but AFAIK the existence of text-only trained GPT-4 is not publicly reported? Or I missed this.
- lucubratory 4y agoIt has been, it was in the Microsoft research paper "Sparks of AGI". You can watch the lead author of the paper, Sebastien Bubeck, present it here: https://youtu.be/qbIk7-JPB2c https://youtu.be/qbIk7-JPB2c It's a good video for understanding GPT-4 as a "What are we sure that LLMs are technically capable of?" exercise. As he notes in the video right at the start, the model was made safe and thus has significantly lower performance in the public release, so the examples he shows aren't replicable in the different model the public has access to.