6 ms·

Isn't GPT essentially tool for rephrasing what it finds on the Internet, it doesn't really think?

• jazzyjackson 4y ago
  It is a internet-commenter-simulator, exactly what the world needs right now /s
• creatonez 4y ago
  No. It is modelling the various text generation processes that lead to the contents of the internet. Some of that modelling could absolutely involve "thinking", for processes that involve human thinking.
  • antoineMoPa 4y ago
    Have any source to demonstrate that claim?
    • Jeff_Brown 4y ago
      Doesn't that "It is modelling the various text generation processes that lead to the contents of the internet," seems tautologically true? (I'm not the person who wrote the grandparent of the present comment.)
    • PeterisP 4y ago
      The Othello-GPT experiment (https://thegradient.pub/othello/ https://thegradient.pub/othello/) probably still is the most relevant argument about these models' capabilities of building an internal world model.
    • creatonez 4y ago
      It's self-evident that GPT is a world-modeller, at least within the confines of the text boundary. It's able to come up with novel ideas seen nowhere in the training data, combinations that demonstrate there is a world concept web and not just a text probability web. It may not "understand" much of the hallucination nonsense it spits out, but there absolutely are moments where it "understands". See the Rome example on this page: https://oneusefulthing.substack.com/p/feats-to-astonish-and-amaze https://oneusefulthing.substack.com/p/feats-to-astonish-and-... This is essentially a completely novel answer to an /r/AskHistorians style question, which I would consider one of the most difficult types of internet text to model, in terms of the amount of understanding and concept webs you need to tie together Here's another example of GPT-4 doing non-trivial world modelling: How would three philosophers review the TV show Severence? https://i.imgur.com/FBi31Qw.png https://i.imgur.com/FBi31Qw.png
• vsareto 4y ago
  It can do some thinking. You can give it instructions to modify a piece of code that definitely isn't on the internet with several steps and it attempts to follow instructions, which, for a human, requires formulating what steps to take. The prompts have to read like good written requirements for something, so they have some degree of specificity. But the fact that it can follow instructions and carry them out almost certainly could be considered some form of thinking, especially on novel text not on the internet.
  • ChatGTP 4y ago
    Uh, it’s not following instructions, you’re altering you’re prompts until it finds the combination you expect to see…