4 ms·

These "it solved X" posts leave me wondering what I'm looking at. Possibilities are: - GPT-4 has never seen Wordle before getting the prompt. If so, I welcom

These "it solved X" posts leave me wondering what I'm looking at. Possibilities are: - GPT-4 has never seen Wordle before getting the prompt. If so, I welcome our new AI overloads. - GPT-4 has seen lots of references to Wordle on the web. This is undoubtedly true for games like Chess, but Wordle was released late 2021. It's still very impressive that it can play a mediocre game a chess just by looking at all the games on the web. - OpenAI anticipated the question, and did some Wordle training just to precipitate a tweet like this. If so - not so impressive as we have been building AI's that some games better than us for some time now. I'm betting it's the last one, but I'd sure like to know if it was one of the others.

DaiPlusPlus 4y ago
It’s the first one. GPT is scary.
SomewhatLikely 4y ago
Wordle is a kind of modified hangman, there's probably lots of data on the web for anagrams and hangman-like games.
- sdenton4 4y ago
  Why is Gpt4 better at these tasks than gpt3 if it only comes down to training data?
- ChatGTP 4y ago
  Likely a lot of "implement a hangman" code. I'm starting to wonder if a large part of GH might be in the training data, hence the "we don't want to release the training data" stance. Bit of a conspiracy maybe, but is it impossible?
  - arlcode 4y ago
    Microsoft is betting really hard on AI. Yesterday I noticed that they integrated a link to "chat" into the start/search Menu of windows 11. I would not at all be surprised if they "sampled" large parts of private GH repositories, paying customers be damned. Probably covered with a corporate fig leave of "tokenizing the data" and hiding behind "to improve our services".
ftxbro 4y ago
It's absolutely the first one. The series of publications related to the GPT family of models show a steady history of unlocking new capabilities just by reducing token perplexity. In the limited info provided by the GPT-4 press release pdf they gave the example of 'Hindsight Neglect' as one that was automatically unlocked in the move from 3.5 to 4.0. I guess wordle is another one!
- ChatGTP 4y ago
  I don't know how helpful this is to keep the back and fourth going on what ChatGPT can and can't do, but I found this on Wikipedia[1]: Wordle is a web-based word game created and developed by Welsh software engineer Josh Wardle. It was bought by the New York Times Company in 2022 who has since maintained and published it. Players have six attempts to guess a five-letter word, with feedback given for each guess in the form of colored tiles indicating when letters match or occupy the correct position. The mechanics are nearly identical to the 1955 pen-and-paper game Jotto and the television game show franchise Lingo. Wordle has a single daily solution, with all players attempting to guess the same word. Wardle created the game for himself and his partner to play, eventually making it public in October 2021. The game gained popularity in December 2021 after Wardle added the ability for players to copy their daily results as emoji squares, which were widely shared on Twitter. Many clones and variations of the game were also created, as were versions in languages besides English. The game was purchased by The New York Times Company in January 2022 for an undisclosed seven-figure sum, with plans to keep it free for all players; it was moved to the company's website in February 2022.* I'm not downplaying this but I mean, if there have been games with nearly identical mechanics since 1955...well you can extrapolate from there. I'm now starting to get the vibe that more harm is being done by OpenAI by keeping the training set secret. We're all confused on what to believe. Maybe it "worked it out" maybe it didn't. It is certainly impressive either way I guess. [1] https://en.wikipedia.org/wiki/Wordle https://en.wikipedia.org/wiki/Wordle
  - ftxbro 4y ago
    Yeah it looks like I was wrong, and that both 3.5 and 4.0 have seen wordle before getting the prompt even though it was released in late 2021 and even though 4.0 can solve it but 3.5 can't.
ChatGTP 4y ago
It's a major problem that no one knows.
skykooler 4y ago
This seems like it would be fairly easy to test by quizzing it on a similar sort of game but with a different set of rules so it wouldn't be in the training data.
jamilton 4y ago
I think it's really unlikely it's the last one. I think it's closer to the second.

DaiPlusPlus 4y ago

It’s the first one. GPT is scary.

SomewhatLikely 4y ago

Wordle is a kind of modified hangman, there's probably lots of data on the web for anagrams and hangman-like games.

sdenton4 4y ago
Why is Gpt4 better at these tasks than gpt3 if it only comes down to training data?
ChatGTP 4y ago
Likely a lot of "implement a hangman" code. I'm starting to wonder if a large part of GH might be in the training data, hence the "we don't want to release the training data" stance. Bit of a conspiracy maybe, but is it impossible?
- arlcode 4y ago
  Microsoft is betting really hard on AI. Yesterday I noticed that they integrated a link to "chat" into the start/search Menu of windows 11. I would not at all be surprised if they "sampled" large parts of private GH repositories, paying customers be damned. Probably covered with a corporate fig leave of "tokenizing the data" and hiding behind "to improve our services".

ftxbro 4y ago

It's absolutely the first one. The series of publications related to the GPT family of models show a steady history of unlocking new capabilities just by reducing token perplexity. In the limited info provided by the GPT-4 press release pdf they gave the example of 'Hindsight Neglect' as one that was automatically unlocked in the move from 3.5 to 4.0. I guess wordle is another one!

ChatGTP 4y ago
I don't know how helpful this is to keep the back and fourth going on what ChatGPT can and can't do, but I found this on Wikipedia[1]: Wordle is a web-based word game created and developed by Welsh software engineer Josh Wardle. It was bought by the New York Times Company in 2022 who has since maintained and published it. Players have six attempts to guess a five-letter word, with feedback given for each guess in the form of colored tiles indicating when letters match or occupy the correct position. The mechanics are nearly identical to the 1955 pen-and-paper game Jotto and the television game show franchise Lingo. Wordle has a single daily solution, with all players attempting to guess the same word. Wardle created the game for himself and his partner to play, eventually making it public in October 2021. The game gained popularity in December 2021 after Wardle added the ability for players to copy their daily results as emoji squares, which were widely shared on Twitter. Many clones and variations of the game were also created, as were versions in languages besides English. The game was purchased by The New York Times Company in January 2022 for an undisclosed seven-figure sum, with plans to keep it free for all players; it was moved to the company's website in February 2022.* I'm not downplaying this but I mean, if there have been games with nearly identical mechanics since 1955...well you can extrapolate from there. I'm now starting to get the vibe that more harm is being done by OpenAI by keeping the training set secret. We're all confused on what to believe. Maybe it "worked it out" maybe it didn't. It is certainly impressive either way I guess. [1] https://en.wikipedia.org/wiki/Wordle https://en.wikipedia.org/wiki/Wordle
- ftxbro 4y ago
  Yeah it looks like I was wrong, and that both 3.5 and 4.0 have seen wordle before getting the prompt even though it was released in late 2021 and even though 4.0 can solve it but 3.5 can't.

ChatGTP 4y ago

It's a major problem that no one knows.

skykooler 4y ago

This seems like it would be fairly easy to test by quizzing it on a similar sort of game but with a different set of rules so it wouldn't be in the training data.

jamilton 4y ago

I think it's really unlikely it's the last one. I think it's closer to the second.