3 ms·

Try something like: The current board state is: board = [['', '', ''], ['', '', ''], ['', '', '']]; write a javascript function c

Try something like: The current board state is: board = [['', '', ''], ['', '', ''], ['', '', '']]; write a javascript function called bestMove(board) that predicts the best tic-tac-toe move to make given a board. use that function to update the board state and return the board state in JSON form. The response will have a bunch of functions like function bestMove(board) { function getEmptySpaces(board) { function predictBestMove(board, player) { function minimax(board, isMaximizing) { function checkWinner(board) { ... These functions should work together to determine the best move to make in a game of Tic-Tac-Toe, using the minimax algorithm to evaluate each possible move and choosing the one with the highest score. Then eval and execute the bestMove function, passing in the initial board state, returning the updated board state. Then the human player makes a move. Then another prompt: Try something like: The current board state is: board = [['X', '', ''], ['', 'O', ''], ['', '', '']]; assume there is a function called bestMove(board) and checkWinner(board) that predicts the best tic-tac-toe move to make given a board. use those functions to update the board state and check the winner and return the board state and current winner in JSON form. etc... Using my little engine, I get this solution: question: "Answering as [rowInt, colInt], writing custom predictBestMove, getEmptySpaces, minimax and checkWinner functions implemented in the thunk, what is the best tic-tac-toe move for player X on this board: [['X', '_', 'X'], ['_', '_', '_'], ['_', '_', '_']]?", answer: [ 0, 1 ], https://gist.github.com/williamcotton/e6bdcca0a96a6e7bf5d2fe422d0611d1 https://gist.github.com/williamcotton/e6bdcca0a96a6e7bf5d2fe...

the_af 4y ago
Interesting. Have you tried playing a full game like this, instead of a single move? In any case, I don't think this is what people expect out of ChatGPT. Your approach is too "programmer centric". I think people expect telling ChatGPT the rules of the game, in almost plain language, and then expect to be able to play a game of Tic Tac Toe interacting with it like one would with a person. This means, not asking it to write functions or remind it of the state of the board at every step. This doesn't work consistently for a well-known game like Tic Tac Toe, much less for an arbitrary game you make up.
- williamcotton 4y ago
  > Interesting. Have you tried playing a full game like this, instead of a single move? No, but it is correctly running the best move functions so through induction we can see it will successfully play a full game. > I think people expect telling ChatGPT the rules of the game, in almost plain language, and then expect to be able to play a game of Tic Tac Toe interacting with it like one would with a person. This is an unreasonable expectation for a large language model. When a person computes the sum of two large numbers they do not use their language facilities. They probably require a pencil and pad so they can externalize the computational process. At the very least they are performing calculations in their head in a manner very different from the cognitive abilities used when they catch a ball. Try playing a game like Risk without a board or pieces, that is, without a concrete mechanism to maintain state. This approach isn’t cheating and an LLM acting as a translator is a key component. This doesn’t “prove that LLMs are useless bullshit generators, snicker snicker” because it can’t maintain state or do math very well, it just means you need to use other existing tools to do math and maintain state… like JS interpreters. One thing that I think will improve is that a larger scale language model would need less internally specific terms for the solution in order to reliably get the same results. Also, translations are necessarily lossy and somewhat arbitrary, so these results need to be considered probabilistically as well. Meaning, generate 10 different thunks and have them act as voting on an answers they compute.
  - the_af 4y ago
    > No, but it is correctly running the best move functions so through induction we can see it will successfully play a full game. I'm not convinced induction applies. ChatGPT tends to "go astray" in conversations where it needs to maintain state; even with your patch for this (essentially reminding it what the state is at every prompt) I would test it just to make sure it can run a game through completion, make good moves all the way, and be able to tell when the game is over. I can make ChatGPT do single "reasonable" moves, the problem surfaces during a full game. > This is an unreasonable expectation for a large language model. Yes, but enough people hold it anyway that it is a concern. And it's made worse because in some contexts ChatGPT fakes this quite effectively!

the_af 4y ago

Interesting. Have you tried playing a full game like this, instead of a single move? In any case, I don't think this is what people expect out of ChatGPT. Your approach is too "programmer centric". I think people expect telling ChatGPT the rules of the game, in almost plain language, and then expect to be able to play a game of Tic Tac Toe interacting with it like one would with a person. This means, not asking it to write functions or remind it of the state of the board at every step. This doesn't work consistently for a well-known game like Tic Tac Toe, much less for an arbitrary game you make up.

williamcotton 4y ago
> Interesting. Have you tried playing a full game like this, instead of a single move? No, but it is correctly running the best move functions so through induction we can see it will successfully play a full game. > I think people expect telling ChatGPT the rules of the game, in almost plain language, and then expect to be able to play a game of Tic Tac Toe interacting with it like one would with a person. This is an unreasonable expectation for a large language model. When a person computes the sum of two large numbers they do not use their language facilities. They probably require a pencil and pad so they can externalize the computational process. At the very least they are performing calculations in their head in a manner very different from the cognitive abilities used when they catch a ball. Try playing a game like Risk without a board or pieces, that is, without a concrete mechanism to maintain state. This approach isn’t cheating and an LLM acting as a translator is a key component. This doesn’t “prove that LLMs are useless bullshit generators, snicker snicker” because it can’t maintain state or do math very well, it just means you need to use other existing tools to do math and maintain state… like JS interpreters. One thing that I think will improve is that a larger scale language model would need less internally specific terms for the solution in order to reliably get the same results. Also, translations are necessarily lossy and somewhat arbitrary, so these results need to be considered probabilistically as well. Meaning, generate 10 different thunks and have them act as voting on an answers they compute.
- the_af 4y ago
  > No, but it is correctly running the best move functions so through induction we can see it will successfully play a full game. I'm not convinced induction applies. ChatGPT tends to "go astray" in conversations where it needs to maintain state; even with your patch for this (essentially reminding it what the state is at every prompt) I would test it just to make sure it can run a game through completion, make good moves all the way, and be able to tell when the game is over. I can make ChatGPT do single "reasonable" moves, the problem surfaces during a full game. > This is an unreasonable expectation for a large language model. Yes, but enough people hold it anyway that it is a concern. And it's made worse because in some contexts ChatGPT fakes this quite effectively!