3 ms·

I wouldn't expect a human to fail at the single line version. It's pretty easy to do. Write a list, at each position n in the list, compute the sum of the nth l

by rcme 3y ago

I wouldn't expect a human to fail at the single line version. It's pretty easy to do. Write a list, at each position n in the list, compute the sum of the nth least-significant bit and the carry from the line above. You can easily get the nth least significant bit by counting from the right side of each number. The end result is the list printed backwards.

shagie 3y ago
Consider the amount of work that you need to do to reverse a word in GPT. https://twitter.com/npew/status/1525900868305833984?lang=en# https://twitter.com/npew/status/1525900868305833984?lang=en# Note that part of the trick that they did to get it to work better is putting spaces around each letter to enforce a particular tokenization. Trying to do the same with mathematical objects rather than language objects would be even harder because GPT doesn't "know" math.
sdenton4 3y ago
You lost the thread at 'write.' Try to do it in your head; we pretty much have to resort to additional tools to succeed at this task. Here's a LLM-generated python solution, which works just fine: def add_binary(a, b): """Adds two binary numbers given as strings. Args: a: The first binary number. b: The second binary number. Returns: The sum of the two binary numbers. """ # Convert the binary strings to numbers. a = int(a, 2) b = int(b, 2) # Add the numbers. sum = a + b # Convert the sum back to a binary string. return bin(sum)[2:] add_binary('10011111110100011100011011110010', '00111010100101000100000101111011')
- rcme 3y ago
  I feel like we're missing the forest for the trees here. I'm not really asking it to add two numbers. I'm asking it to apply a simple set of logical rules on two symbols with an operation '+'. Of course it can generate code to add two numbers, because the code to add two numbers exists in its training data. What transformers seem to be good at is generalizing over the training data. Where they seem to perform poorly is data outside of the training set. Binary numbers are interesting because you can add a bit of entropy just by increasing the size of your binary number, so it's very easy to generate examples outside of the training data. You also see this, albeit less directly, when trying to ask it to generate code that wasn't part of its training data. E.g. https://twitter.com/cHHillee/status/1635790330854526981 https://twitter.com/cHHillee/status/1635790330854526981

by rcme 3y ago

shagie 3y ago

Consider the amount of work that you need to do to reverse a word in GPT. https://twitter.com/npew/status/1525900868305833984?lang=en# https://twitter.com/npew/status/1525900868305833984?lang=en# Note that part of the trick that they did to get it to work better is putting spaces around each letter to enforce a particular tokenization. Trying to do the same with mathematical objects rather than language objects would be even harder because GPT doesn't "know" math.

sdenton4 3y ago

You lost the thread at 'write.' Try to do it in your head; we pretty much have to resort to additional tools to succeed at this task. Here's a LLM-generated python solution, which works just fine: def add_binary(a, b): """Adds two binary numbers given as strings. Args: a: The first binary number. b: The second binary number. Returns: The sum of the two binary numbers. """ # Convert the binary strings to numbers. a = int(a, 2) b = int(b, 2) # Add the numbers. sum = a + b # Convert the sum back to a binary string. return bin(sum)[2:] add_binary('10011111110100011100011011110010', '00111010100101000100000101111011')

rcme 3y ago
I feel like we're missing the forest for the trees here. I'm not really asking it to add two numbers. I'm asking it to apply a simple set of logical rules on two symbols with an operation '+'. Of course it can generate code to add two numbers, because the code to add two numbers exists in its training data. What transformers seem to be good at is generalizing over the training data. Where they seem to perform poorly is data outside of the training set. Binary numbers are interesting because you can add a bit of entropy just by increasing the size of your binary number, so it's very easy to generate examples outside of the training data. You also see this, albeit less directly, when trying to ask it to generate code that wasn't part of its training data. E.g. https://twitter.com/cHHillee/status/1635790330854526981 https://twitter.com/cHHillee/status/1635790330854526981