6 ms·

I like to see how AlphaCode will solve a problem no human has solved before (or very unlikely). For example, given 3 beads and 2 stacks for the 10er and 1er po

I like to see how AlphaCode will solve a problem no human has solved before (or very unlikely). For example, given 3 beads and 2 stacks for the 10er and 1er positions of a number. One can make 4 different numbers when stack all the beads like in an abacus. Without using all the beads, one can of course make more numbers. The question is how many different numbers one can make using n beads, giving full and partial usage of the beads. It's indeed a very simple problem for 5-7 year olds and I hardly think anyone cannot solve it. Nevertheless, I seriously doubt AlphaCode can solve the problem. What does it say about its supposed Intelligence?

recursive 5y ago
Why do you think that's more difficult than the backspace question? I see no reason to doubt AlphaCode can solve it.
- 37ef_ced3 5y ago
  Because the backspace question (essentially: is T a subsequence of S with a deletion size of N?) probably occurs hundreds of times, in one form or another, within AlphaCode's training corpus. Any leetcode grinder can tell you there are a few dozen types of competitive programming problem (monostack, breadth-first state search, binary search over solution space, etc.) so solutions to new problems are often very similar to solutions for old problems. The training corpus for these code transformers is so large that almost all evaluation involves asking them to generate code from the training corpus. To evaluate CoPilot, we should ask questions that are unusual enough they can't be answered through regurgitation of the training corpus. What does CoPilot generate, given this prompt: // A Go function to set the middle six bits of an unsigned 64-bit integer to 1. Here is a human solution: func six1s(x uint64) uint64 { const ( ones = uint64(63) shift = (64 - 6) / 2 mask = ones << shift ) return x | mask } Can it solve this simple but unusual problem?
  - notamy 5y ago
    > What does CoPilot generate, given this prompt: Here's what it generated for me: // A Go function to set the middle six bits of an unsigned 64-bit integer to 1. func set6(x uint64) uint64 { return x | 0x3f }
    - 37ef_ced3 5y ago
      Incorrect. Thank you. CoPilot is helpless if it needs to do more than just regurgitate someone else's code. The training of these models on GitHub, so they regurgitate licensed code without attribution, is the greatest theft of intellectual property in the history of Man. Perhaps not according to the letter of the law, but surely according to the spirit.
      - notfed 5y ago
        I like CoPilot's answer better than yours, and I think it's closer to what most people would do; clearly 0x3F is the wrong constant but the approach is good.
        37ef_ced3 5y ago
        CoPilot's solution is totally wrong. Sorry. CoPilot regurgitated somebody's solution... to a different problem. It's pathetic.
        notfed 5y ago
        Here's my solution (I'm a human): func set6(x uint64) uint64 { return x | 0x7E0000000 } Is this also pathetic?
        37ef_ced3 5y ago
        A good solution! You SOLVED the problem. CoPilot got it WRONG. You got it RIGHT. You UNDERSTAND the problem but CoPilot does NOT. Is that clear?

recursive 5y ago

Why do you think that's more difficult than the backspace question? I see no reason to doubt AlphaCode can solve it.

37ef_ced3 5y ago
Because the backspace question (essentially: is T a subsequence of S with a deletion size of N?) probably occurs hundreds of times, in one form or another, within AlphaCode's training corpus. Any leetcode grinder can tell you there are a few dozen types of competitive programming problem (monostack, breadth-first state search, binary search over solution space, etc.) so solutions to new problems are often very similar to solutions for old problems. The training corpus for these code transformers is so large that almost all evaluation involves asking them to generate code from the training corpus. To evaluate CoPilot, we should ask questions that are unusual enough they can't be answered through regurgitation of the training corpus. What does CoPilot generate, given this prompt: // A Go function to set the middle six bits of an unsigned 64-bit integer to 1. Here is a human solution: func six1s(x uint64) uint64 { const ( ones = uint64(63) shift = (64 - 6) / 2 mask = ones << shift ) return x | mask } Can it solve this simple but unusual problem?
- notamy 5y ago
  > What does CoPilot generate, given this prompt: Here's what it generated for me: // A Go function to set the middle six bits of an unsigned 64-bit integer to 1. func set6(x uint64) uint64 { return x | 0x3f }
  - 37ef_ced3 5y ago
    Incorrect. Thank you. CoPilot is helpless if it needs to do more than just regurgitate someone else's code. The training of these models on GitHub, so they regurgitate licensed code without attribution, is the greatest theft of intellectual property in the history of Man. Perhaps not according to the letter of the law, but surely according to the spirit.
    - notfed 5y ago
      I like CoPilot's answer better than yours, and I think it's closer to what most people would do; clearly 0x3F is the wrong constant but the approach is good.
      - 37ef_ced3 5y ago
        CoPilot's solution is totally wrong. Sorry. CoPilot regurgitated somebody's solution... to a different problem. It's pathetic.
        notfed 5y ago
        Here's my solution (I'm a human): func set6(x uint64) uint64 { return x | 0x7E0000000 } Is this also pathetic?
        37ef_ced3 5y ago
        A good solution! You SOLVED the problem. CoPilot got it WRONG. You got it RIGHT. You UNDERSTAND the problem but CoPilot does NOT. Is that clear?