3 ms·
Can someone explain for a dummy how this is possible? How does it know that range() is zero indexed? Was it specifically trained on Python input/function/output
by lopatin 5y ago
Can someone explain for a dummy how this is possible? How does it know that range() is zero indexed? Was it specifically trained on Python input/function/output data? Or did it just "learn" it? Do the researchers know how it learned it?
Does it actually "run" the code? Like, if it was looping over 1 billion iterations would it take 1B times longer than if it was just one iteration? I have so many questions.
- etskinner 5y agoIt seems more likely that it learned it. If you knew nothing about Python, but understood the word "for" a little, and understood code a little, you're likely to figure out that range() is zero-indexed after you see something like this a few times >>> for i in range(3): print(i) 0 1 2
- lopatin 5y agoMy mind is just blown that it learned a language runtime based on examples. What would happen if you gave it an infinitely recusrive function? It can't stack overflow, there's no stack! Wait, is there?
- stevenhuang 5y agoMy guess is it would respond with the standard stack overflow error, from examples of similar output posted in its training set.
- deleted 5y ago[deleted]
- YeGoblynQueenne 5y agoIt hasn't. It's memorised the examples and can generate them with some variation and according to what output is more likely given the input. But that's generation, not computation. I don't know if this example helps, but a computer can generate (pseudo-)random numbers by executing an algorithm. A pair of dice can also generate random numbers because they're thrown so that they land at random and someone has marked pips on their faces that a human can read as numbers. The result may be similar, but one is generated by a computation and the other by a random process. The random process is not a computation. It's a random process. (Or just unpredictable).
- lucidrains 5y agohttps://alphacode.deepmind.com/ https://alphacode.deepmind.com/ gives you a glimpse inside of what emerged from a similar attention net trained on code. however, whether the attention net has been forced upon pixels, language, amino acid sequences, the resultant representations are a bit beyond human reasoning, even if we can examine what individual attention heads are 'looking' at
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- MauranKilom 5y ago> How does it know that range() is zero indexed? If you read through all of the internet once, would you know that range() is zero indexed? > Like, if it was looping over 1 billion iterations would it take 1B times longer than if it was just one iteration? It clearly cannot, because querying the network for a token executes the exact same sequence of operations every time. But it's very impressive that it can basically recognize the Collatz Conjecture in the code and mostly guess in the right ballpark for the results. The fact that it's just liking (in a loose sense) inputs to inputs it has seen is quite visible in the f(g(x)) vs g(f(x)) behavior - the former is significantly more common, so it struggles to work with the latter.
- mhh__ 5y agoHow do you know a range is zero indexed? (As in how is it stored in your brain)