4 ms·

These systems can't count in general. Doesn't matter if its fingers, n clocks on a wall (for all 1 digit vals of n), some variation of n items on an x, or even

These systems can't count in general. Doesn't matter if its fingers, n clocks on a wall (for all 1 digit vals of n), some variation of n items on an x, or even "a picture within a picture within..." k times. The machine can't count. >I ended up just taking a photo of my daughter and blurring the background. Ah, the diffusion process.

pwinnski 4y ago
I suspect inability to count is a thing that can be trained for, although whether that will come at the expense of something else, I'm not sure. I appreciate the humor. :)
- IIAOPSW 4y ago
  Why the optimism? If it hasn't picked up on the gist of the meaning of 1-10 after 10 billion examples, why expect it to work after 20 billion examples? These models have worked unreasonably well so far, but there has to be a limit to how much we can substitute more data for our lack of conceptual understanding of cognitive processes.
  - pwinnski 4y ago
    Because I'm not sure counting is something that has been explicitly tagged or trained for. Clearly always-accurate counts are not an emergent feature, but if you tagged a large set of images with tags we think are too obvious to tag as humans, like "ten fingers" and so on, if that was an actual goal to be rewarded, I think it could improve results. I'm overall in the skeptic camp, but it seems like these models can generally deliver what they're trained for. It doesn't appear any have been trained with counting as a primary goal.
    - IIAOPSW 4y ago
      Ten fingers surely wouldn't be tagged, but the thing about numbers is that they work as well for 10 apples as they do for 10 coins and 10 cups and 10 whatever. It never made the leap of abstraction to learn in the latent space the meaning of numbers 1-10 independent of the particular object. This lack of extrapolation lends it to a very rote learned style. I already have a way around the specific numerical vocabulary/training problem. When I ask for "a picture in a frame", "a picture of a picture in a frame", "a picture of a picture of a picture in a frame" etc, I'm trying to use linguistic recursion to make numeracy emerge. But even that form of counting without predefined numbers fails. There's no reliable way to make a prompt the produces k of something. That's a deeper issue than it not deducing the specific meaning of characters 0-9 by example. It can't learn that by example as there really isn't training data with more than 2 nested pictures of pictures, and by itself it will never realize it can just fill in the nested painting by prompting itself with the nested statement. It lacks thought loops.
gwern 4y ago
Wrong. Look at eg Parti. Solely a matter of scaling the text encoder and then not screwing it up with unCLIP etc.
- IIAOPSW 4y ago
  Its not wrong its an experimental observation based on systematic tests I've done. Maybe, with great strain either in quantity of training or ad hoc tweaking, this particular low bar can be hurdled. But it is still the case in general that numeracy is not prone to naturally and unexpectedly emerge from this type of system. The extent to which that matters depends on what your goal is. If you care about results in the here and now, then random patches optimized to use cases is fine. If you care about figuring out where to look for things we can do cognitively that are not very well suited to these generator models, the surprising lack of numeracy so far is a great starting point. The next obvious question is why it takes billions to notice the numbers 1 - 10? What models wouldn't suck at that?
  - gwern 4y ago
    > Its not wrong its an experimental observation based on systematic tests I've done. You've done systematic tests on Parti?

pwinnski 4y ago

I suspect inability to count is a thing that can be trained for, although whether that will come at the expense of something else, I'm not sure. I appreciate the humor. :)

IIAOPSW 4y ago
Why the optimism? If it hasn't picked up on the gist of the meaning of 1-10 after 10 billion examples, why expect it to work after 20 billion examples? These models have worked unreasonably well so far, but there has to be a limit to how much we can substitute more data for our lack of conceptual understanding of cognitive processes.
- pwinnski 4y ago
  Because I'm not sure counting is something that has been explicitly tagged or trained for. Clearly always-accurate counts are not an emergent feature, but if you tagged a large set of images with tags we think are too obvious to tag as humans, like "ten fingers" and so on, if that was an actual goal to be rewarded, I think it could improve results. I'm overall in the skeptic camp, but it seems like these models can generally deliver what they're trained for. It doesn't appear any have been trained with counting as a primary goal.
  - IIAOPSW 4y ago
    Ten fingers surely wouldn't be tagged, but the thing about numbers is that they work as well for 10 apples as they do for 10 coins and 10 cups and 10 whatever. It never made the leap of abstraction to learn in the latent space the meaning of numbers 1-10 independent of the particular object. This lack of extrapolation lends it to a very rote learned style. I already have a way around the specific numerical vocabulary/training problem. When I ask for "a picture in a frame", "a picture of a picture in a frame", "a picture of a picture of a picture in a frame" etc, I'm trying to use linguistic recursion to make numeracy emerge. But even that form of counting without predefined numbers fails. There's no reliable way to make a prompt the produces k of something. That's a deeper issue than it not deducing the specific meaning of characters 0-9 by example. It can't learn that by example as there really isn't training data with more than 2 nested pictures of pictures, and by itself it will never realize it can just fill in the nested painting by prompting itself with the nested statement. It lacks thought loops.

gwern 4y ago

Wrong. Look at eg Parti. Solely a matter of scaling the text encoder and then not screwing it up with unCLIP etc.

IIAOPSW 4y ago
Its not wrong its an experimental observation based on systematic tests I've done. Maybe, with great strain either in quantity of training or ad hoc tweaking, this particular low bar can be hurdled. But it is still the case in general that numeracy is not prone to naturally and unexpectedly emerge from this type of system. The extent to which that matters depends on what your goal is. If you care about results in the here and now, then random patches optimized to use cases is fine. If you care about figuring out where to look for things we can do cognitively that are not very well suited to these generator models, the surprising lack of numeracy so far is a great starting point. The next obvious question is why it takes billions to notice the numbers 1 - 10? What models wouldn't suck at that?
- gwern 4y ago
  > Its not wrong its an experimental observation based on systematic tests I've done. You've done systematic tests on Parti?