4 ms·

My first response on seeing these results: 1. Could it be that the (small fraction) of problems being solved correctly involve feature recognition and spatial

My first response on seeing these results: 1. Could it be that the (small fraction) of problems being solved correctly involve feature recognition and spatial position? Eg: all images have a shape in the top right corner -vs- bottom left corner 2. Using a model with 1050 features makes me uncomfortable, because of the ability to overfit, or guess solutions randomly. However, I'm not able to make a watertight argument and would appreciate an analysis along these lines. Numbers which such an analysis might involve: There are 10choose5=252 ways to partition 10 images into two sets of 5 (and 2 ways to assign the remaining two images). If one wants to improve the training by some form of augmentation or cross-validation, there are 36 ways to drop one image each from the left and right side, for each Bongard problem.

nerdponx 9y ago
1. I think this is the idea behind convolutional neural networks. You learn individual features in the image in the convolutional layers, and you learn their arrangement in the subsequent layers. It turns out that networks don't actually do that AFAIK, but that's the idea as I've been taught it. 2. You can't think of these features the same way you think of features in a linear model, or something like that. This is typical for a neural network.
coolvision 9y ago
1. Yes, you are right. Most of solved problems are quite simple. Still, have to start somewhere? :) 2. For testing, definitely should all add cross-testing on many partitions, and then report average or worst result. Still, for mt better than random results were good for a start.
- ssivark 9y ago
  Oh, I found the experiment and blog post very interesting... Thanks for that! Also, I’d never heard of Bongard problems, so I enjoyed reading about that too. I realize now that my comments might have sounded overly critical; didn’t mean to. The intention behind my probing was to understand precisely the regimes in which the algorithm works and fails :-) In particular, towards understanding how it might be improved and whether a NN approach is capable of solving Bongard problems.

nerdponx 9y ago

1. I think this is the idea behind convolutional neural networks. You learn individual features in the image in the convolutional layers, and you learn their arrangement in the subsequent layers. It turns out that networks don't actually do that AFAIK, but that's the idea as I've been taught it. 2. You can't think of these features the same way you think of features in a linear model, or something like that. This is typical for a neural network.

coolvision 9y ago

1. Yes, you are right. Most of solved problems are quite simple. Still, have to start somewhere? :) 2. For testing, definitely should all add cross-testing on many partitions, and then report average or worst result. Still, for mt better than random results were good for a start.

ssivark 9y ago
Oh, I found the experiment and blog post very interesting... Thanks for that! Also, I’d never heard of Bongard problems, so I enjoyed reading about that too. I realize now that my comments might have sounded overly critical; didn’t mean to. The intention behind my probing was to understand precisely the regimes in which the algorithm works and fails :-) In particular, towards understanding how it might be improved and whether a NN approach is capable of solving Bongard problems.