3 ms·

1) Training with synthetic data is definitely a thing in computer vision, exactly the way you describe. You can even throw a GAN on top of the results to make t

1) Training with synthetic data is definitely a thing in computer vision, exactly the way you describe. You can even throw a GAN on top of the results to make the renderings look less artificial. 2) They do something like this, in section 4.3 by splitting ObjectNet in half. They fine-tune (the last layer of) ResNet-152 (I wonder what happens if you fine-tune more layers) on half and test on the other half. This pushes results up by about 15% points. There's still a gap, but it can be plausibly argued that the gap would close up if we scaled things up by one or two orders of magnitude. The question is whether there's a better way.

nl 7y ago
> I wonder what happens if you fine-tune more layers Generally it improves some, but most of the gains are in the final layer retraining. But it's a lot more data hungry. > They do something like this, in section 4.3 by splitting ObjectNet in half.... here's still a gap, but it can be plausibly argued that the gap would close up if we scaled things up by one or two orders of magnitude This is interesting. It's worth noting that this training is on only 64 images per class, and it is unclear if they augment this in anyway. Before retraining, the paper itself notes: Classes such as plunger, safety pin and drill have 60-80% accuracy, while French press, pitcher, and plate have accuracies under 5% It is worth noting that the plunger, safety pin and drill classes are ones that have multiple orientations already in ImageNet, while French press, pitcher, and plate are almost all the "right" way up. To me this indicates this is simply a data problem - the model has never seen what an upside-down French press looks like so it gets it wrong.