While you are at it, I'd like to point to some popular existing datasets for object recognition: The labelme dataset: http://labelme.csail.mit.edu/instructions

While you are at it, I'd like to point to some popular existing datasets for object recognition: The labelme dataset: http://labelme.csail.mit.edu/instructions.html http://labelme.csail.mit.edu/instructions.html This is close to what you were thinking (and I liked) -- having annotation done by public. A list: http://www.computervisiononline.com/datasets http://www.computervisiononline.com/datasets PASCAL VOC: http://pascallin.ecs.soton.ac.uk/challenges/VOC/voc2011/ http://pascallin.ecs.soton.ac.uk/challenges/VOC/voc2011/ CALTECH 256, and several others. Many papers presented in CVPR 2012 (http://www.cvpr2012.org/ http://www.cvpr2012.org/) were indeed on very large datasets. Do note that some of these datasets are fairly large (may or may not be as broad though). The algorithms you use will depend on the complexity you can handle. OpenCV, IPP, and CCV (which was on hacker news few days back) could provide good options for algorithms with a training dataset you choose/create.