4 ms·
I think this is a great thing to put out into the community, and I hope it gains widespread adoption. I do have a few things to comment on here: The architect
by kastnerkyle 13y ago
I think this is a great thing to put out into the community, and I hope it gains widespread adoption.
I do have a few things to comment on here:
The architectures themselves have never been closed source (hence publication). Frankly, I think Alex didn't release his implementation because until TITAN came along, you needed two GTX580 GPUs to do distributed training. This is unique (and difficult) to setup and support! Just "putting the code out there" is not useful to many researchers, and the architecture is in the paper already to implement and extend or modify. You will be bombed to death with support requests, even if you just say "this is provided as is". Many researchers don't want to deal with that IMO.
Data is EVERYTHING for training these architectures, and the ImageNet dataset itself is still semi-closed source, and hard to get. You have to sign up to get their license coverage if you want to download the original images for research. I think you can get weblinks to the original data, which you could then ostensibly crawl yourself, but that gets murky from a copyright perspective.
It will also take a very, very long time to wget all 1.2TB of ImageNet. My download took about 45 days, so keep that in mind!
Most people just do not have the hardware to train a network of this magnitude. CIFAR10 is a much more appropriate dataset for consumer grade hardware/non lab-funded work IMO. I think this is one of the reasons most advances on this front have simply released the trained network parameters as a "black-box" preprocessor - it is much more useful for general tasks on normal laptops and PCs! See http://fastml.com/yesterday-a-kaggler-today-a-kaggle-master-a-wrap-up-of-the-cats-and-dogs-competition/ http://fastml.com/yesterday-a-kaggler-today-a-kaggle-master-...
I personally used a simple wrapper to DeCAF + a sklearn logistic regression to score a pretty good (96%) score for the kaggle Cats vs. Dogs competition on a Lenovo T400, and I think this is where the commercial usefulness will go as well. The training + data to get the "magic numbers" is probably going to be the KFC/Coke tradesecret formula of data science in coming years. Plowing through a bunch of floats in a particular format, then applying simple classifiers is good enough for many standard computer vision tasks, and could possibly be fit into a very small formfactor. Think FPGAs...