3 ms·
The progress in this field is astonishing. It wasn't even a few years ago that the typical demonstration in a paper on GAN would have been an array of very smal
by a1371 9y ago
The progress in this field is astonishing. It wasn't even a few years ago that the typical demonstration in a paper on GAN would have been an array of very small images. I recall often seeing numbers between 64x64 and 256x256. At the time, the argument was raised that the resolution can hardly be increased because it is our brain that tries to identify objects and faces in the photos to figure out what they are representing.
Then here we are, with indistinguishable 1024x1024 recreations and trippy latent space interpolations. I know not every researcher or entrepreneur has the resources of NVIDIA to train for this many days, but let's not forget, that part needs to occur only once. It makes me wonder about the day that a GAN manages to bankrupt stock photography services.
- rjtavares 9y ago> I know not every researcher or entrepreneur has the resources of NVIDIA to train for this many days, but let's not forget, that part needs to occur only once. It's not like they trained this on a GPU farm. According to the paper [1], they "trained the network on a single NVIDIA Tesla P100 GPU for 20 days". [1] http://research.nvidia.com/sites/default/files/pubs/2017-10_Progressive-Growing-of//karras2017gan-paper.pdf http://research.nvidia.com/sites/default/files/pubs/2017-10_...
- 317070 9y agoYes, but they probably did not get it right on first try. You usually need hundreds of iterations to get it right. This does not imply 2000 days of GPU, but it quickly pushes this outside of reach of consumer hardware.
- a1371 9y agoWhere I live that thing alone is $17,000.
- bitL 9y agoP100 is essentially 1080Ti, so you can grab 1080 Ti and have the same speed. V100 might be the expensive version, up to 10x faster.
- gmueckl 9y agoIt may be the same hardware, but nVidia is known to block existing hardware features in their drivers on consumer level chips to drive prices. They did this in the past with quad buffer stereo (full screen only, not available through OpenGL) or decreased double precision performance (transparently added NOPs after each double precision opcode). I will believe that these devices are equal when someone shows me benchmarks proving that. Until then, I am skeptical based on past experience.
- jamesblonde 9y agoYou should look at GPU memory bandwidth for a proxy for performance when training DNNs. The P100 is about 40% faster than a 1080Ti. The V100 is only about 75% faster than a 1080Ti. Based on this, i expect these commodity GPU servers (with 10 1080Ti cards) that cost 1/10th of the DGX-1 will be huge: https://www.servethehome.com/deeplearning11-10x-nvidia-gtx-1080-ti-single-root-deep-learning-server-part-1/ https://www.servethehome.com/deeplearning11-10x-nvidia-gtx-1...