The first commit of Lasagne in 2014 already had the 'untie bias' option [0], which achieves the same effect as the paper, but in a different way (and is in my o

The first commit of Lasagne in 2014 already had the 'untie bias' option [0], which achieves the same effect as the paper, but in a different way (and is in my opinion more elegant). And while I cannot find a paper, I think it is one of those tricks which have been around since the Schmidhuber days. Moreover, it is one of the tricks which has been actively used since as long as I have been involved with convolutional neural networks (since 2010). So, the Uber paper is kind of silly, other for that I now know where to point to for a confirmation of the effectiveness of the idea. But I agree with you that the mischaracterisation is not appropriate. The main criticisms in the blogpost are missing the point too. The paper is not particularly interesting and might not be appropriate for the big conferences, but in my opinion not for the reasons in the blog post. Also, who cares about 100 GPU's? Nobody is complaining that all algorithms require 1 GPU these days and don't run on a smartphone, but suddenly 100 GPU's is too much? For some researchers (and I think Uber falls in the category), 100 GPU's are pocket change. Science does not require that your algorithm also runs on your lab's DIY Phd GPU cluster. If these guys have the GPU's available and it allowed them to be home earlier to spend time with the family, why would it be a problem for them to use the compute? [0]https://github.com/Lasagne/Lasagne/commit/2f9147497493f71ab6d79f35a041150f0c881d9d https://github.com/Lasagne/Lasagne/commit/2f9147497493f71ab6...