3 ms·
Give it a few days and lucidrains will have the code up[0]. But in honesty, it is probably how people react. We saw this with Pulse, GPT, and many others. The
by godelski 4y ago
Give it a few days and lucidrains will have the code up[0].
But in honesty, it is probably how people react. We saw this with Pulse, GPT, and many others. The authors are clear about the limitations but people talk it up too much and others shit on it. There's also a reproducibility crisis in ML (many famous networks, like Swin[1][2][3], can't be reproduced (even worse when reviewers concentrate on benchmarks)). It isn't like many can train a model like this anyways. It gives them benefit of the doubt and maintains good publicity rather than controversial.
Of course, this is extremely bad from an academic perspective and personally I believe you should have your paper revoked if it isn't reproducible. You'd be surprised how many don't track the random seed or measure variance. We have GitHub. You should be able to write training options that get approximately the same results as the paper. Otherwise I don't trust your results.
[0] https://github.com/lucidrains/parti-pytorch https://github.com/lucidrains/parti-pytorch
[1] https://github.com/microsoft/Swin-Transformer/issues/183 https://github.com/microsoft/Swin-Transformer/issues/183
[2] https://github.com/microsoft/Swin-Transformer/issues/180 https://github.com/microsoft/Swin-Transformer/issues/180
[3] https://github.com/microsoft/Swin-Transformer/issues/148 https://github.com/microsoft/Swin-Transformer/issues/148