3 ms·

So what? This isn't some kind of gotcha. They're being explicit about the limitation. The built an interesting thing and showed that it does interesting things

• modeless 4y ago
  I disagree that the blog post was explicit enough. Tacking parenthesized disclaimers onto the end of your sentences is not really very explicit. What they're comparing against isn't even Adam; calling it Adam at all is misleading. The subtitle of the Adam paper is "A Method for Stochastic Optimization". It's not designed or intended for full-batch, non-stochastic training. What they should do is lead that paragraph with the fact that this is full-batch, note that this is not normal, and explain why they chose to do it anyway. And they shouldn't call it plain "Adam". The paper is probably good, and it's fine to test things in toy settings, but you need to be upfront with your assumptions and limitations.
• godelski 4y ago
  With the hype many people have a difficult time interpreting research. Thinking it is all results oriented. For readers, you need to remember that there's still a lot of theory research that's important. Research is far more than getting benchmark results or flashy images. Benchmarks are only proxies for what we're actually trying to learn and as such need to be evaluated with care (I wrote about this recently). This is especially true for metics that use parametric models to evaluate, such as FID. It's problematic if we can't do research in a research field. Research isn't production, though the two can overlap.