4 ms·

Is there an accompanying paper out there?

• blt 2y ago
  +1, curious to see if the paper has a convergence rate proof for convex objectives.
  • cutkosky 2y ago
    (an) author here: paper will likely be coming out in O(month). But, yes it turns out that the method is minimax optimal for stochastic convex optimization for a wide variety of parameter settings. Of course, minimax optimality alone does not fully explain empirical success - we've had minimax optimal algorithms for decades!
    • xpe 2y ago
      This would have been witty IMO: "the paper will be out in O(negative_peer_reviews)" > (an) author here: paper will likely be coming out in O(month) Ug. I'm adding "O(month)" to my list of bootless metaphors. Why? (1) Because in Big-O notation, O(month) would equal O(day), which is not the intended meaning in the comment above; (2) It is non-sensical; one would never say e.g. "the run-time of an algorithm is O(seconds)" -- we write some kind of input inside the parens, not the output Anyhow, we already have the words roughly and about; e.g. "about a month". Feel free to call me pedantic, but words matter.
      • goldemerald 2y ago
        I thought it was a clever/nerdy way to say in the worst case it will be out in a month. I imagine they have an internal review they have to get through first, and it's not clear if that will be done next week or in May.
        • pigpang 2y ago
          We can safely assume that approximate time needed to produce papers is t_approx = O(n), where n is number of papers. O(t_approx) makes no sense.
          • xpe 2y ago
            I think the above comment probably meant "where n is number of _pages_".
    • blt 2y ago
      agree, just a good "sanity check" for first-order optimization algos.
• miven 2y ago
  I think the author of this method said it's coming in a month or so
• JieJie 2y ago
  From the Related Work section (best guess): Stochastic Weight Averaging (Izmailov et al 2018) https://arxiv.org/abs/1803.05407 https://arxiv.org/abs/1803.05407 Latest Weight Averaging (Kaddour 2022) https://arxiv.org/abs/2209.14981 https://arxiv.org/abs/2209.14981 Latest Weight Averaging? (Sanyal et al 2023) https://arxiv.org/abs/2311.16294 https://arxiv.org/abs/2311.16294 Cyclic Learning Rates (Portes et al 2022) https://arxiv.org/abs/2206.00832 https://arxiv.org/abs/2206.00832 Exponential Moving Average? (Zhanghan? et al 2019) https://arxiv.org/abs/1909.01804 https://arxiv.org/abs/1909.01804
  • mkl 2y ago
    Those are other people's papers about other methods.
    • JieJie 2y ago
      My mistake. I misread the comment that it was looking for links to the included research and I went to find them.