4 ms·

1-bit weights have been a thing since at least 2016: https://arxiv.org/abs/1606.06160 https://arxiv.org/abs/1606.06160

• buildbot 3y ago
  XNOR-Net was the "first" in that generation, as I recall. It's not a new idea at all though. Check out the dates on these papers - https://ieeexplore.ieee.org/abstract/document/286901 https://ieeexplore.ieee.org/abstract/document/286901 < 1994 https://ieeexplore.ieee.org/abstract/document/344783 https://ieeexplore.ieee.org/abstract/document/344783 < 1993 https://link.springer.com/article/10.1007/BF00337115 https://link.springer.com/article/10.1007/BF00337115 < 1986
  • bjornsing 3y ago
    First version of BNN was submitted to ArXiv a month before XNOR-Net: https://arxiv.org/abs/1602.02830 https://arxiv.org/abs/1602.02830
    • buildbot 3y ago
      Oh nice, good catch!
  • thechao 3y ago
    We're just speed-running the NN pendulum, at this point.
• rapatel0 3y ago
  Don't forget Michael https://arxiv.org/abs/1511.00363 https://arxiv.org/abs/1511.00363 < 2015
• elcomet 3y ago
  They have been a thing for decades https://ieeexplore.ieee.org/abstract/document/363432 https://ieeexplore.ieee.org/abstract/document/363432
  • WithinReason 3y ago
    I don't think that paper's methods could be applied to LLMs