39 ms·

How usable are 1 and less than 1 bit quants?

• danielhanchen 1y ago
  Oh 1bit reminder isn't 1bit, but a mixture of 1 to 8bit! I would still use 2bit dynamic - 1bit dynamic sometimes can go into weird repetitive loops, but it still does produce reasonable output. Larger models with 1bit do better - for eg 480B Coder 1bit actually does very well!