3 ms·

Model 4 bit inference 4.2x faster than 16 bit with full HF support

• convexstrictly 3y ago
  Release notes: https://github.com/TimDettmers/bitsandbytes/releases/tag/0.40.0 https://github.com/TimDettmers/bitsandbytes/releases/tag/0.4...