Here’s an example of a custom 4 bits/weight codec for ML weights: https://github.com/Const-me/Cgml/blob/master/Readme.md#bcml1-codec https://github.com/Const-m

Here’s an example of a custom 4 bits/weight codec for ML weights: https://github.com/Const-me/Cgml/blob/master/Readme.md#bcml1-codec https://github.com/Const-me/Cgml/blob/master/Readme.md#bcml1... llama.cpp does it slightly differently but still, AFAIK their quantized data formats are conceptually similar to my codec.