I see, the 2.9 GB requirements seems to imply a 3 bit weights? In any case I am happy to see these projects taking form. Perhaps one can eventually make the le

I see, the 2.9 GB requirements seems to imply a 3 bit weights? In any case I am happy to see these projects taking form. Perhaps one can eventually make the level of quantization dynamic based on the available vram etc :) I will definitively play around with it (on linux though, not a phone!)