Indeed. But TheBloke doesn't (yet) do exl2 quantization. That one is almost better done yourself, as the bpw target is arbitrary, and requires profiling text t

Indeed. But TheBloke doesn't (yet) do exl2 quantization. That one is almost better done yourself, as the bpw target is arbitrary, and requires profiling text that should probably fit your intended use (though many just use wikitext as a good default).