3 ms·
Thats because its unoptimized, right? My impression is that the Falcon GPTQ (and 8 bit bnb inference?) code is just immature.
by brucethemoose2 3y ago
Thats because its unoptimized, right? My impression is that the Falcon GPTQ (and 8 bit bnb inference?) code is just immature.