30b 4bit is definitely useful because it runs on 3090s/4090s. You see a lot of people running 30b models. The jump in quality is very significant as well. 65b

30b 4bit is definitely useful because it runs on 3090s/4090s. You see a lot of people running 30b models. The jump in quality is very significant as well. 65b is definitely a lot less common.