Parameter count is not everything. 20b parameters * 1.5 bits per parameter is just 30 billion bits, about 3.75gb a full 20b fp16 is about 40GB. I find it wei

Parameter count is not everything. 20b parameters * 1.5 bits per parameter is just 30 billion bits, about 3.75gb a full 20b fp16 is about 40GB. I find it weird how a smaller model still produces decent text, except it bullshits all the way.