I'd guess they already trained it on high quality (and highly censored) data. It's GPT-2-like architecture, but not GPT-2 weights. phi-1.5 is 1.3 BILLION, this

I'd guess they already trained it on high quality (and highly censored) data. It's GPT-2-like architecture, but not GPT-2 weights. phi-1.5 is 1.3 BILLION, this one is a lot smaller at 34 million.