2 ms·

This isn't even 'GPT-2'! Since that usually refers to GPT-2-1.5b. It's GPT-2-117M... distilled, so an even smaller version of the smallest GPT-2. Amazing it wor

by gwern 5y ago

This isn't even 'GPT-2'! Since that usually refers to GPT-2-1.5b. It's GPT-2-117M... distilled, so an even smaller version of the smallest GPT-2. Amazing it works so well anyway.