I think you would still need a large model to train on general-purpose knowledge and then train on domain specific things to get the specialized knowledge to be

I think you would still need a large model to train on general-purpose knowledge and then train on domain specific things to get the specialized knowledge to be truly useful. For example, without a general-purpose model, if someone wanted domain-specific language "translated" for a neophyte, I doubt it would be able to without having also been trained on a general-purpose dataset