There is so much available in the open model world. Take a look at the Hugging Face Hub - there are 1000s of models that can be used as-is or as a starting poin

There is so much available in the open model world. Take a look at the Hugging Face Hub - there are 1000s of models that can be used as-is or as a starting point. And those models don't have to be LLMs. It's still a valid approach to use a smaller BERT model as a text classifier.