The example here isn't great, but the idea of using an ensemble of LLMs when compute is cheaper is cool. As the foundational models can parse super complex stu

The example here isn't great, but the idea of using an ensemble of LLMs when compute is cheaper is cool. As the foundational models can parse super complex stuff like dense human language, music, etc. with context - like a really good pre-built auto-encoder, which would be a nightmare with classic machine learning feature selection (remember bag of words? and word2vec?). I wonder how such an approach would compare to just fine-tuning one model though? And how the cost of fine-tuning vs. greater inference cost for an ensemble compares?