It's probably covered many other machine learning books, but generally assumed to be common knowledge here: The final validation for what you need the applicati

It's probably covered many other machine learning books, but generally assumed to be common knowledge here: The final validation for what you need the application for should be done on your end. Need to use CRFs for NER? Test on your own validation set Need to use LLMs for NER? Test on your own validation set The validation set should be made up of carefully curated samples that you've seen errors on, edge cases, just like Test-driven Development, but at a much larger scale. Assessing LLMs by demo is a horrible habit that we've taken to that needs to be changed.