I prefer to use local models when running data extraction or processing over 10k or more records. Hosted services would be slow and brittle at this point. Mist

I prefer to use local models when running data extraction or processing over 10k or more records. Hosted services would be slow and brittle at this point. Mistral 7B fine-tunes (OpenChat is my favorite) just chug through the data and get the job done. Details: using vLLM to run the models. Using ChatGPT-4 to condense information for complex prompts (that the local models will execute). I think, the situation will just keep on getting better with each month.