3 ms·
> ...our experiment showed that fine-tuned GPT-3.5 models were significantly faster than [base GPT-3.5] by 3.6 to 3.76 times. does this pass the sniff test? un
by swyx 3y ago
> ...our experiment showed that fine-tuned GPT-3.5 models were significantly faster than [base GPT-3.5] by 3.6 to 3.76 times.
does this pass the sniff test? under what situations could a finetuned single tenant model be faster than a batchable multitenant base model?
- gnahzby 3y agoThis result really surprised us too, but it seems like we were not alone in observing this: https://medium.com/@gabrielgrinberg/openai-api-fine-tuned-gpt-3-5-vs-base-gpt-3-5-a-case-study-f3619b4f8cd8 https://medium.com/@gabrielgrinberg/openai-api-fine-tuned-gp... That being said, both accounts that we experimented on were personal accounts ( with a very limited budget) so maybe there are some special behaviors (e.g. some rate limit that only exists for base models)
- eqian99 3y agoYeah we were definitely surprised! This was a small scale experiment; we uploaded our notebook for latency tests here if anyone is interested taking a look at the code / trying it their own finetuned models: https://github.com/eqian99/finetuning-experiment https://github.com/eqian99/finetuning-experiment
- dnnssl2 3y agoUnder the same conditions where enterprise versions of the API have significantly less latency and better reliability than personal. OpenAI can change anything about the underlying infrastructure.