3 ms·
Do you also have results of GPT4 somewhere? or text-davinci-003-turbo
by guywithabowtie 3y ago
Do you also have results of GPT4 somewhere? or text-davinci-003-turbo
- lhl 3y agoI'm still on the waitlist for GPT-4 API access. Note, that text-davinci-003 cost about $90 to benchmark at $0.02/1K tokens, so if you're able to use a GPT-4 model (for completion and not just instruction) that'll probably be $270-$540 in credits to benchmark...
- jimsimmons 3y agoJust think about benchmarking 32K GPT4 haha
- hhh 3y agoI have GPT-4 8k access and am willing to run the evals if someone wants to pay. Email in my acc info (the character is h) Just a note, I get errors semi-frequently when running queries against GPT-4 often (timeouts mostly…) so any code would need to handle that well.
- MacsHeadroom 3y agoYou should benchmark GPT-3 Curie (7B) for comparison since it is the same size as llama-7B and StableLM-7B. That will give us some indication of how much better these models are than GPT-3 at the same size.