3 ms·

The results really aren’t striking enough that it’s clear that this model blows GPT-4 away. It seems roughly equivalent, give or take a bit. Why can we still n

• tempusalaria 3y ago
  Once you’ve taken all the data in the world and trained a sufficiently large model on it, it’s very hard to improve on that base. It’s possible that GPT-4 basically represents that benchmark, and improvements will require better parsing/tokenization, clever synthetic data methods, building expert datasets. Much harder than just scraping the internet and doing next token after some basic data cleaning.