4 ms·
It sure feels like everything after GPT-4 has just been an incremental improvement (putting aside multimodal capabilities and longer context windows) - GPT-4 Tu
by DebtDeflation 2y ago
It sure feels like everything after GPT-4 has just been an incremental improvement (putting aside multimodal capabilities and longer context windows) - GPT-4 Turbo, GPT-4o, o1.
GPT-3.5 with the ChatGPT front end was just such a massive improvement over everything that came before it, then 4 months later they release GPT-4 which was a significant improvement but more importantly gave the impression that progress was actually accelerating, then for the last 20 months we got incremental tweaks and new models that are a little better at certain tasks and a little worse at others.
- cma 2y agoGPT 3 released in 2020 though, 2.5 years before GPT 4, so 2.5 years to GPT 5 could be still same rate minus the RLHF breakthrough but maybe adding inference time additional reasoning of o1-preview plus more multimodal of 4o. So, chatgpt 3.5 potentially was to 3 as 4o and o1-preview are to 4, with 5 still to come, but it definitely could hit a data wall and maybe they lost some of the pirated book and scientific paper and textbook sources after scrubbing for the lawsuits, but they also have a lot more user interaction data and mass amounts of uploaded data from users that may not exist on the web. They are now ranked in the top 10 websites in the world or close and are popular especially among knowledge workers leaking all kinds of data to them.
- DebtDeflation 2y agoHuge difference between the original GPT-3 text-davinci released in 2020 and the 3.5 version that powered ChatGPT when it was released in late 2022.
- tiahura 2y agoData wall? If it had consumed everything already, then isn’t the problem the model, not the input?