GPT-4-base was completed in early 2022, its existence only went public much later. Whatever OpenAI has right now is a year and a half more advanced than current

GPT-4-base was completed in early 2022, its existence only went public much later. Whatever OpenAI has right now is a year and a half more advanced than current models. Since open source models can't even touch GPT-3.5 yet, they are very far behind. OpenAI has been cutting deals left and right with newspapers, publishers, and other organizations to train on vast high quality private datasets that simply are not available to open source development. Looking at how datasets are at least as important as volume of compute, we can expect the gap between OAI and open source models to continue to widen. None of this addresses the gigantic (possibly 1:1000) advantage services with constant workload have when it comes to memory bottleneck efficacy.