3 ms·
In all the fast moving/low barrier technologies I've tracked over the years, competitors tend to leap frog each other. In this case, it's not happening despite
by xadhominemx 3y ago
In all the fast moving/low barrier technologies I've tracked over the years, competitors tend to leap frog each other. In this case, it's not happening despite scaled compute resources. Again, not conclusive but consistent with a slowdown.
- famouswaffles 3y agoYou've completely ignored what I just said. Competitors can leapfrog each other but it's not happening instantly. Gathering personnel, performing the necessary research, training, post-training red teaming. Each of these things take months of time and millions of dollars. That competitors are reaching GPT-4 level a year from it's release is perfectly normal and indicates nothing. Google released the paper for Gopher a year and a half after GPT-3 and that was never intended to be a product. >In all the fast moving/low barrier GPT-4 level compute is not low barrier. That's literally the main point of this paper.
- xadhominemx 3y agoIf 5+ companies can build it up in ~6 months, it's a low barrier
- famouswaffles 3y agoThere are only 2 models on GPT-4's level. 5+ companies? Lol
- xadhominemx 3y ago5+ companies with GPU clusters capable of training GPT-4 class models (as you said compute was the barrier)
- famouswaffles 3y agoWhat are the 5 companies? If most of them fall under, "the biggest and most valuable companies on the planet" then no, that's not low barrier.
- xadhominemx 3y agoGoogle Amazon Microsoft Meta Nvidia Oracle... if 6 companies can accomplish something in 6 months, that's low barrier. Expensive, but that's not the same as high barrier.