5 ms·
People could tell scaling wasn't working well before the release of GPT 4.5 Who could tell? Who has tried scaling up to this level?
by kadushka 2y ago
People could tell scaling wasn't working well before the release of GPT 4.5
Who could tell? Who has tried scaling up to this level?
- throwaway314155 2y agohttps://www.reuters.com/technology/artificial-intelligence/openai-rivals-seek-new-path-smarter-ai-current-methods-hit-limitations-2024-11-11/ https://www.reuters.com/technology/artificial-intelligence/o... > Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that use s a vast amount of unlabeled data to understand language patterns and structures - have plateaued.
- anshumankmr 2y agoOpenAI took a bullet for the team, by perhaps scaling the model to something bigger than the 1.6T params GPT4 possibly had and basically telling its competitors its not gonna be worth scaling much beyond those number of params in GPT4, without a change in the model architecture