3 ms·

Pretty sure he said a few months ago that scaling further than GPT 4 isn't going to help enough, and that other breakthroughs are needed or something about buil

• ahefner 3y ago
  Were you limited by GPU capacity, would you intentionally downplay the potential benefit to further scaling the model so as to discourage competitors (e.g., Google) who would be less GPU-limited from pursuing such a direction themselves? I don't necessarily think this is the case, but it occurs to me that were there any practical impediment (however emphemeral) to them commercially exploiting a larger scale model, they'd likely adopt a posture of dismissing further scaling until they were ready.
  • whatyesaid 3y ago
    I heard that ChatGPT 4 runs in the red even with $20 subscription. Makes sense since only 25 messages every 4 hours. The nerf coincides with browsing taking more power up probably. Google probably doesn't want people using GPT 6 tech instead of search entirely, which is a real possibility since Google is often filled with SEO spam nowadays. I think they just want LLM + integrated with ads. Google had better results like 15 years ago for general information. Of course their nerfs are all intentional. When compute and techniques improve/scale they can unnerf. If they catch wind of a competing product they can also unnerf. They still have stuff developed they can't affordably deploy like multimodal, it can already take images.