3 ms·

I also think this will happen. Graphics cards with humongous RAM, etc. But I don't see how that changes the advantages that companies such as OpenAI have. Cons

by oezi 3y ago

I also think this will happen. Graphics cards with humongous RAM, etc. But I don't see how that changes the advantages that companies such as OpenAI have. Consumers will have access to GPT4-level LLMs, OpenAI will have a next gen offering.

yieldcrv 3y ago
A more efficient and faster GPT4-level LLM coming from a smaller model running in consumer hardware is going to be good enough. It will be competitive if it is easier to train on current events and keep updated, by anyone for nearly free. OpenAI's next gen offerings need to be something entirely different than smarter, and good for them if they do. All the multimodal capabilities have to be out, readily available, and stuff I haven't thought of yet. I think everyone's right, or "accurate" that OpenAI will have far more resources, I don't think its important.
- mycall 3y ago
  At the end of last year, OpenAI was talking about GPT-5 being a dead-end and something besides transformers needs to be made to do much better. We all wait for this.
- Nevermark 3y ago
  > OpenAI will have far more resources, I don't think its important. Who knows which direction they go, but I expect improvements like the following would obsolete GPT-4 and equivalents for most people: Multi-modal? I.e. text, audio, image, video, physics in video and interactive simulation, etc. the result will be major new features and a much deeper understanding of many topics. Audio and video input output? Engage with the model using voice, tone, gestures expressions and any artifacts you have: whiteboards, objects, whatever. Bring the model into conversations and meetings with other people to add its input. Long term memory, summarizing previous interactions, providing long term continuity? “Your” instance keeps up with you without a context limit. For days. Or months. Or longer. Have multiple instances for work and home. Share an instance with family, as a joint assistant. Live audio and long term memory would create the ultimate voice memo machine. Never forget anything again. An accurate confidence in its own accuracy? Clear distinction between answers backed by real information vs. confabulation. Built in ability to use the web and other tools, such as programming and executing calculations on the fly? Much wider access to up to date information and ability to come up with complex calculated answers in real time. Ability to manage your resources? Make sure bills are paid, proactively suggest ways to save, services to cancel. Manage your health by proactively suggesting & tracking changes, making appointments for troubling symptoms, etc. Remind you of tasks with all information you need proactively collected. Schedule one off services to solve problems, and follow up to be sure they were solved. Not only create a trip itinerary, but manage the whole trip, including intervening to solve inevitable problems. On and on here.
65a 3y ago
Will they? It's pretty clear that the trend is toward smaller but more efficient models, which further reduces their moat.
jasfi 3y ago
There's another type of player you shouldn't underestimate: companies that base their business model on hosting Open Source models. The consumers will pay far less than they do for GPT-4, and perhaps in a year or two, for sometimes that approaches the same quality. OpenAI will have to do everything they can to stay ahead of the pack. But they're an impressive company, and have a good chance of doing that.

by oezi 3y ago

yieldcrv 3y ago

A more efficient and faster GPT4-level LLM coming from a smaller model running in consumer hardware is going to be good enough. It will be competitive if it is easier to train on current events and keep updated, by anyone for nearly free. OpenAI's next gen offerings need to be something entirely different than smarter, and good for them if they do. All the multimodal capabilities have to be out, readily available, and stuff I haven't thought of yet. I think everyone's right, or "accurate" that OpenAI will have far more resources, I don't think its important.

mycall 3y ago
At the end of last year, OpenAI was talking about GPT-5 being a dead-end and something besides transformers needs to be made to do much better. We all wait for this.
Nevermark 3y ago
> OpenAI will have far more resources, I don't think its important. Who knows which direction they go, but I expect improvements like the following would obsolete GPT-4 and equivalents for most people: Multi-modal? I.e. text, audio, image, video, physics in video and interactive simulation, etc. the result will be major new features and a much deeper understanding of many topics. Audio and video input output? Engage with the model using voice, tone, gestures expressions and any artifacts you have: whiteboards, objects, whatever. Bring the model into conversations and meetings with other people to add its input. Long term memory, summarizing previous interactions, providing long term continuity? “Your” instance keeps up with you without a context limit. For days. Or months. Or longer. Have multiple instances for work and home. Share an instance with family, as a joint assistant. Live audio and long term memory would create the ultimate voice memo machine. Never forget anything again. An accurate confidence in its own accuracy? Clear distinction between answers backed by real information vs. confabulation. Built in ability to use the web and other tools, such as programming and executing calculations on the fly? Much wider access to up to date information and ability to come up with complex calculated answers in real time. Ability to manage your resources? Make sure bills are paid, proactively suggest ways to save, services to cancel. Manage your health by proactively suggesting & tracking changes, making appointments for troubling symptoms, etc. Remind you of tasks with all information you need proactively collected. Schedule one off services to solve problems, and follow up to be sure they were solved. Not only create a trip itinerary, but manage the whole trip, including intervening to solve inevitable problems. On and on here.

65a 3y ago

Will they? It's pretty clear that the trend is toward smaller but more efficient models, which further reduces their moat.

jasfi 3y ago

There's another type of player you shouldn't underestimate: companies that base their business model on hosting Open Source models. The consumers will pay far less than they do for GPT-4, and perhaps in a year or two, for sometimes that approaches the same quality. OpenAI will have to do everything they can to stay ahead of the pack. But they're an impressive company, and have a good chance of doing that.