4 ms·
It looks like OpenAI's first mover advantage won't last long, considering the speed at which these models are improving and compacting themselves. Seems like th
by seydor 4y ago
It looks like OpenAI's first mover advantage won't last long, considering the speed at which these models are improving and compacting themselves. Seems like this new 'Moore's law' will be even more steep, at least in the beginning . So steep, that we can hope to be running these in our desktops instead of on someone else's computer.
- sharemywin 4y agobut what kind of model will run in the cloud?
- eppp 4y agoWe could call it a weather model
- interestica 4y agoA model that augments the local? Runs above it? A supermodel?
- Tepix 4y agoI don't think Moore's law is the most important factor, instead algorithmic improvements will enable to have smaller models that are as capable as these humongeous models. Llama 65b is hopefully just the beginning of this trend. It outperforms OPT-175b.
- buildbot 4y agoThat holds true if scaling laws don’t hold true, otherwise a hypothetical Llama 175b would be even better. So the high end will always be on big clusters.
- deleted 4y ago[deleted]
- 2OEH8eoCRo0 4y ago'Moore's Law' was in quotes for a reason.
- sgk284 4y agoWe saw the same phenomenon with DALL-E. It was state-of-the-art for a blip in time before Midjourney and StableDiffusion surpassed it. And now with ControlNet + the ecosystem of domain-specific models, if you're doing serious generative art, OpenAI isn't even a part of the conversation. If OpenAI makes their revenue off of charging exorbitant fees to use the LLMs, they'll no longer have incentive to ever open them (even if abuse / misuse concerns are addressed) AND they'll have no incentive to make them more efficient. OpenAI has yet to show it can have a sustainable advantage. Every other player in the space benefits from an open model ecosystem and efficiency gains.
- l33tman 4y agoThey do pay for the compute time in some way or another so surely they have the same incentive as everybody else to make it more efficient, at least, to increase their margin. But yeah, they probably need to evolve from a (very good) two-trick pony into "the microsoft of AI" or something to stay afloat in the long run. That goes for the rest of the smaller AI companies as well though..
- andybak 4y agoMidjourney and StableDiffusion have not surpassed Dall-E for many types of image. They are still well behind in certain critical factors. I often jump to Dall-E when I hit a brick wall with SD. My money is on Deep Floyd (if and when it gets released)
- tmaly 4y agoWill be interesting to see what Nvidia H100s will do to the scene and what will come after them.