3 ms·
I'm highly skeptical that people are just going to leave the secret sauce out in the open for anyone to directly compete with them, feels a bit too idealistic t
by ianbutler 3y ago
I'm highly skeptical that people are just going to leave the secret sauce out in the open for anyone to directly compete with them, feels a bit too idealistic to me.
If you look at Mosaic, their value, their secret sauce, is the infrastructure. The model, to say it cynically, was marketing. Now it's true it can be marketing and help push open source forward at the same time.
More of a separate point, but I also think models are going to be fairly quickly commoditized on a couple years timescale, so the infrastructure to train a model or serve a model, or the thing built on the model is the value, but right now models are still the value because they're so new and we haven't finished (in fact we're really only starting) the race to the bottom.
- simonw 3y agoSure, OpenAI and Anthropic clearly have some secret sauce that they're holding back... but the rate at which the open source research community has been catching up is quite frankly astonishing. I would be very surprised if the closed source research teams can maintain their advantage over the next 12-24 months - especially given the number of enormous GPU clusters that are becoming available now.
- ianbutler 3y agoI don't want to diminish the efforts of the community, I agree the work is enormous and the strides are pretty crazy. I honestly hope I'm wrong. I was talking with a friend and discussing that this is one of the coolest collaborative periods I've seen in tech so far. Hopefully it plays out your way and not mine :)
- cced 3y agoWhat is secret about infrastructure?
- bugglebeetle 3y agoI’ve been testing the MPT models and they’re better than all the other open source stuff I’ve tried. I’m using my task-based, one-shot prompts that are about 1-2K tokens. Seems pretty on par with GPT-3.5. Marketing or whatever, first time I’ve been at all impressed.