4 ms·
I think I worded this poorly. What he said was that a lot of people say they want open-source models but they underestimate how hard it is to serve them well. S
by razcle 3y ago
I think I worded this poorly. What he said was that a lot of people say they want open-source models but they underestimate how hard it is to serve them well. So he wondered how much real benefit would come from open-sourcing them.
I think this is reasonable. Giving researchers access is great but for most small companies they're likely better off having a service provider manage inference for them rather than navigate the infra challenge.
- roganartu 3y agoThe beauty of open source is that the community will either figure out how to make it easier, or collectively decide it’s not worth the effort. We saw this with stable diffusion, and we are seeing it with all the existing OSS LLMs. “It’s too hard, trust us” doesn’t really make sense in that context. If it is indeed too hard for small orgs to self host then they won’t. Hiding behind the guise of protecting these people by not open sourcing it seems a bit disingenuous.
- rushingcreek 3y agoIf he says he's inclined to open-source GPT-3, I don't see any good arguments not in favor of giving startups the choice of how they can run inference.
- solarkraft 3y agoYou're saying the same thing. "I'm not sharing my chocolate with you because you probably wouldn't like it"
- choppaface 3y agoHere is how hard it is to serve and use LLMs: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
- doctor_eval 3y ago“The original implementation of llama.cpp was hacked in an evening.”
- antupis 3y agoIf it goes same way as other open sourced models it takes about 5 days that someone will get it running at m1.