3 ms·
I’m building a side project app (that hopefully becomes a revenue generating SaaS, stay tuned) that will leverage AI to summarize lots of content at scale. My p
by dchuk 3y ago
I’m building a side project app (that hopefully becomes a revenue generating SaaS, stay tuned) that will leverage AI to summarize lots of content at scale. My plan is to just use OpenAI for now for speed to launch, but I have to imagine it will be way more economical and likely technically feasible to migrate to some self hosted LLM option later on.
Anyone else done this? Any tips/tricks?
- cyanydeez 3y agoassume OpenAI's business model is the same as Ubers and you're not getting the full price for the current offerings, and it'll quadruple in costs. Then ask if your product will survive that. If not, don't tie your wagon to their service without a secession plan.
- CityOfThrowaway 3y agoI think it'll be more like AWS, where the cost of each service keeps going down but they entice you into more usage and better models
- cyanydeez 3y agobetter is probably debatable. more tailored might be appropriate.
- icyfox 3y agoI've done some research involving parameterized summarization recently. GPT-4 does a really good (like, pretty outstanding) job in following the tone and density of a few one-shot example inputs. It's a lot more difficult to encourage the OSS models to do the same so you're going to have to experiment with a lot of different techniques to get it to what you want. In the worse case it could actually require some serious R&D investment to achieve the same results as an API out of the box. All that to say, my general philosophy is to worry about vendor costs / scale once you start approaching an adoption threshold where it matters. If the core idea doesn't have legs there's no point wasting time in developing your own summarization layer.