3 ms·

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lo

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rather expensive simultaniously. For a bit more expense, you can get more quality and lower running costs and much slower processing from buying a 128gb/192gb apple silicon setup and that's much much much slower than the "Turbo" services that OpenAI offers. I think the biggest thing pushing me away from OpenAI was they were subsidizing the chat experience much more than the API and this seems to reconcile that quite a bit. Quite simply OpenAI is sweetening the pot here too much for me to really ignore, this is a massively subsdizised service. I honestly don't feel the switching costs in the future will outweigh the benefits I'm getting now.

muttled 3y ago
For me personally, being able to fine-tune the local LLM's at a much higher rank and training more layers is very useful for (somewhat unreliably) embedding information. AFAIK the OpenAI fine-tuning is more geared towards formatting the output.
- bigfudge 3y ago
  As I understand it, fine tuning is never really about adding content. RAG and related techniques are likely cheaper/better if that’s what you want.
  - antupis 3y ago
    yup how I understand fine tuning is more about adding context and bigger picture. RAG is more about adding actual content. Good system probably needs both in long run.
swatcoder 3y ago
Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it. This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine tuning, and for competing hosted solutions are going to vastly different as months go by. What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now. That's what they're hoping you chase after, and if your product is defined by this technology, maybe that's what you have to do. But if you're just thinking about feature opportunities for a more robust product, judiciousness could pay off better than rushing. For now.
- layoric 3y ago
  "What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now." Just wanted to highlight this as such a great, concise way to look at the Buy vs Build with pretty much any cloud service, thanks!
  - startages 3y ago
    Seems right to me. That's why it's good to build with extendability in mind to allow switching easily in the future if needed.
- keyle 3y ago
  While I agree with you, as a happy GPT4 plus customer, I'm worried about the inevitable enshittification downhill roll that will eventually ensue. Once marketing gets in charge of product, it's doomed. And I can't think of a product startup that it hasn't happened to. Particularly with this type of growth, at some point, the suits start to out number the techies 10:1. This is why openeness and healthy competition is primordial.
  - ethbr1 3y ago
    It's not marketing, it's economics. If you set money on fire -- eventually there's a time when you need to stop doing that.
jpalomaki 3y ago
ChatGPT only costs a few dollars, but I'm also "paying" for the service by contributing training data to OpenAI. Getting access to this type of interaction data with (mostly) humans must be quite valuable asset.
mirekrusin 3y ago
Obviously it is good strategy, surely created from GPT.
smcleod 3y ago
A good strategy for who? Society? Customers? The future? Or just for making money for the owners?
s3p 3y ago
This isn't subsidized. OpenAI makes money on their API and ChatGPT pricing.

muttled 3y ago

For me personally, being able to fine-tune the local LLM's at a much higher rank and training more layers is very useful for (somewhat unreliably) embedding information. AFAIK the OpenAI fine-tuning is more geared towards formatting the output.

bigfudge 3y ago
As I understand it, fine tuning is never really about adding content. RAG and related techniques are likely cheaper/better if that’s what you want.
- antupis 3y ago
  yup how I understand fine tuning is more about adding context and bigger picture. RAG is more about adding actual content. Good system probably needs both in long run.

swatcoder 3y ago

Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it. This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine tuning, and for competing hosted solutions are going to vastly different as months go by. What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now. That's what they're hoping you chase after, and if your product is defined by this technology, maybe that's what you have to do. But if you're just thinking about feature opportunities for a more robust product, judiciousness could pay off better than rushing. For now.

layoric 3y ago
"What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now." Just wanted to highlight this as such a great, concise way to look at the Buy vs Build with pretty much any cloud service, thanks!
- startages 3y ago
  Seems right to me. That's why it's good to build with extendability in mind to allow switching easily in the future if needed.
keyle 3y ago
While I agree with you, as a happy GPT4 plus customer, I'm worried about the inevitable enshittification downhill roll that will eventually ensue. Once marketing gets in charge of product, it's doomed. And I can't think of a product startup that it hasn't happened to. Particularly with this type of growth, at some point, the suits start to out number the techies 10:1. This is why openeness and healthy competition is primordial.
- ethbr1 3y ago
  It's not marketing, it's economics. If you set money on fire -- eventually there's a time when you need to stop doing that.

jpalomaki 3y ago

ChatGPT only costs a few dollars, but I'm also "paying" for the service by contributing training data to OpenAI. Getting access to this type of interaction data with (mostly) humans must be quite valuable asset.

mirekrusin 3y ago

Obviously it is good strategy, surely created from GPT.

smcleod 3y ago

A good strategy for who? Society? Customers? The future? Or just for making money for the owners?

s3p 3y ago

This isn't subsidized. OpenAI makes money on their API and ChatGPT pricing.