7 ms·

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increas

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on the API docs on all of these things, which have been updated just now: https://platform.openai.com/docs/api-reference https://platform.openai.com/docs/api-reference The pricing page has now updated as well: https://openai.com/pricing https://openai.com/pricing Notably, the DALL-E 3 API is $0.04 per image which is an order of magnitude above everyone else in the space. EDIT: One interesting observation with the new OpenAI pricing structure not mentioned during the keynote: finetuned ChatGPT 3.5 is now 3x of the cost of the base ChatGPT 3.5, down from 8x the cost. That makes finetuning a more compelling option.

visarga 3y ago
Mistral + 2 weeks of work from the community. Not as good, but private and free. It will trail OpenAI by 6-12 months in capabilities.
- coder543 3y ago
  OpenAI offering 128k context is very appealing, however. I tried some Mistral variants with larger context windows, and had very poor results… the model would often offer either an empty completion or a nonsensical completion, even though the content fit comfortably within the context window, and I was placing a direct question either at the beginning or end, and either with or without an explanation of the task and the content. Large contexts just felt broken. There are so many ways that we are more than “two weeks” from the open source solutions matching what OpenAI offers. And that’s to say nothing of how far behind these smaller models are in terms of accuracy or instruction following. For now, 6-12 months behind also isn’t good enough. In the uncertain case that this stays true, then a year from now the open models could be perfectly adequate for many use cases… but it’s very hard to predict the progression of these technologies.
  - pclmulqdq 3y ago
    Comparing a 7B parameter model to a 1.8T parameter model is kind of silly. Of course it's behind on accuracy, but it also takes 1% of the resources.
    - coder543 3y ago
      The person I replied to had decided to compare Mistral to what was launched, so I went along with their comparison and showed how I have been unsatisfied with it. But, these open models can certainly be fun to play with. Regardless, where did you find 1.8T for GPT-4 Turbo? The Turbo model is the one with the 128K context size, and the Turbo models tend to have a much lower parameter count from what people can tell. Nobody outside of OpenAI even knows how many parameters regular GPT-4 has. 1.8T is one of several guesses I have seen people make, but the guesses vary significantly. I’m also not convinced that parameter counts are everything, as your comment clearly implies, or that chinchilla scaling is fully understood. More research seems required to find the right balance: https://espadrine.github.io/blog/posts/chinchilla-s-death.html https://espadrine.github.io/blog/posts/chinchilla-s-death.ht...
      - danielmarkbruce 3y ago
        It's an order of magnitude comparison. Let's just agree it's 100x-300x more parameters, and let's assume the open ai folks are pretty smart and have a sense for the optimal number of tokens to train on.
        razodactyl 3y ago
        This definitely. Andrej Karpathy himself mentions tuned weight initialisation in one of his lectures. The TinyGPT code he wrote goes through it. Additionally explanations for the raw mathematics of log likelihoods and their loss ballparks. Interesting low-level stuff. These researchers are the best of the best working for the company that can afford them working on the best models available.
      - razodactyl 3y ago
spankalee 3y ago
A friend of mine is building Zep (https://www.getzep.com/ https://www.getzep.com/), which seems to offer a lot of the Assistant + Retrieval functionality in a self-hostable and model-agnostic way. That type of project may the way around lock-in.
davidbarker 3y ago
Also, DALL·E 3 "HD" is double the price at $0.08. I'm curious to play around with it once the API changes go live later today. The docs say: > By default, images are generated at standard quality, but when using DALL·E 3 you can set quality: "hd" for enhanced detail. Square, standard quality images are the fastest to generate. https://platform.openai.com/docs/guides/images/usage https://platform.openai.com/docs/guides/images/usage
- maranathapemje 3y ago
  This is great, but Dall.E still has a long way to go before reaching midjourney standards and I'm curious to see how they can pull that off.
faeriechangling 3y ago
It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rather expensive simultaniously. For a bit more expense, you can get more quality and lower running costs and much slower processing from buying a 128gb/192gb apple silicon setup and that's much much much slower than the "Turbo" services that OpenAI offers. I think the biggest thing pushing me away from OpenAI was they were subsidizing the chat experience much more than the API and this seems to reconcile that quite a bit. Quite simply OpenAI is sweetening the pot here too much for me to really ignore, this is a massively subsdizised service. I honestly don't feel the switching costs in the future will outweigh the benefits I'm getting now.
- muttled 3y ago
  For me personally, being able to fine-tune the local LLM's at a much higher rank and training more layers is very useful for (somewhat unreliably) embedding information. AFAIK the OpenAI fine-tuning is more geared towards formatting the output.
  - bigfudge 3y ago
    As I understand it, fine tuning is never really about adding content. RAG and related techniques are likely cheaper/better if that’s what you want.
    - antupis 3y ago
      yup how I understand fine tuning is more about adding context and bigger picture. RAG is more about adding actual content. Good system probably needs both in long run.
- swatcoder 3y ago
  Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it. This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine tuning, and for competing hosted solutions are going to vastly different as months go by. What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now. That's what they're hoping you chase after, and if your product is defined by this technology, maybe that's what you have to do. But if you're just thinking about feature opportunities for a more robust product, judiciousness could pay off better than rushing. For now.
ebiester 3y ago
I don't understand the lock-in argument here. Yes, if a competitor comes in there will be switching cost as everything is re-learned. However, from a code perspective, it is a function of the key and a relatively small API. New regulations outstanding, what is stoping someone from moving from OpenAI to Anthropic (for example) other than the cost of learning how to effectively utilize Anthropic for your use case? OpenAI doesn't have some sort of egress feed for your database.
- pclmulqdq 3y ago
  [flagged]
  - minimaxir 3y ago
    I most definitely am not paid by OpenAI and am very confused how my original (critical) comment could be seen as astroturfing.
  - airstrike 3y ago
    > Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - PH95VuimJjqBqy 3y ago
      likewise, stop posting these references to the guidelines, if you feel the guidelines are being broken report it to the moderators and move on. This type of post is actually more disruptive than the post you're replying to.
      - 0x0000000 3y ago
        > This type of post is actually more disruptive than the post you're replying to. As is yours. The author of that comment has contact information in their information, why not unicast them?
vsareto 3y ago
>The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. This just rings hollow to me. We lost the fights for database portability, cloud portability, payments/billing portability, and other individual SaaS lock-in. I don't see why it'll be different this time around.
- kortilla 3y ago
  > We lost the fights for database portability, cloud portability, payments/billing portability, and other individual SaaS lock-in. No we didn’t. There are viable on-prem alternatives or cross cloud alternatives for everything popular on the cloud. Many companies did choose to hand their destiny over to cloud providers but lots didn’t.
activescott 3y ago
I think it's more about finding places to add value than "lock in" per se. It seems they're adding value with improved developer experience and cost/performance rather than on the models themselves. Not necessarily nefarious attempts to lock in customers, but it may have the same outcome :)
Terretta 3y ago
> most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform OpenAI is currently refusing far more enterprises than these products could "lock-in" even with 100% stickiness. Makes it unlikely this is about lock-in or fighting churn when arguably, the best advertisement for GPT-4 is comparing its raw results to any other LLM. If you said their goal was fomenting FOMO, I'd buy it. Curious, though, when they'll let the FOMO fulfillment rate go up by accepting revenue for servicing that demand.
mvkel 3y ago
> Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'd argue the opposite. The new "Threads" interface in the OpenAI admin section lets you see exactly how it's interpreting input/output specifically to address the black box effect. Source: https://platform.openai.com/docs/api-reference/runs/listRunSteps https://platform.openai.com/docs/api-reference/runs/listRunS... tells you exactly how it's stepping through the chain. Even more visibility than there used to be.
- DerJacques 3y ago
  I agree that some parts of the process now seem more like “open”, but there is definitely a lot more magic in the new processing. Namely, threads can have an arbitrary length, and OpenAI automatically handles context window management for you. Their API now also handles retrieval of information from raw files, so you don’t need to worry about embeddings. Lastly, you don’t even need any sort of database to keep track of threads and messages. The API is now stateful! I think that most of these changes are exciting and make it a lot easier for people to get started. There is no doubt in my mind though that the API is now an even bigger blackbox, and lock-in is slightly increased depending on how you integrate with it.
  - mvkel 3y ago
    I wouldn't say the black box issue is unique to OpenAI. I suspect nobody could explain certain behaviors, including them. As for lock in, agreed completely.
  - kevindamm 3y ago
    Indeterminate context, unknown/hidden values and a stateful API are usually reasons for me to look elsewhere for a solution.
Vipsy 3y ago
Anything open about OpenAI starts and ends with the name
asaddhamani 3y ago
Their products are incredible though. I’ve tried the alternatives and even Claude is not nearly as good as even ChatGPT. Claude gives an ethics lecture with every second reply, which costs me money each time and makes their product very difficult to (want to) embed.
- ChatGTP 3y ago
  What are you using it for? I want to know what people actually use these things for damn it !
  - asaddhamani 3y ago
    I'd built a bot to use ChatGPT from Telegram (this was before the ChatGPT API), and currently building a tool to help make writing easier (https://www.penpersona.com https://www.penpersona.com). This is the API. Apart from that, it's pretty much replaced 80% of my search engine usage, I can ask it to collate reviews for a product from reddit and other sites, get the critical reception of a book, etc. You don't have to go and read long posts and articles, have GPT do it for you. There's many other use cases like this. For the second part, I'm using a UI called Typing Mind (which also works with the API).
    - toyg 3y ago
      > currently building a tool to help make writing easier That's cool! > it's pretty much replaced 80% of my search engine usage That's not cool. That's how you end up relying on nonexisting sources or other hallucinations.
      - Karunamon 3y ago
        As opposed to raw information surfaced by the search engine, which we all know is perfectly reliable, unbiased, and up to date? That aside, this particular admonishment was worn out a couple of months after ChatGPT was released. It does not need to be repeated every time someone mentions doing something interesting with an LLM.
stavros 3y ago
What do you mean "orders of magnitude above" for DALL-E? As far as I can see, Midjourney is $0.05 per image, and that's if you don't forget you have a subscription. I've ended up paying $10 per image.

visarga 3y ago

Mistral + 2 weeks of work from the community. Not as good, but private and free. It will trail OpenAI by 6-12 months in capabilities.

coder543 3y ago
OpenAI offering 128k context is very appealing, however. I tried some Mistral variants with larger context windows, and had very poor results… the model would often offer either an empty completion or a nonsensical completion, even though the content fit comfortably within the context window, and I was placing a direct question either at the beginning or end, and either with or without an explanation of the task and the content. Large contexts just felt broken. There are so many ways that we are more than “two weeks” from the open source solutions matching what OpenAI offers. And that’s to say nothing of how far behind these smaller models are in terms of accuracy or instruction following. For now, 6-12 months behind also isn’t good enough. In the uncertain case that this stays true, then a year from now the open models could be perfectly adequate for many use cases… but it’s very hard to predict the progression of these technologies.
- pclmulqdq 3y ago
  Comparing a 7B parameter model to a 1.8T parameter model is kind of silly. Of course it's behind on accuracy, but it also takes 1% of the resources.
  - coder543 3y ago
    The person I replied to had decided to compare Mistral to what was launched, so I went along with their comparison and showed how I have been unsatisfied with it. But, these open models can certainly be fun to play with. Regardless, where did you find 1.8T for GPT-4 Turbo? The Turbo model is the one with the 128K context size, and the Turbo models tend to have a much lower parameter count from what people can tell. Nobody outside of OpenAI even knows how many parameters regular GPT-4 has. 1.8T is one of several guesses I have seen people make, but the guesses vary significantly. I’m also not convinced that parameter counts are everything, as your comment clearly implies, or that chinchilla scaling is fully understood. More research seems required to find the right balance: https://espadrine.github.io/blog/posts/chinchilla-s-death.html https://espadrine.github.io/blog/posts/chinchilla-s-death.ht...
    - danielmarkbruce 3y ago
      It's an order of magnitude comparison. Let's just agree it's 100x-300x more parameters, and let's assume the open ai folks are pretty smart and have a sense for the optimal number of tokens to train on.
      - razodactyl 3y ago
        This definitely. Andrej Karpathy himself mentions tuned weight initialisation in one of his lectures. The TinyGPT code he wrote goes through it. Additionally explanations for the raw mathematics of log likelihoods and their loss ballparks. Interesting low-level stuff. These researchers are the best of the best working for the company that can afford them working on the best models available.
    - razodactyl 3y ago

spankalee 3y ago

A friend of mine is building Zep (https://www.getzep.com/ https://www.getzep.com/), which seems to offer a lot of the Assistant + Retrieval functionality in a self-hostable and model-agnostic way. That type of project may the way around lock-in.

davidbarker 3y ago

Also, DALL·E 3 "HD" is double the price at $0.08. I'm curious to play around with it once the API changes go live later today. The docs say: > By default, images are generated at standard quality, but when using DALL·E 3 you can set quality: "hd" for enhanced detail. Square, standard quality images are the fastest to generate. https://platform.openai.com/docs/guides/images/usage https://platform.openai.com/docs/guides/images/usage

maranathapemje 3y ago
This is great, but Dall.E still has a long way to go before reaching midjourney standards and I'm curious to see how they can pull that off.

faeriechangling 3y ago

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rather expensive simultaniously. For a bit more expense, you can get more quality and lower running costs and much slower processing from buying a 128gb/192gb apple silicon setup and that's much much much slower than the "Turbo" services that OpenAI offers. I think the biggest thing pushing me away from OpenAI was they were subsidizing the chat experience much more than the API and this seems to reconcile that quite a bit. Quite simply OpenAI is sweetening the pot here too much for me to really ignore, this is a massively subsdizised service. I honestly don't feel the switching costs in the future will outweigh the benefits I'm getting now.

muttled 3y ago
For me personally, being able to fine-tune the local LLM's at a much higher rank and training more layers is very useful for (somewhat unreliably) embedding information. AFAIK the OpenAI fine-tuning is more geared towards formatting the output.
- bigfudge 3y ago
  As I understand it, fine tuning is never really about adding content. RAG and related techniques are likely cheaper/better if that’s what you want.
  - antupis 3y ago
    yup how I understand fine tuning is more about adding context and bigger picture. RAG is more about adding actual content. Good system probably needs both in long run.
swatcoder 3y ago
Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it. This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine tuning, and for competing hosted solutions are going to vastly different as months go by. What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now. That's what they're hoping you chase after, and if your product is defined by this technology, maybe that's what you have to do. But if you're just thinking about feature opportunities for a more robust product, judiciousness could pay off better than rushing. For now.

ebiester 3y ago

I don't understand the lock-in argument here. Yes, if a competitor comes in there will be switching cost as everything is re-learned. However, from a code perspective, it is a function of the key and a relatively small API. New regulations outstanding, what is stoping someone from moving from OpenAI to Anthropic (for example) other than the cost of learning how to effectively utilize Anthropic for your use case? OpenAI doesn't have some sort of egress feed for your database.

pclmulqdq 3y ago
[flagged]
- minimaxir 3y ago
  I most definitely am not paid by OpenAI and am very confused how my original (critical) comment could be seen as astroturfing.
- airstrike 3y ago
  > Please don't post insinuations about astroturfing, shilling, brigading, foreign agents, and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - PH95VuimJjqBqy 3y ago
    likewise, stop posting these references to the guidelines, if you feel the guidelines are being broken report it to the moderators and move on. This type of post is actually more disruptive than the post you're replying to.
    - 0x0000000 3y ago
      > This type of post is actually more disruptive than the post you're replying to. As is yours. The author of that comment has contact information in their information, why not unicast them?

vsareto 3y ago

>The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. This just rings hollow to me. We lost the fights for database portability, cloud portability, payments/billing portability, and other individual SaaS lock-in. I don't see why it'll be different this time around.

kortilla 3y ago
> We lost the fights for database portability, cloud portability, payments/billing portability, and other individual SaaS lock-in. No we didn’t. There are viable on-prem alternatives or cross cloud alternatives for everything popular on the cloud. Many companies did choose to hand their destiny over to cloud providers but lots didn’t.

activescott 3y ago

I think it's more about finding places to add value than "lock in" per se. It seems they're adding value with improved developer experience and cost/performance rather than on the models themselves. Not necessarily nefarious attempts to lock in customers, but it may have the same outcome :)

Terretta 3y ago

> most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform OpenAI is currently refusing far more enterprises than these products could "lock-in" even with 100% stickiness. Makes it unlikely this is about lock-in or fighting churn when arguably, the best advertisement for GPT-4 is comparing its raw results to any other LLM. If you said their goal was fomenting FOMO, I'd buy it. Curious, though, when they'll let the FOMO fulfillment rate go up by accepting revenue for servicing that demand.

mvkel 3y ago

> Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'd argue the opposite. The new "Threads" interface in the OpenAI admin section lets you see exactly how it's interpreting input/output specifically to address the black box effect. Source: https://platform.openai.com/docs/api-reference/runs/listRunSteps https://platform.openai.com/docs/api-reference/runs/listRunS... tells you exactly how it's stepping through the chain. Even more visibility than there used to be.

DerJacques 3y ago
I agree that some parts of the process now seem more like “open”, but there is definitely a lot more magic in the new processing. Namely, threads can have an arbitrary length, and OpenAI automatically handles context window management for you. Their API now also handles retrieval of information from raw files, so you don’t need to worry about embeddings. Lastly, you don’t even need any sort of database to keep track of threads and messages. The API is now stateful! I think that most of these changes are exciting and make it a lot easier for people to get started. There is no doubt in my mind though that the API is now an even bigger blackbox, and lock-in is slightly increased depending on how you integrate with it.
- mvkel 3y ago
  I wouldn't say the black box issue is unique to OpenAI. I suspect nobody could explain certain behaviors, including them. As for lock in, agreed completely.
- kevindamm 3y ago
  Indeterminate context, unknown/hidden values and a stateful API are usually reasons for me to look elsewhere for a solution.

Vipsy 3y ago

Anything open about OpenAI starts and ends with the name

asaddhamani 3y ago

Their products are incredible though. I’ve tried the alternatives and even Claude is not nearly as good as even ChatGPT. Claude gives an ethics lecture with every second reply, which costs me money each time and makes their product very difficult to (want to) embed.

ChatGTP 3y ago
What are you using it for? I want to know what people actually use these things for damn it !
- asaddhamani 3y ago
  I'd built a bot to use ChatGPT from Telegram (this was before the ChatGPT API), and currently building a tool to help make writing easier (https://www.penpersona.com https://www.penpersona.com). This is the API. Apart from that, it's pretty much replaced 80% of my search engine usage, I can ask it to collate reviews for a product from reddit and other sites, get the critical reception of a book, etc. You don't have to go and read long posts and articles, have GPT do it for you. There's many other use cases like this. For the second part, I'm using a UI called Typing Mind (which also works with the API).
  - toyg 3y ago
    > currently building a tool to help make writing easier That's cool! > it's pretty much replaced 80% of my search engine usage That's not cool. That's how you end up relying on nonexisting sources or other hallucinations.
    - Karunamon 3y ago
      As opposed to raw information surfaced by the search engine, which we all know is perfectly reliable, unbiased, and up to date? That aside, this particular admonishment was worn out a couple of months after ChatGPT was released. It does not need to be repeated every time someone mentions doing something interesting with an LLM.

stavros 3y ago

What do you mean "orders of magnitude above" for DALL-E? As far as I can see, Midjourney is $0.05 per image, and that's if you don't forget you have a subscription. I've ended up paying $10 per image.