3 ms·

> It's twice as fast and half the cost. There are plenty of usecases that just want the smartest possible AI, and could easily afford 1000x the price before pa

> It's twice as fast and half the cost. There are plenty of usecases that just want the smartest possible AI, and could easily afford 1000x the price before paying a human expert. How much would you pay a lawyer to write a letter for you? $200 maybe? Well GPT-4o will do it for $0.0025. Would you pay 1000x more ($2.50) for an AI which will shoot you with fewer legal footguns? - of course you would. Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds.

paulcole 2y ago
> Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds theres a massive gulf separating what people say they want and what people want. As well as a massive gulf separating what they will pay and what they say they’ll pay. while there are plenty of situations where you think you want the smartest possible ai, it’s clear that a pretty smart pretty quick one is good enough for a lot of things.
sebzim4500 2y ago
It could be that using prompt engineering/search techniques like CoT or ToT gives a bigger improvement than scaling up the model.
- waldrews 2y ago
  Exactly. In the long run we want a cheap/fast LLM layer that we can call millions of times in a search/conversation-with-itself pattern. It may be that we'll interact with it the same way, and GPT-6 will be lots of GPT-5's talking to each other transparently to the user, but the path to next-level intelligence needs to get past generating the next token. Since we don't know the real GPT-4 architecture, it may be doing some of this already.
  - sebzim4500 2y ago
    I think we can be pretty confident GPT-4 isn't doing something like this based on its performance characteristics. i) The complexity of the prompt/answer does not effect the time per token. ii) We start getting a response pretty quickly (for short prompts) and then get new tokens at a roughly constant rate I don't think either of these properties would hold if there were a bunch of models that had to coordinate before they could start writing the output. Certainly techniques like ToT violate them.
muyuu 2y ago
expanding to a high revenue but small niche is very challenging, and most importantly it's slow if some company is going to use a GPT version to do critical work, they'd be insane to jump on it before testing the solution and gaining confidence for a long time, while also evaluating how to exactly incorporate in their workflow without creating total dependency and other such ancillary issues you would pay 1000x more for the certainty, which you can never have right away - together with the legal liability of the service provider

paulcole 2y ago

> Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds theres a massive gulf separating what people say they want and what people want. As well as a massive gulf separating what they will pay and what they say they’ll pay. while there are plenty of situations where you think you want the smartest possible ai, it’s clear that a pretty smart pretty quick one is good enough for a lot of things.

sebzim4500 2y ago

It could be that using prompt engineering/search techniques like CoT or ToT gives a bigger improvement than scaling up the model.

waldrews 2y ago
Exactly. In the long run we want a cheap/fast LLM layer that we can call millions of times in a search/conversation-with-itself pattern. It may be that we'll interact with it the same way, and GPT-6 will be lots of GPT-5's talking to each other transparently to the user, but the path to next-level intelligence needs to get past generating the next token. Since we don't know the real GPT-4 architecture, it may be doing some of this already.
- sebzim4500 2y ago
  I think we can be pretty confident GPT-4 isn't doing something like this based on its performance characteristics. i) The complexity of the prompt/answer does not effect the time per token. ii) We start getting a response pretty quickly (for short prompts) and then get new tokens at a roughly constant rate I don't think either of these properties would hold if there were a bunch of models that had to coordinate before they could start writing the output. Certainly techniques like ToT violate them.

muyuu 2y ago

expanding to a high revenue but small niche is very challenging, and most importantly it's slow if some company is going to use a GPT version to do critical work, they'd be insane to jump on it before testing the solution and gaining confidence for a long time, while also evaluating how to exactly incorporate in their workflow without creating total dependency and other such ancillary issues you would pay 1000x more for the certainty, which you can never have right away - together with the legal liability of the service provider