4 ms·

I think the data will show that it is going to take exponential costs in either train-time or inference-time for linear or even sub-linear improvements. We're

I think the data will show that it is going to take exponential costs in either train-time or inference-time for linear or even sub-linear improvements. We're scraping the bottom of the ice-cream container. I expect a lot of benchmark shenanigans to conceal this like what we saw with o1. Most use-cases won't tolerate using orders of magnitude more time and tokens for marginal improvements. These companies are still valuable but the multiples will need to be reassessed.

itsdrewmiller 2y ago
I think you mean exponential - log costs for linear improvement would be incredibly good.
logicchains 2y ago
>I think the data will show that it is going to take logarithmic costs in either train-time or inference-time for linear or even sub-linear improvements. The data already shows this, it's a well-established result: https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361 . Well, less established for inference-time compute, but OpenAI even said they saw the same thing for inference compute in their release post for o1. But there's still room for at least an order of magnitude speedup with specialised hardware (as opposed to more general-purpose GPUs) and ternary nets.
airstrike 2y ago
This sounds like a reasonable take (exponential vs log discussion notwithstanding). It is also corroborated by their recent pitch to the White House for 5-gigawatt data centers. https://www.bloomberg.com/news/articles/2024-09-24/openai-pitched-white-house-on-unprecedented-data-center-buildout https://www.bloomberg.com/news/articles/2024-09-24/openai-pi... Posted here https://news.ycombinator.com/item?id=41642905 https://news.ycombinator.com/item?id=41642905 but it didn't get traction
- 93po 2y ago
  I think don't you can assume the massive data centers are an indication of the training needed or expected. I think it's also perfectly plausible that they're expecting massive demand of actual services, and as with most things, the larger scale you can do something, the lower the cost per user.
  - airstrike 2y ago
    We've seen lots of "massive demand" of software services, but none have historically required 5GW data centers
    - deleted 2y ago
      [deleted]
ratg13 2y ago
Isn’t that the plan? This is why they are planning what 5,6,7 data centers and re-opening 3-mile island. They will need more nuclear reactors as well to reach their goals.

itsdrewmiller 2y ago

I think you mean exponential - log costs for linear improvement would be incredibly good.

logicchains 2y ago

>I think the data will show that it is going to take logarithmic costs in either train-time or inference-time for linear or even sub-linear improvements. The data already shows this, it's a well-established result: https://arxiv.org/abs/2001.08361 https://arxiv.org/abs/2001.08361 . Well, less established for inference-time compute, but OpenAI even said they saw the same thing for inference compute in their release post for o1. But there's still room for at least an order of magnitude speedup with specialised hardware (as opposed to more general-purpose GPUs) and ternary nets.

airstrike 2y ago

This sounds like a reasonable take (exponential vs log discussion notwithstanding). It is also corroborated by their recent pitch to the White House for 5-gigawatt data centers. https://www.bloomberg.com/news/articles/2024-09-24/openai-pitched-white-house-on-unprecedented-data-center-buildout https://www.bloomberg.com/news/articles/2024-09-24/openai-pi... Posted here https://news.ycombinator.com/item?id=41642905 https://news.ycombinator.com/item?id=41642905 but it didn't get traction

93po 2y ago
I think don't you can assume the massive data centers are an indication of the training needed or expected. I think it's also perfectly plausible that they're expecting massive demand of actual services, and as with most things, the larger scale you can do something, the lower the cost per user.
- airstrike 2y ago
  We've seen lots of "massive demand" of software services, but none have historically required 5GW data centers
  - deleted 2y ago
    [deleted]

ratg13 2y ago

Isn’t that the plan? This is why they are planning what 5,6,7 data centers and re-opening 3-mile island. They will need more nuclear reactors as well to reach their goals.