3 ms·

> I also don't get why there commiting so much to the future, are they sure of the quality of the products and their demand that much? It's one big game of mus

> I also don't get why there commiting so much to the future, are they sure of the quality of the products and their demand that much? It's one big game of musical chairs, and everyone can hear the phonograph slowing down. OpenAI is making these desperation plays because they've ran out of hype. GPT-5 "bombed", the wider public doesn't believe AI is going to keep getting exponentially better anymore. They're out of options to generate new hype beyond spewing ever larger numbers into the news cycle. AMD is making this desperation play because soon, once the AI bubble pops, there'll be a flood of cheap unused GPUs & GPU compute. Nobody's going to be buying their new cards when you can get Nvidia's prior gen for pennies on the dollar.

CyanLite2 1y ago
On the flip side of it (and where most institutional investors are mentally) is that if OpenAI is to ever achieve AGI, it must invest nearly a trillion dollars towards that effort. We all know LLMs have their limitations, but next phase of AI growth is going to come from OpenAI, Anthropic, Google, maybe even Microsoft, and not some stealth startup. E.g., Only Big Tech can get us to AGI due to sheer massive amounts of investments, not a traditional silicon valley garage startup looking for their Series A. So institutional investors have no choice but to continue to throw money into Big Tech hoping for the Big Payoff, rather than investing in VC funds like 10 years ago. AMD did this deal because it's literally offering financing to them. OpenAI doesn't have access to capital markets like AMD does. So it's selling off shares of its own stock to finance the purchase of billions of dollars worth of GPUs. And the trick appears to be working since the stock is up 30% today, meaning it has paid for itself and then some.
- rhetocj23 1y ago
  Theres a phrase for this. Financial engineering.
  - zerosizedweasle 1y ago
    In other words they are stealing capital from the rest of the economy. Starving it.
  - CuriouslyC 1y ago
    The difference this time is that it's global coordinated collusion, and it's not just the superwealthy, it's states that are willing to go all in on this. If you thought the banks were too big to fail, the result here is going to be a nationalization of AI resources and doubling down.
- jakewins 1y ago
  That “only big tech can solve AGI” bit doesn’t make sense to me - the scale argument was made back when people thought just more scale and more training was gonna keep yielding results. Now it seems clear that what’s missing is another architectural leap like transformers, likely many different ones. That could come from almost anywhere? Or what makes this something where big tech is the only potential source of innovation?
  - nemomarx 1y ago
    If it comes from anywhere else but it needs a lot of capital to execute, big tech will just acquire them right? They'll have all the data centers and compute contracts locked up I guess.
  - CuriouslyC 1y ago
    Yup. LLMs can get arbitrarily good at anything with RL, but RL produces spiky capabilities, and getting LLMs arbitrarily good at things they're not designed for (like reasoning, which is absolutely stupid to do in natural language) is very expensive due to the domain mismatch (as we're seeing in realtime). Neurosymbolic architectures are the future, but I think LLMs have a place as orchestrators and translators from natural language -> symbolic representation. I'm working on an article that lays out a pretty strong case for a lot of this based on ~30 studies, hopefully I can tighten it up and publish soon.
  - CyanLite2 1y ago
    The barrier of entry is too high for traditional SV startups or a group of folks with a good research idea like transformers. You now need hundreds of billions if not trillions to get access to compute. OpenAI themselves have cornered 40% of global output of DRAM modules. This isn't like 2012, where you could walk into your local BestBuy, get a laptop, open an AWS account, and start a SaaS over the weekend. Even the AI researchers themselves are commanding 7- and 8-figure salaries that rival NFL players. At best, they can sell their IP to BigTech, who will then commercialize it.
    - jakewins 1y ago
- N70Phone 1y ago
  > And the trick appears to be working since the stock is up 30% today, meaning it has paid for itself and then some. It's a bubble. The tricks keep working until they suddenly don't, and then all the prior tricks unwind themselves.
- chasd00 1y ago
  no amount of investment is going to make AGI just appear. It's looking more and more like current architectures are a dead end and then it's back to the AI drawing board just like the past 30 years.
programjames 1y ago
I find it funny how people say GPT-5 "bombed". I noticed a significant improvement in maths and coding with GPT-5. To quantify were I've found the models useful: - GPT 3.5: Good for finding reference terms. I could not trust anything it said, but it could help me find some general terms in fields I was unfamiliar with. - GPT 4: Good for cached, obscure knowledge. I generally could trust the stuff it said to be true, but none of its logic or conclusions. - GPT 4.5: Good for reference proofs/code. I cannot trust its proofs or code, but I can get a decent outline for writing my own. - GPT 5: Good for directed thinking. I cannot trust it to come up with the best solution on its own, but if I tell it what I'm working on, it's pretty decent at using all the tricks in its repertoire (across many fields) to get me a correct solution. I can trust its proofs or code to be about as correct as my own. My main issues are I cannot trust it to point out confusion or ask me, "is this actually the problem we should be solving here?" My guess is this is mostly a byproduct of shallow human feedback, rather than an actual issue with intelligence (as it will often ask me at the end of spending a bunch of computation if I want to try something mildly different). For me, GPT 5 is way more useful than the previous models, because I don't have a lot of paper-pushing problems I'm trying to solve. My guess is the wider public may disagree because it's hard to tell the difference between something better at the task than you, and something much better.
- CuriouslyC 1y ago
  Gemini 2.5 was the first breakthrough model, people didn't know how to use it but it's incredibly powerful. GPT5 is the second true breakthrough model, it's ability to deal with math/logic/etc complexity and its depth of knowledge in engineering/science is amazing. Every time I talk to someone who stans Claude and is down on GPT5 I know they're building derivative CRUD apps with simple business logic in Python/Typescript.
- N70Phone 1y ago
  > I find it funny how people say GPT-5 "bombed". I used scare quotes for a reason. It didn't "bomb" in the sense of failing [insert metric], it bombed in the sense that OpenAI needed it to generate exponentially more hype and it just didn't. (And on a lesser level, GPT-5 was supposed to cut OpenAI's costs but has failed to do so) > I can trust its proofs or code to be about as correct as my own. I have little to say about this, as I find such claims to be broadly irreplicable. GPT-5 scores better on the metrics, but still has the same "classes" of faults.

CyanLite2 1y ago

On the flip side of it (and where most institutional investors are mentally) is that if OpenAI is to ever achieve AGI, it must invest nearly a trillion dollars towards that effort. We all know LLMs have their limitations, but next phase of AI growth is going to come from OpenAI, Anthropic, Google, maybe even Microsoft, and not some stealth startup. E.g., Only Big Tech can get us to AGI due to sheer massive amounts of investments, not a traditional silicon valley garage startup looking for their Series A. So institutional investors have no choice but to continue to throw money into Big Tech hoping for the Big Payoff, rather than investing in VC funds like 10 years ago. AMD did this deal because it's literally offering financing to them. OpenAI doesn't have access to capital markets like AMD does. So it's selling off shares of its own stock to finance the purchase of billions of dollars worth of GPUs. And the trick appears to be working since the stock is up 30% today, meaning it has paid for itself and then some.

rhetocj23 1y ago
Theres a phrase for this. Financial engineering.
- zerosizedweasle 1y ago
  In other words they are stealing capital from the rest of the economy. Starving it.
- CuriouslyC 1y ago
  The difference this time is that it's global coordinated collusion, and it's not just the superwealthy, it's states that are willing to go all in on this. If you thought the banks were too big to fail, the result here is going to be a nationalization of AI resources and doubling down.
jakewins 1y ago
That “only big tech can solve AGI” bit doesn’t make sense to me - the scale argument was made back when people thought just more scale and more training was gonna keep yielding results. Now it seems clear that what’s missing is another architectural leap like transformers, likely many different ones. That could come from almost anywhere? Or what makes this something where big tech is the only potential source of innovation?
- nemomarx 1y ago
  If it comes from anywhere else but it needs a lot of capital to execute, big tech will just acquire them right? They'll have all the data centers and compute contracts locked up I guess.
- CuriouslyC 1y ago
  Yup. LLMs can get arbitrarily good at anything with RL, but RL produces spiky capabilities, and getting LLMs arbitrarily good at things they're not designed for (like reasoning, which is absolutely stupid to do in natural language) is very expensive due to the domain mismatch (as we're seeing in realtime). Neurosymbolic architectures are the future, but I think LLMs have a place as orchestrators and translators from natural language -> symbolic representation. I'm working on an article that lays out a pretty strong case for a lot of this based on ~30 studies, hopefully I can tighten it up and publish soon.
- CyanLite2 1y ago
  The barrier of entry is too high for traditional SV startups or a group of folks with a good research idea like transformers. You now need hundreds of billions if not trillions to get access to compute. OpenAI themselves have cornered 40% of global output of DRAM modules. This isn't like 2012, where you could walk into your local BestBuy, get a laptop, open an AWS account, and start a SaaS over the weekend. Even the AI researchers themselves are commanding 7- and 8-figure salaries that rival NFL players. At best, they can sell their IP to BigTech, who will then commercialize it.
  - jakewins 1y ago
N70Phone 1y ago
> And the trick appears to be working since the stock is up 30% today, meaning it has paid for itself and then some. It's a bubble. The tricks keep working until they suddenly don't, and then all the prior tricks unwind themselves.
chasd00 1y ago
no amount of investment is going to make AGI just appear. It's looking more and more like current architectures are a dead end and then it's back to the AI drawing board just like the past 30 years.

programjames 1y ago

I find it funny how people say GPT-5 "bombed". I noticed a significant improvement in maths and coding with GPT-5. To quantify were I've found the models useful: - GPT 3.5: Good for finding reference terms. I could not trust anything it said, but it could help me find some general terms in fields I was unfamiliar with. - GPT 4: Good for cached, obscure knowledge. I generally could trust the stuff it said to be true, but none of its logic or conclusions. - GPT 4.5: Good for reference proofs/code. I cannot trust its proofs or code, but I can get a decent outline for writing my own. - GPT 5: Good for directed thinking. I cannot trust it to come up with the best solution on its own, but if I tell it what I'm working on, it's pretty decent at using all the tricks in its repertoire (across many fields) to get me a correct solution. I can trust its proofs or code to be about as correct as my own. My main issues are I cannot trust it to point out confusion or ask me, "is this actually the problem we should be solving here?" My guess is this is mostly a byproduct of shallow human feedback, rather than an actual issue with intelligence (as it will often ask me at the end of spending a bunch of computation if I want to try something mildly different). For me, GPT 5 is way more useful than the previous models, because I don't have a lot of paper-pushing problems I'm trying to solve. My guess is the wider public may disagree because it's hard to tell the difference between something better at the task than you, and something much better.

CuriouslyC 1y ago
Gemini 2.5 was the first breakthrough model, people didn't know how to use it but it's incredibly powerful. GPT5 is the second true breakthrough model, it's ability to deal with math/logic/etc complexity and its depth of knowledge in engineering/science is amazing. Every time I talk to someone who stans Claude and is down on GPT5 I know they're building derivative CRUD apps with simple business logic in Python/Typescript.
N70Phone 1y ago
> I find it funny how people say GPT-5 "bombed". I used scare quotes for a reason. It didn't "bomb" in the sense of failing [insert metric], it bombed in the sense that OpenAI needed it to generate exponentially more hype and it just didn't. (And on a lesser level, GPT-5 was supposed to cut OpenAI's costs but has failed to do so) > I can trust its proofs or code to be about as correct as my own. I have little to say about this, as I find such claims to be broadly irreplicable. GPT-5 scores better on the metrics, but still has the same "classes" of faults.