12 ms·

Same funding as OpenAI when they started, but SSI explicitly declared their intention not to release a single product until superintelligence is reached. Closes

Same funding as OpenAI when they started, but SSI explicitly declared their intention not to release a single product until superintelligence is reached. Closest thing we have to a Manhattan Project in the modern era?

paxys 2y ago
> Closest thing we have to a Manhattan Project in the modern era? Minus the urgency, scientific process, well-defined goals, target dates, public ownership, accountability...
- HPMOR 2y ago
  The Manhattan Project had none of these things publicly declared. And Ilya is a top flight scientist.
  - pclmulqdq 2y ago
    The word "publicly" is doing a lot of heavy lifting here. There is no indication that SSI has any of these at all.
- Quinner 2y ago
  If public ownership means we give one guy a button to end the world, I'm not sure how's that's a meaningful difference.
  - nativeit 2y ago
    We all get to vote for that person.
    - louthy 2y ago
      > all Some of you do. The rest of us are left with the consequences.
    - latexr 2y ago
      Well, not exactly “we all”, just the citizens of the country in possession of the kill switch. And in some countries, the person in question was either not elected or elections are a farce to keep appearances.
    - nwiswell 2y ago
      Oh, that's super. I've been really impressed recently with the wisdom of our collective selections.
      - riffraff 2y ago
        Democracy sucks, but we haven't found anything better.
        pnut 2y ago
        See section 5 https://www.schneier.com/blog/archives/2024/05/how-ai-will-change-democracy.html https://www.schneier.com/blog/archives/2024/05/how-ai-will-c...
  - vasco 2y ago
    No one single person can cause a nuclear detonation alone.
    - Quinner 2y ago
      The President of the United States has sole nuclear launch authority. To stop him would either take the cabinet and VP invoking the 25th amendment and removing him from office, or a military officer to disobey direct orders.
      - vasco 2y ago
        Are you under the impression the president can actually do it? It's not true, someone else at least needs to at least push another button. I'm 100% sure of what I said in regards to the USA, just not hidden nuke programs I wouldn't know about. No person in the USA can single handedly trigger a nuclear weapon launch. What he has authority to do is ask someone else to launch a nuke, and that person will then need to decide to do it. Even the president needs someone else to push a button (and in those rooms there's also more than one person). There's literally no human that can do it alone without convincing at least 1 or 2 other people, depending on who it is.
  - wil421 2y ago
    Pretty sure the military made it clear they aren’t launching any nukes, despite what the last President said publicly. They also made it clear they weren’t invading China. https://amp.cnn.com/cnn/2017/11/18/politics/air-force-general-john-hyten-nuclear-strike-donald-trump https://amp.cnn.com/cnn/2017/11/18/politics/air-force-genera... https://www.bbc.com/news/world-us-canada-58581296.amp https://www.bbc.com/news/world-us-canada-58581296.amp
  - paxys 2y ago
    The fact that the world hasn't ended and no nuke has been launched since the 1940s shows that the system is working. Give the button to a random billionaire and half of us will be dead by next week to improve profit margins.
    - vunderba 2y ago
      Bikini atoll and the islanders that no longer live there due to nuclear contamination would like a word with you. Split hairs however you like with the definition of "launch" but those tests went on well through the 1950s.
      - __loam 2y ago
        A nuclear weapon hasn't been detonated in anger since 1945.
- whimsicalism 2y ago
  none of these things are true of public knowledge about the manhattan project… but oookay
- digging 2y ago
  Interesting attributes to mention... The urgency was faked and less true of the Manhattan Project than it is of AGI safety. There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest, saying it was unnecessary and dangerous. However, the race to develop AGI is very real, and we also have no way of knowing how close anyone is to reaching it. Likewise, the target dates were pretty meaningless. There was no race, and the atomic bombs weren't necessary to end the war with Japan either. (It can't be said with certainty one way or the other, but there's pretty strong evidence that their existence was not the decisive factor in surrender.) Public ownership and accountability are also pretty odd things to say! Congress didn't even know about the Manhattan Project. Even Truman didn't know for a long time. Sure, it was run by employees of the government and funded by the government, but it was a secret project with far less public input than any US-based private AI companies today.
  - subsubzero 2y ago
    I agree and also disagree. > There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest You are forgetting Japan in WWII and given casualty numbers from island hopping it was going to be a absolutely huge casualty count with US troops, probably something on the order of Englands losses during WW1. Which for them sent them on a downward trajectory due to essentially an entire generation dying or being extremely traumatized. If the US did not have Nagasaki and Hiroshima we would probably not have the space program and US technical prowess post WWII, so a totally different reality than where we are today.
    - digging 2y ago
      Did you stop reading my comment there? I debunked this already.
      - koops 2y ago
        Asserting that there is strong evidence against a claim is not "debunking" a claim.
- chakintosh 2y ago
  ... Hiroshima
  - zombiwoof 2y ago
    And Nagasaki , not once but twice. Why? Just because
    - lazide 2y ago
      Once could be a fluke, twice sends an entirely different message.
      - chakintosh 2y ago
        Even once is a -fluke- war crime.
        lazide 2y ago
        Oppsie daisy. Still, you’d have to be quite an idiot to wait for the third time to listen eh? Besides, the winners get to decide what’s a war crime or not. And when the US started mass firebombing civilian Tokyo, it’s not like they were going to be able to just ‘meh, we’re good’ on that front. Compared to that hell, being nuked was humane. And I don’t say that lightly.
        chakintosh 2y ago
        By that point, Japan was already on its way out and resorted to flying manned bombs and airplanes into american warships. Nuking Japan wasn't for Japan, it was a show of force for the soviets who were developing their own nukes.
        lazide 2y ago
        Neutralizing Japan the rest of the way would have cost millions of additional American lives, at a minimum. Japan was never going to surrender unless they saw the axe swinging for their neck, and knew they couldn’t dodge. They didn’t care about their own civilians. As made quite apparent by, as you note, kamikaze tactics and more. The Bomb was a cleaner, sharper, and faster Axe than invading the main island. That it also sent a message to the rest of the world was a bonus. But do you think they would have not used it, if for example the USSR wasn’t waiting? Of course not, they’d still have nuked the hell out of the Japanese.
- alexilliamson 2y ago
  Well-defined goal is the big one. We wanted a big bomb. What does AGI do? AGI is up against a philosophical barrier, not a technical one. We'll continue improving AI's ability to automate and assist human decisions, but how does it become something more? Something more "general"?
  - lucubratory 2y ago
    "General" is every activity a human can do or learn to do. It was coined along with "narrow" to contrast with the then decidedly non-general AI systems. This was generally conceived of as a strict binary - every AI we've made is narrow, whereas humans are general, able to do a wide variety of tasks and do things like transfer learning, and the thinking was that we were missing some grand learning algorithm that would create a protointelligence which would be "general at birth" like a human baby, able to learn anything & everything in theory. An example of an AI system that is considered narrow is a calculator, or a chess engine - these are already superhuman in intelligence, in that they can perform their tasks better than any human ever possibly could, but a calculator or a chess engine is so narrow that it seems absurd to think of asking a calculator for an example of a healthy meal plan, or asking a chess engine to make sense of an expense report, or asking anything to write a memoir. Even in more modern times, with AlexNet we had a very impressive image recognition AI system, but it couldn't calculate large numbers or win a game of chess or write poetry - it was impressive, but still narrow. With transformers, demonstrated first by LLMs, I think we've shown that the narrow-general divide as a strict binary is the wrong way to think about AI. Instead, LLMs are obviously more general than any previous AI system, in that they can do math or play chess or write a poem, all using the same system. They aren't as good as our existing superhuman computer systems at these tasks (aside from language processing, which they are SOTA at), not even as good at humans, but they're obviously much better than chance. With training to use tools (like calculators and chess engines) you can easily make an AI system with an LLM component that's superhuman in those fields, but there are still things that LLMs cannot do as well as humans, even when using tools, so they are not fully general. One example is making tools for themselves to use - they can do a lot of parts of that work, but I haven't seen an example yet of an LLM actually making a tool for itself that it can then use to solve a problem it otherwise couldn't. This is a subproblem of the larger "LLMs don't have long term memory and long term planning abilities" problem - you can ask an LLM to use python to make a little tool for itself to do one specific task, but it's not yet capable of adding that tool to its general toolset to enhance its general capabilities going forward. It can't write a memoir, or a book that people want to read, because they suck at planning or refining from drafts, and they have limited creativity because they're typically a blank slate in terms of explicit memory before they're asked to write - they have a gargantuan of implicitly remembered things from training, which is where what creativity they do have comes from, but they don't yet have a way to accrue and benefit from experience. A thought exercise I think is helpful for understanding what the "AGI" benchmark should mean is: can this AI system be a drop-in substitute for a remote worker? As in, any labour that can be accomplished by a remote worker can be performed by it, including learning on the job to do different or new tasks, and including "designing and building AI systems". Such a system would be extremely economically valuable, and I think it should meet the bar of "AGI".
- berz01 2y ago
  nailed it bro, someone give this man a podium
Yizahi 2y ago
There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks. So it is closer to some tokenbro project than to nuclear research. Or he will simply shift goalposts, and call some LLM superintelligent.
- liminvorous 2y ago
  No one had built a nuclear bomb before the Manhattan project either.
  - Yizahi 2y ago
    Theoretical foundation was slowly built over decades before it started though. And correct me if I'm wrong, but calculations that it was feasible were present before the start too. They had to calculate how to do it, what will be the processes, how to construct it and so on, but theoretically scientists knew that this amount of material can start such process. On the other hand not only there is no clear path to AI today (also known as AGI, ASI, SI etc.), but even foundations are largely missing. We are debating what is intelligence, how it works, how to even start simulating it, or construct from scratch.
    - Vecr 2y ago
      There are algorithms that should work, they're just galactic[0] or are otherwise expected to use far too much space and time to be practical. [0]: https://en.wikipedia.org/wiki/Galactic_algorithm https://en.wikipedia.org/wiki/Galactic_algorithm
      - ryan93 2y ago
        That wiki article has nothing to do with AI. The whole AI space attracts BS talk
        Vecr 2y ago
        What do you think AI is? On that one page there's simulated annealing with a logarithmic cooling schedule, Hutter search, and Solomonoff induction, all very much applicable to AI. If you want a fully complete galactic algorithm for AI, look up AIXItl. Edit: actually I'm not sure if AIXItl is technically galactic or just terribly inefficient, but there's been trouble making it faster and more compact.
swader999 2y ago
Both need a crap tonne of electricity.
torginus 2y ago
A non-cynical take is that Ilya wanted to do research without the pressure of having to release a marketable product and figuring out how to monetize their technology, which is why he left OpenAI. A very cynical take is that this is an extreme version of 'we plan to spend all money on growth and figure out monetization later' model that many social media companies with a burn rate of billions of $$, but no business model, have used.
- layer8 2y ago
  That’s not a cynical take, it’s the obvious take.
- signatoremo 2y ago
  He was on the record that their first product will be a safe superintelligence and it won’t do anything else until then, which sounds like they won’t have paid customers until they can figure out how to build a superintelligent model. That’s certainly a lofty goal and a very long term play.
  - lupire 2y ago
    OpenAI was "on the record" with a lot of obsolete claims too. Money changes people.
    - signatoremo 2y ago
      He didn’t promise world peace nor did he claim his work belongs to the humanity. The company is still a for profit corporation. He is saying he will try to build something head and shoulders above anything else, and he got a billion dollars to do it with no expectation of revenue until his product is ready. The likelihood that he fails is very high, but his backers are willing to bet on that.
jchonphoenix 2y ago
OpenAI initially raised 50m in their institutional round. 1b was a non profit donation, so there wasn't an expectation of returns on that one.
apwell23 2y ago
> superintelligence is reached i read the article but I am not sure how they know when this condition will be true. Is this obvious to ppl reading this article? is it emperor has no clothes type situation ?
- Propelloni 2y ago
  You are not alone. This is the litmus test many people are contemplating for a long time now, mostly philosophers, which is not surprising since it is a philosophical question. Most of the heavy stuff is hidden behind paywalls, but here's a nice summary of the state of the art by two CS guys: https://arxiv.org/pdf/2212.06721 https://arxiv.org/pdf/2212.06721
- crappybird 2y ago
  They can dilute the term to whatever they want. I think when the pressure to release becomes too high, they can just stick a patch of "Superintelligence™" on their latest LLM and release it.
  - apwell23 2y ago
    what do you make of ppl commenting here saying 'well they won't release till superintelligence' . Are these ppl merely gullible or coconspirators in the scam ?
    - sensanaty 2y ago
      There's billions (with a B), probably closing in on Trillions, riding on the AI hypewave. This forum is full of VCs and VC-adjacent people who have vested interests in AI companies blowing up and being successful, regardless of if it's actually useful. If you check the 2024 YC batch, you'll notice pretty much every single one of them mentions AI in some form or another. I guarantee you the large majority of them are just looking to be bought out by some megacorp, because it's free money right now.
TrackerFF 2y ago
To my ears, it's more like a ambitious pharma project. There's plenty of players going for the same goal. R&D is wildly expensive. No guarantee they'll reach the goal, first or even at all.
choppaface 2y ago
Could be more comparable to Clubhouse, which VCs quickly piled $100m into[1a], and which Clubhouse notably turned into layoffs [1b]. In this case, the $1b in funding and high valuation might function predominantly as a deterrent to any flippers (in contrast, many Clubhouse investors got quick gains). Moreover, the majority of the capital likely goes into GPU hardware and/or opex, which VCs have currently arbitraged themselves [3], so to some extent this is VCs literally paying themselves to pay off their own hardware bet. While hints of the ambition of the Manhattan project might be there, the economics really are not. [1a] https://www.getpin.xyz/post/clubhouse-lessons-for-investors https://www.getpin.xyz/post/clubhouse-lessons-for-investors [1b] https://www.theverge.com/2023/4/27/23701144/clubhouse-layoffs-half-employees https://www.theverge.com/2023/4/27/23701144/clubhouse-layoff... [3] https://observer.com/2024/07/andreessen-horowitz-stocking-ai-chips-win-deals-gpu-shortage/ https://observer.com/2024/07/andreessen-horowitz-stocking-ai...
janalsncm 2y ago
> Closest thing we have to a Manhattan Project in the modern era? No. The Manhattan Project started after we understood the basic mechanism of runaway fission reactions. The funding was mostly spent purifying uranium. AGI would be similar if we understood the mechanism of creating general intelligence and just needed to scale it up. But there are still fundamental questions we still aren’t close to understanding for AGI. A more apt comparison today is probably something like fusion reactors although progress has been slow there too. We know how fusion works in theory. We have done it before (thermonuclear weapons). There are sub-problems we need to solve, but people are working on them. For AGI we don’t even know what the sub-problems are yet.
Runsthroughit 2y ago
Literally everyone from OpenAI lied 100% about everything of substance. Sutskever lied about "world model" inside of LLMs, which is such a despicable lie, because he knows that "latent space" is a TOTAL MESS. Proven everytime anyone looked at it. Shameless grifters. When end?
m3kw9 2y ago
Super Intelligence, even OpenAI when getting investment from Microsoft, OpenAI won’t have to share its “AGI” model to them and it is up to OpenAI to define what that is and who the heck knows how they will define it. The point is that that phrase is the most ambiguous word in tech right now and almost everyone thinks what’s in their head is AGI, some will think Skynet, some will think enough reason ability, some will think god like undecipherable logic and everywhere in between
porti 2y ago
[dead]
troupo 2y ago
No. It's the next Magic Leap of our era. Or the next Juicero of our era. Or the next any of the hundreds of unprofitable startups losing billions of dollars a year without any business plan beyond VC subsidies and a hope for an exit of our era.

paxys 2y ago

> Closest thing we have to a Manhattan Project in the modern era? Minus the urgency, scientific process, well-defined goals, target dates, public ownership, accountability...

HPMOR 2y ago
The Manhattan Project had none of these things publicly declared. And Ilya is a top flight scientist.
- pclmulqdq 2y ago
  The word "publicly" is doing a lot of heavy lifting here. There is no indication that SSI has any of these at all.
Quinner 2y ago
If public ownership means we give one guy a button to end the world, I'm not sure how's that's a meaningful difference.
- nativeit 2y ago
  We all get to vote for that person.
  - louthy 2y ago
    > all Some of you do. The rest of us are left with the consequences.
  - latexr 2y ago
    Well, not exactly “we all”, just the citizens of the country in possession of the kill switch. And in some countries, the person in question was either not elected or elections are a farce to keep appearances.
  - nwiswell 2y ago
    Oh, that's super. I've been really impressed recently with the wisdom of our collective selections.
    - riffraff 2y ago
      Democracy sucks, but we haven't found anything better.
      - pnut 2y ago
        See section 5 https://www.schneier.com/blog/archives/2024/05/how-ai-will-change-democracy.html https://www.schneier.com/blog/archives/2024/05/how-ai-will-c...
- vasco 2y ago
  No one single person can cause a nuclear detonation alone.
  - Quinner 2y ago
    The President of the United States has sole nuclear launch authority. To stop him would either take the cabinet and VP invoking the 25th amendment and removing him from office, or a military officer to disobey direct orders.
    - vasco 2y ago
      Are you under the impression the president can actually do it? It's not true, someone else at least needs to at least push another button. I'm 100% sure of what I said in regards to the USA, just not hidden nuke programs I wouldn't know about. No person in the USA can single handedly trigger a nuclear weapon launch. What he has authority to do is ask someone else to launch a nuke, and that person will then need to decide to do it. Even the president needs someone else to push a button (and in those rooms there's also more than one person). There's literally no human that can do it alone without convincing at least 1 or 2 other people, depending on who it is.
- wil421 2y ago
  Pretty sure the military made it clear they aren’t launching any nukes, despite what the last President said publicly. They also made it clear they weren’t invading China. https://amp.cnn.com/cnn/2017/11/18/politics/air-force-general-john-hyten-nuclear-strike-donald-trump https://amp.cnn.com/cnn/2017/11/18/politics/air-force-genera... https://www.bbc.com/news/world-us-canada-58581296.amp https://www.bbc.com/news/world-us-canada-58581296.amp
- paxys 2y ago
  The fact that the world hasn't ended and no nuke has been launched since the 1940s shows that the system is working. Give the button to a random billionaire and half of us will be dead by next week to improve profit margins.
  - vunderba 2y ago
    Bikini atoll and the islanders that no longer live there due to nuclear contamination would like a word with you. Split hairs however you like with the definition of "launch" but those tests went on well through the 1950s.
    - __loam 2y ago
      A nuclear weapon hasn't been detonated in anger since 1945.
whimsicalism 2y ago
none of these things are true of public knowledge about the manhattan project… but oookay
digging 2y ago
Interesting attributes to mention... The urgency was faked and less true of the Manhattan Project than it is of AGI safety. There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest, saying it was unnecessary and dangerous. However, the race to develop AGI is very real, and we also have no way of knowing how close anyone is to reaching it. Likewise, the target dates were pretty meaningless. There was no race, and the atomic bombs weren't necessary to end the war with Japan either. (It can't be said with certainty one way or the other, but there's pretty strong evidence that their existence was not the decisive factor in surrender.) Public ownership and accountability are also pretty odd things to say! Congress didn't even know about the Manhattan Project. Even Truman didn't know for a long time. Sure, it was run by employees of the government and funded by the government, but it was a secret project with far less public input than any US-based private AI companies today.
- subsubzero 2y ago
  I agree and also disagree. > There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest You are forgetting Japan in WWII and given casualty numbers from island hopping it was going to be a absolutely huge casualty count with US troops, probably something on the order of Englands losses during WW1. Which for them sent them on a downward trajectory due to essentially an entire generation dying or being extremely traumatized. If the US did not have Nagasaki and Hiroshima we would probably not have the space program and US technical prowess post WWII, so a totally different reality than where we are today.
  - digging 2y ago
    Did you stop reading my comment there? I debunked this already.
    - koops 2y ago
      Asserting that there is strong evidence against a claim is not "debunking" a claim.
chakintosh 2y ago
... Hiroshima
- zombiwoof 2y ago
  And Nagasaki , not once but twice. Why? Just because
  - lazide 2y ago
    Once could be a fluke, twice sends an entirely different message.
    - chakintosh 2y ago
      Even once is a -fluke- war crime.
      - lazide 2y ago
        Oppsie daisy. Still, you’d have to be quite an idiot to wait for the third time to listen eh? Besides, the winners get to decide what’s a war crime or not. And when the US started mass firebombing civilian Tokyo, it’s not like they were going to be able to just ‘meh, we’re good’ on that front. Compared to that hell, being nuked was humane. And I don’t say that lightly.
        chakintosh 2y ago
        By that point, Japan was already on its way out and resorted to flying manned bombs and airplanes into american warships. Nuking Japan wasn't for Japan, it was a show of force for the soviets who were developing their own nukes.
        lazide 2y ago
        Neutralizing Japan the rest of the way would have cost millions of additional American lives, at a minimum. Japan was never going to surrender unless they saw the axe swinging for their neck, and knew they couldn’t dodge. They didn’t care about their own civilians. As made quite apparent by, as you note, kamikaze tactics and more. The Bomb was a cleaner, sharper, and faster Axe than invading the main island. That it also sent a message to the rest of the world was a bonus. But do you think they would have not used it, if for example the USSR wasn’t waiting? Of course not, they’d still have nuked the hell out of the Japanese.
alexilliamson 2y ago
Well-defined goal is the big one. We wanted a big bomb. What does AGI do? AGI is up against a philosophical barrier, not a technical one. We'll continue improving AI's ability to automate and assist human decisions, but how does it become something more? Something more "general"?
- lucubratory 2y ago
  "General" is every activity a human can do or learn to do. It was coined along with "narrow" to contrast with the then decidedly non-general AI systems. This was generally conceived of as a strict binary - every AI we've made is narrow, whereas humans are general, able to do a wide variety of tasks and do things like transfer learning, and the thinking was that we were missing some grand learning algorithm that would create a protointelligence which would be "general at birth" like a human baby, able to learn anything & everything in theory. An example of an AI system that is considered narrow is a calculator, or a chess engine - these are already superhuman in intelligence, in that they can perform their tasks better than any human ever possibly could, but a calculator or a chess engine is so narrow that it seems absurd to think of asking a calculator for an example of a healthy meal plan, or asking a chess engine to make sense of an expense report, or asking anything to write a memoir. Even in more modern times, with AlexNet we had a very impressive image recognition AI system, but it couldn't calculate large numbers or win a game of chess or write poetry - it was impressive, but still narrow. With transformers, demonstrated first by LLMs, I think we've shown that the narrow-general divide as a strict binary is the wrong way to think about AI. Instead, LLMs are obviously more general than any previous AI system, in that they can do math or play chess or write a poem, all using the same system. They aren't as good as our existing superhuman computer systems at these tasks (aside from language processing, which they are SOTA at), not even as good at humans, but they're obviously much better than chance. With training to use tools (like calculators and chess engines) you can easily make an AI system with an LLM component that's superhuman in those fields, but there are still things that LLMs cannot do as well as humans, even when using tools, so they are not fully general. One example is making tools for themselves to use - they can do a lot of parts of that work, but I haven't seen an example yet of an LLM actually making a tool for itself that it can then use to solve a problem it otherwise couldn't. This is a subproblem of the larger "LLMs don't have long term memory and long term planning abilities" problem - you can ask an LLM to use python to make a little tool for itself to do one specific task, but it's not yet capable of adding that tool to its general toolset to enhance its general capabilities going forward. It can't write a memoir, or a book that people want to read, because they suck at planning or refining from drafts, and they have limited creativity because they're typically a blank slate in terms of explicit memory before they're asked to write - they have a gargantuan of implicitly remembered things from training, which is where what creativity they do have comes from, but they don't yet have a way to accrue and benefit from experience. A thought exercise I think is helpful for understanding what the "AGI" benchmark should mean is: can this AI system be a drop-in substitute for a remote worker? As in, any labour that can be accomplished by a remote worker can be performed by it, including learning on the job to do different or new tasks, and including "designing and building AI systems". Such a system would be extremely economically valuable, and I think it should meet the bar of "AGI".
berz01 2y ago
nailed it bro, someone give this man a podium

Yizahi 2y ago

There is significant possibility that true AI (what Ilia calls superintelligence) is impossible to build using neural networks. So it is closer to some tokenbro project than to nuclear research. Or he will simply shift goalposts, and call some LLM superintelligent.

liminvorous 2y ago
No one had built a nuclear bomb before the Manhattan project either.
- Yizahi 2y ago
  Theoretical foundation was slowly built over decades before it started though. And correct me if I'm wrong, but calculations that it was feasible were present before the start too. They had to calculate how to do it, what will be the processes, how to construct it and so on, but theoretically scientists knew that this amount of material can start such process. On the other hand not only there is no clear path to AI today (also known as AGI, ASI, SI etc.), but even foundations are largely missing. We are debating what is intelligence, how it works, how to even start simulating it, or construct from scratch.
  - Vecr 2y ago
    There are algorithms that should work, they're just galactic[0] or are otherwise expected to use far too much space and time to be practical. [0]: https://en.wikipedia.org/wiki/Galactic_algorithm https://en.wikipedia.org/wiki/Galactic_algorithm
    - ryan93 2y ago
      That wiki article has nothing to do with AI. The whole AI space attracts BS talk
      - Vecr 2y ago
        What do you think AI is? On that one page there's simulated annealing with a logarithmic cooling schedule, Hutter search, and Solomonoff induction, all very much applicable to AI. If you want a fully complete galactic algorithm for AI, look up AIXItl. Edit: actually I'm not sure if AIXItl is technically galactic or just terribly inefficient, but there's been trouble making it faster and more compact.

swader999 2y ago

Both need a crap tonne of electricity.

torginus 2y ago

A non-cynical take is that Ilya wanted to do research without the pressure of having to release a marketable product and figuring out how to monetize their technology, which is why he left OpenAI. A very cynical take is that this is an extreme version of 'we plan to spend all money on growth and figure out monetization later' model that many social media companies with a burn rate of billions of $$, but no business model, have used.

layer8 2y ago
That’s not a cynical take, it’s the obvious take.
signatoremo 2y ago
He was on the record that their first product will be a safe superintelligence and it won’t do anything else until then, which sounds like they won’t have paid customers until they can figure out how to build a superintelligent model. That’s certainly a lofty goal and a very long term play.
- lupire 2y ago
  OpenAI was "on the record" with a lot of obsolete claims too. Money changes people.
  - signatoremo 2y ago
    He didn’t promise world peace nor did he claim his work belongs to the humanity. The company is still a for profit corporation. He is saying he will try to build something head and shoulders above anything else, and he got a billion dollars to do it with no expectation of revenue until his product is ready. The likelihood that he fails is very high, but his backers are willing to bet on that.

jchonphoenix 2y ago

OpenAI initially raised 50m in their institutional round. 1b was a non profit donation, so there wasn't an expectation of returns on that one.

apwell23 2y ago

> superintelligence is reached i read the article but I am not sure how they know when this condition will be true. Is this obvious to ppl reading this article? is it emperor has no clothes type situation ?

Propelloni 2y ago
You are not alone. This is the litmus test many people are contemplating for a long time now, mostly philosophers, which is not surprising since it is a philosophical question. Most of the heavy stuff is hidden behind paywalls, but here's a nice summary of the state of the art by two CS guys: https://arxiv.org/pdf/2212.06721 https://arxiv.org/pdf/2212.06721
crappybird 2y ago
They can dilute the term to whatever they want. I think when the pressure to release becomes too high, they can just stick a patch of "Superintelligence™" on their latest LLM and release it.
- apwell23 2y ago
  what do you make of ppl commenting here saying 'well they won't release till superintelligence' . Are these ppl merely gullible or coconspirators in the scam ?
  - sensanaty 2y ago
    There's billions (with a B), probably closing in on Trillions, riding on the AI hypewave. This forum is full of VCs and VC-adjacent people who have vested interests in AI companies blowing up and being successful, regardless of if it's actually useful. If you check the 2024 YC batch, you'll notice pretty much every single one of them mentions AI in some form or another. I guarantee you the large majority of them are just looking to be bought out by some megacorp, because it's free money right now.

TrackerFF 2y ago

To my ears, it's more like a ambitious pharma project. There's plenty of players going for the same goal. R&D is wildly expensive. No guarantee they'll reach the goal, first or even at all.

choppaface 2y ago

Could be more comparable to Clubhouse, which VCs quickly piled $100m into[1a], and which Clubhouse notably turned into layoffs [1b]. In this case, the $1b in funding and high valuation might function predominantly as a deterrent to any flippers (in contrast, many Clubhouse investors got quick gains). Moreover, the majority of the capital likely goes into GPU hardware and/or opex, which VCs have currently arbitraged themselves [3], so to some extent this is VCs literally paying themselves to pay off their own hardware bet. While hints of the ambition of the Manhattan project might be there, the economics really are not. [1a] https://www.getpin.xyz/post/clubhouse-lessons-for-investors https://www.getpin.xyz/post/clubhouse-lessons-for-investors [1b] https://www.theverge.com/2023/4/27/23701144/clubhouse-layoffs-half-employees https://www.theverge.com/2023/4/27/23701144/clubhouse-layoff... [3] https://observer.com/2024/07/andreessen-horowitz-stocking-ai-chips-win-deals-gpu-shortage/ https://observer.com/2024/07/andreessen-horowitz-stocking-ai...

janalsncm 2y ago

> Closest thing we have to a Manhattan Project in the modern era? No. The Manhattan Project started after we understood the basic mechanism of runaway fission reactions. The funding was mostly spent purifying uranium. AGI would be similar if we understood the mechanism of creating general intelligence and just needed to scale it up. But there are still fundamental questions we still aren’t close to understanding for AGI. A more apt comparison today is probably something like fusion reactors although progress has been slow there too. We know how fusion works in theory. We have done it before (thermonuclear weapons). There are sub-problems we need to solve, but people are working on them. For AGI we don’t even know what the sub-problems are yet.

Runsthroughit 2y ago

Literally everyone from OpenAI lied 100% about everything of substance. Sutskever lied about "world model" inside of LLMs, which is such a despicable lie, because he knows that "latent space" is a TOTAL MESS. Proven everytime anyone looked at it. Shameless grifters. When end?

m3kw9 2y ago

Super Intelligence, even OpenAI when getting investment from Microsoft, OpenAI won’t have to share its “AGI” model to them and it is up to OpenAI to define what that is and who the heck knows how they will define it. The point is that that phrase is the most ambiguous word in tech right now and almost everyone thinks what’s in their head is AGI, some will think Skynet, some will think enough reason ability, some will think god like undecipherable logic and everywhere in between

porti 2y ago

[dead]

troupo 2y ago

No. It's the next Magic Leap of our era. Or the next Juicero of our era. Or the next any of the hundreds of unprofitable startups losing billions of dollars a year without any business plan beyond VC subsidies and a hope for an exit of our era.