4 ms·

> There is no technical moat, but that doesn't mean there isn't a moat. Gruber writes: " My take on OpenAI is that both of the following are true: OpenAI cur

by noch 2y ago

> There is no technical moat, but that doesn't mean there isn't a moat. Gruber writes: " My take on OpenAI is that both of the following are true: OpenAI currently offers, by far, the best product experience of any AI chatbot assistant. There is no technical moat in this field, and so OpenAI is the epicenter of an investment bubble. " It's amusing to me that he seems to think that OpenAI (or xAI or DeepSeek or DeepMind) is in the business of building "chatbots". The prize is the ability to manufacture intelligence. How much risk investors are willing to undertake for this prize is evident from their investments, after all, these investors all lived through prior business cycles and bubbles and have the institutional knowledge to know what they're getting into, financially. How much would you invest for a given probability that the company you invest in will be able to manufacture intelligence at scale in 10 years?

swiftcoder 2y ago
> How much would you invest for a given probability that the company you invest in will be able to manufacture intelligence at scale in 10 years? What's the expected return on investment for "intelligence"? This is extremely hard to quantify, and if you listen to the AI-doomer folks, potentially an extremely negative return.
- jay_kyburz 2y ago
  The race to singularity. Will it bring untold wealth to its masters, or will it slip its leash and seek its own agenda. Once you have an AI that can actually write code, what will it be able to do with its own source? How much better would open AI be with a super intelligence looking for efficiencies and improvements? What will the super intelligence (and or its masters) do to build that moat and secure its position?
- noch 2y ago
  > What's the expected return on investment for "intelligence"? This is extremely hard to quantify […] if you listen to the AI-doomer folks, potentially an extremely negative return. Indeed. And that asymmetry is what makes a market: people who can more accurately quantify the value or risk of stuff are the ones who win. If it were easy then we'd all invest in the nearest AI startup or short the entire market and 100x our net worth essentially overnight.
  - ben_w 2y ago
    That logic applies for AI-cynics rather than AI-doomers — the latter are metaphorically the equivalent of warning about CO2-induced warming causing loss of ice caps and consequent sea level rises and loss of tens of trillions of dollars of real estate as costal cities are destroyed… in 1896*, when it was already possible to predict, but we were a long way from both the danger and the zeitgeist to care. But only metaphorically the equivalent, as the maximum downside is much worse than that. https://en.m.wikipedia.org/wiki/Svante_Arrhenius https://en.m.wikipedia.org/wiki/Svante_Arrhenius
    - noch 2y ago
      > That logic applies for AI-cynics rather than AI-doomers Fwiw, I don't believe that there are any AI doomers. I've hung out in their forums for several years and watched all their lectures and debates and bookmarked all their arguments with strangers on X and read all their articles and … They talk of bombing datacentres, and how their children are in extreme danger within a decade or how in 2 decades, the entire earth and everything on it will have been consumed for material or, best case, in 2000 years, the entire observable universe will have been consumed for energy. The doomers have also been funded to the tune of half a billion dollars and counting. If these Gen-X'ers and millennials really believed their kids right now were in extreme peril due to the stupidity of everyone else, they'd be using their massive warchest full of blank cheques to stockpile weapons and hire hitmen to de-map every significant person involved in building AI. After all, what would you do if you knew precisely who are in the groups of people coming to murder your child in a few years? But the true doomer would have to be the ultimate nihilist, and he would simply take himself off the map because there's no point in living.
      - ben_w 2y ago
        > or, best case, in 2000 years, the entire observable universe will have been consumed for energy You're definitely mixing things up, and the set of things may include fiction. 2000 years doesn't get you out of the thick disk region of our own galaxy at the speed of light. > The doomers have also been funded to the tune of half a billion dollars and counting. I've never heard such a claim. LessWrong.com has funding more like a few million: https://www.lesswrong.com/posts/5n2ZQcbc7r4R8mvqc https://www.lesswrong.com/posts/5n2ZQcbc7r4R8mvqc > If these Gen-X'ers and millennials really believed their kids right now were in extreme peril due to the stupidity of everyone else, they'd be using their massive warchest full of blank cheques to stockpile weapons and hire hitmen to de-map every significant person involved in building AI. After all, what would you do if you knew precisely who are in the groups of people coming to murder your child in a few years? The political capital to ban it worldwide and enforce the ban globally with airstrikes — what Yudkowsky talked about was "bombing" in the sense of a B2, not Ted Kaczynski — is incompatible with direct action of that kind. And that's even if such direct action worked. They're familiar with the luddites breaking looms, and look how well that worked at stopping the industrialisation of that field. Or the communist revolutions, promising a great future, actually taking over a few governments, but it didn't actually deliver the promised utopia. Even more recently, I've not heard even one person suggest that the American healthcare system might actually change as a result of that CEO getting shot recently. But also, you have a bad sense of scale to think that "half a billion dollars" would be enough for direct attacks. Police forces get to arrest people for relatively little because "you and whose army" has an obvious answer. The 9/11 attacks may have killed a lot of people on the cheap, but most were physically in the same location, not distributed between several in different countries: USA (obviously), Switzerland (including OpenAI, Google), UK (Google, Apple, I think Stability AI), Canada (Stability AI, from their jobs page), China (including Alibaba and at least 43 others), and who knows where all the remote workers are. Doing what you hypothesise about would require a huge, global, conspiracy — not only exceeding what Al Qaida was capable of, but significantly in excess of what's available to either the Russian or Ukrainian governments in their current war. Also: > After all, what would you do if you knew precisely who are in the groups of people coming to murder your child in a few years? You presume they know. They don't, and they can't, because some of the people who will soon begin working on AI have not yet even finished their degrees. If you take Altman's timeline of "thousands of days", plural, then some will not yet have even gotten as far as deciding which degree to study.
lodovic 2y ago
Part of the risk is the possibility that a few key employees find a much more profitable business model and leave OpenAI, while the early investors are left holding the bag. This seems to be a recurring theme in the tech world.
- noch 2y ago
  > Part of the risk is the possibility that a few key employees find a much more profitable business model and leave OpenAI, The fact that you can state this risk means that market participants already know this risk and account for it in their investment model. Usually employees are given stock options (or something similar to a vesting instrument) to align them with the company, that is, they lose[^1] significant wealth if they leave the company. In the case of OpenAI: "PPUs vest evenly over 4 years (25% per year). Unlike stock options, employees do not need to purchase PPUs […] PPUs also are restricted by a 2-year lock, meaning that if there’s a liquidation event, a new hire can’t sell their units within their first 2 years."[^0] -- [0]: https://www.levels.fyi/blog/openai-compensation.html https://www.levels.fyi/blog/openai-compensation.html [1]: Ex-OpenAI employee reported losing 85% of his family's net worth. https://news.ycombinator.com/item?id=40406307 https://news.ycombinator.com/item?id=40406307
n_ary 2y ago
> How much risk investors are willing to undertake for this prize is evident from their investments, after all, these investors all lived through prior business cycles and bubbles and have the institutional knowledge to know what they're getting into, financially. There are not that many unicorns these days, so anyone missing out on last unicorn decades are now in immense FOMO and is willing to bet big. Besides, AGI is considered(own opinion) personal skynet(wet dream of all nations’ military) that will do your bidding. Hence everyone wants a piece of that Pie. Also when the bigCo(M$/Google/Meta) are willing to bet on it, makes the topic much more interesting and puts invisible seal of approval from technically savvy corps, as the previous scammy cryptocurrency gold rush was not participated by any bigCo(to best of my knowledge) but GenAI is full game with all.

by noch 2y ago

swiftcoder 2y ago

> How much would you invest for a given probability that the company you invest in will be able to manufacture intelligence at scale in 10 years? What's the expected return on investment for "intelligence"? This is extremely hard to quantify, and if you listen to the AI-doomer folks, potentially an extremely negative return.

jay_kyburz 2y ago
The race to singularity. Will it bring untold wealth to its masters, or will it slip its leash and seek its own agenda. Once you have an AI that can actually write code, what will it be able to do with its own source? How much better would open AI be with a super intelligence looking for efficiencies and improvements? What will the super intelligence (and or its masters) do to build that moat and secure its position?
noch 2y ago
> What's the expected return on investment for "intelligence"? This is extremely hard to quantify […] if you listen to the AI-doomer folks, potentially an extremely negative return. Indeed. And that asymmetry is what makes a market: people who can more accurately quantify the value or risk of stuff are the ones who win. If it were easy then we'd all invest in the nearest AI startup or short the entire market and 100x our net worth essentially overnight.
- ben_w 2y ago
  That logic applies for AI-cynics rather than AI-doomers — the latter are metaphorically the equivalent of warning about CO2-induced warming causing loss of ice caps and consequent sea level rises and loss of tens of trillions of dollars of real estate as costal cities are destroyed… in 1896*, when it was already possible to predict, but we were a long way from both the danger and the zeitgeist to care. But only metaphorically the equivalent, as the maximum downside is much worse than that. https://en.m.wikipedia.org/wiki/Svante_Arrhenius https://en.m.wikipedia.org/wiki/Svante_Arrhenius
  - noch 2y ago
    > That logic applies for AI-cynics rather than AI-doomers Fwiw, I don't believe that there are any AI doomers. I've hung out in their forums for several years and watched all their lectures and debates and bookmarked all their arguments with strangers on X and read all their articles and … They talk of bombing datacentres, and how their children are in extreme danger within a decade or how in 2 decades, the entire earth and everything on it will have been consumed for material or, best case, in 2000 years, the entire observable universe will have been consumed for energy. The doomers have also been funded to the tune of half a billion dollars and counting. If these Gen-X'ers and millennials really believed their kids right now were in extreme peril due to the stupidity of everyone else, they'd be using their massive warchest full of blank cheques to stockpile weapons and hire hitmen to de-map every significant person involved in building AI. After all, what would you do if you knew precisely who are in the groups of people coming to murder your child in a few years? But the true doomer would have to be the ultimate nihilist, and he would simply take himself off the map because there's no point in living.
    - ben_w 2y ago
      > or, best case, in 2000 years, the entire observable universe will have been consumed for energy You're definitely mixing things up, and the set of things may include fiction. 2000 years doesn't get you out of the thick disk region of our own galaxy at the speed of light. > The doomers have also been funded to the tune of half a billion dollars and counting. I've never heard such a claim. LessWrong.com has funding more like a few million: https://www.lesswrong.com/posts/5n2ZQcbc7r4R8mvqc https://www.lesswrong.com/posts/5n2ZQcbc7r4R8mvqc > If these Gen-X'ers and millennials really believed their kids right now were in extreme peril due to the stupidity of everyone else, they'd be using their massive warchest full of blank cheques to stockpile weapons and hire hitmen to de-map every significant person involved in building AI. After all, what would you do if you knew precisely who are in the groups of people coming to murder your child in a few years? The political capital to ban it worldwide and enforce the ban globally with airstrikes — what Yudkowsky talked about was "bombing" in the sense of a B2, not Ted Kaczynski — is incompatible with direct action of that kind. And that's even if such direct action worked. They're familiar with the luddites breaking looms, and look how well that worked at stopping the industrialisation of that field. Or the communist revolutions, promising a great future, actually taking over a few governments, but it didn't actually deliver the promised utopia. Even more recently, I've not heard even one person suggest that the American healthcare system might actually change as a result of that CEO getting shot recently. But also, you have a bad sense of scale to think that "half a billion dollars" would be enough for direct attacks. Police forces get to arrest people for relatively little because "you and whose army" has an obvious answer. The 9/11 attacks may have killed a lot of people on the cheap, but most were physically in the same location, not distributed between several in different countries: USA (obviously), Switzerland (including OpenAI, Google), UK (Google, Apple, I think Stability AI), Canada (Stability AI, from their jobs page), China (including Alibaba and at least 43 others), and who knows where all the remote workers are. Doing what you hypothesise about would require a huge, global, conspiracy — not only exceeding what Al Qaida was capable of, but significantly in excess of what's available to either the Russian or Ukrainian governments in their current war. Also: > After all, what would you do if you knew precisely who are in the groups of people coming to murder your child in a few years? You presume they know. They don't, and they can't, because some of the people who will soon begin working on AI have not yet even finished their degrees. If you take Altman's timeline of "thousands of days", plural, then some will not yet have even gotten as far as deciding which degree to study.

lodovic 2y ago

Part of the risk is the possibility that a few key employees find a much more profitable business model and leave OpenAI, while the early investors are left holding the bag. This seems to be a recurring theme in the tech world.

noch 2y ago
> Part of the risk is the possibility that a few key employees find a much more profitable business model and leave OpenAI, The fact that you can state this risk means that market participants already know this risk and account for it in their investment model. Usually employees are given stock options (or something similar to a vesting instrument) to align them with the company, that is, they lose[^1] significant wealth if they leave the company. In the case of OpenAI: "PPUs vest evenly over 4 years (25% per year). Unlike stock options, employees do not need to purchase PPUs […] PPUs also are restricted by a 2-year lock, meaning that if there’s a liquidation event, a new hire can’t sell their units within their first 2 years."[^0] -- [0]: https://www.levels.fyi/blog/openai-compensation.html https://www.levels.fyi/blog/openai-compensation.html [1]: Ex-OpenAI employee reported losing 85% of his family's net worth. https://news.ycombinator.com/item?id=40406307 https://news.ycombinator.com/item?id=40406307

n_ary 2y ago

> How much risk investors are willing to undertake for this prize is evident from their investments, after all, these investors all lived through prior business cycles and bubbles and have the institutional knowledge to know what they're getting into, financially. There are not that many unicorns these days, so anyone missing out on last unicorn decades are now in immense FOMO and is willing to bet big. Besides, AGI is considered(own opinion) personal skynet(wet dream of all nations’ military) that will do your bidding. Hence everyone wants a piece of that Pie. Also when the bigCo(M$/Google/Meta) are willing to bet on it, makes the topic much more interesting and puts invisible seal of approval from technically savvy corps, as the previous scammy cryptocurrency gold rush was not participated by any bigCo(to best of my knowledge) but GenAI is full game with all.