6 ms·

Who would have thought that vectorized linear algebra will be at the center of so much financial speculation? There is a silver lining though. Even if it all g

Who would have thought that vectorized linear algebra will be at the center of so much financial speculation? There is a silver lining though. Even if it all goes to near-zero (most likely outcome for all VC investments anyway) the digital world will be one where fast matrix multiply is thoroughly commoditized. This is not a trivial feat. In a sense this will be the true end of the Wintel era. The old world of isolated, CISC, deterministic desktops giving way not to "AGI", but widely available, networked, vector "supercomputers" that can digest and transform practically everything that has ever been digitized. Who knows what the actual (financial) winners of this brave new era will be. In an ideal world there should be no winner-takes-all entity but a broad-based leveling up, i.e., spreading these new means of production as widely as possible. Heck, maybe we will even eventually see the famously absent productivity gains from digital tech?

gizajob 2y ago
“Who knows what the actual (financial) winners of this brave new era will be” Nvidia
- openrisk 2y ago
  They are now, but will they be winning in 10 (or even 3-5) years? Their shtick is a GPU on steroids. In the bigger picture its a well positioned hack that has ridden two successive speculative bubbles (crypto mining and AI) but its unclear how far this can go. Currently this approach is wildly successful cause nobody else bothered to toil a serious vision about the post-Moore's law era. But make no mistake, people's minds will get focused.
  - JonChesterfield 2y ago
    It's not a graphics card hacked to do math any more. It's a general purpose computer with some legacy cruft added to handle graphics work if necessary. Lots of people are working very hard to find something better and have been for at least a decade, probably several.
- __loam 2y ago
  Maybe I'm naive but there seems to be way too much financial incentive in this space for CUDA to continue to be the top dog. Just like microprocessors, these devices are going to get commodified, standardized, open sourced, etc. Nvidia making massive profits is a sign of huge market inefficiency and potential opportunities for competition.
  - robertlagrant 2y ago
    > Nvidia making massive profits is a sign of huge market inefficiency and potential opportunities for competition. Yep, but just as the first reading glasses were only available to the wealthy, and now anyone can have them, the inefficiency takes time to work out. It'll take a long time, especially given how vertically integrated Nvidia are.
    - Obscurity4340 2y ago
      How could people who actually like NEEDED glasses be productive or get around without them prior to glasses being developed and commodified?
      - robertlagrant 2y ago
        > who actually like NEEDED glasses I don't understand this phrasing - are you implying I'm not aware of these people? People...had bad eyesight before. Now they have bad eyesight with corrective lenses.
      - satvikpendem 2y ago
        Bad eyesight, especially myopia, was much rarer than today, with the leading hypothesis being that not enough sunlight exposure over one's growing years leads to it.
    - __loam 2y ago
      Weird analogy considering alternatives already exist in this case, it's just a matter of making projects like opencl or metal better.
- trashtester 2y ago
  My guess is that whoever develops superintelligence first will not release it to the public, but rather use it for their own purposes to gain an edge. They may still release AI products to the public that are good enough and cheap enough to prevent competitors from being profitable or receive funding (to prevent them from catching up), but that's not where the value would come from. Just as an example, let's say xAI is first. Instead of releasing the full capability as GROK 7, they would use the ASI to create a perfected version of their self driving software, to power their Optimus robots. And to speed up the development of future manufacturing products (including, but not limited to cars and humanoid robots) And as such winners may be challenged by anti-trust regulations, the ASI may also be utilized to gain leverage over the political system. Twitter/X could be one arena that would allow this. Eventually, Tesla robots might even be used to replace police officers and military personnel. If so, the company might be a single software update away from total control.
  - robertlagrant 2y ago
    > whoever develops superintelligence first will not release it to the public, but rather use it for their own purposes to gain an edge We have no evidence that superintelligence will be developed. There's no "first". There's only "remote possibility".
    - trashtester 2y ago
      > We have no evidence that superintelligence will be developed. Fundamentally, we have no evidence of anything that will happen in the future. All we do is to extrapolate from the past through the present, typically using some kind of theory of how the world operates. The belief that we will eventually (whether it's this year or in 1000+ years), really only hinges on the following 3 assumptions: 1) The human brain is fully material (no divine souls is necessary for intelligence) 2) The human brain does not represent a global optimum for how intelligent a material intelligence-having-object can be. 3) We will eventually have the ability to build intelligence-having-objects (similar or different from the brain) that not only can do the same as a brain (that would be mere AGI), but also surpass it in many ways. Assumptions 1 and 2 have a lot of support in the current scientific consensus. Those who reject them either do not know the science or they have a belief system that would be invalidated if one of those assumptions were true. (That could be anything from a Christian belief in the soul to an ideological reliance of a "Tabula Rasa" brain). Assumption 3 is mostly techno-optimism, or an extrapolation of the trend that we are able to build ever more advanced devices. As for WHEN we get there, there is a fourth assumption required for it to happens soon: 4. For intelligence-having-objects to do their thing, they don't need some exotic mechanisms we don't yet need to build. For instance, there is no need to build large quantum computers for this. This assumption is mostly about belief, and we really don't know. Yet, given the current rate of progress, and if we accept assumptions 1-3, I don't think assumption 4 is unreasonable. If so, it's not unreasonable to assume that our synthetic brains reach roughly human level intelligence when their size/complexity becomes similar to that of human brains. Human brains have ~200 trillion synapses. That's about 100x-1000x more than the latest neural nets that we're building. Based only on scale, current nets (GPT-4 generation) should have total capabilities similar or slightly better than a rat. I think that's not very far off from what we're seeing, even if the nets tend to have those capabilities linked to text/images rather than the physical world that a rat navigates. In other words, I think we DO have SOME evidence (not conclusive) that the capabilities of a neural net can reach similar "intelligence" to animals with a similar number of synapses. So IF that hypothesis holds true, and given assumptions 1-3 above, there is a fair possibility that human level intelligence will be reached when we scale up to about 200 trillion weights (and have the ability to train such nets). And currently, several of the largest and most valuable companies in the world are making a huge gamble on this being the case, with plans to scale up nets by 100x over the next few years, which will be enough to get very close to human brain sized nets.
- js8 2y ago
  Computronium is the new gold.
lobochrome 2y ago
This is a fantastic comment. Very insightful.
- hcta 2y ago
  [flagged]
pennaMan 2y ago
>Who would have thought that vectorized linear algebra will be at the center of so much financial speculation? Wait till you hear that a bunch of meat[1] is behind all said speculation. [1]https://stuff.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html https://stuff.mit.edu/people/dpolicar/writing/prose/text/thi...
trashtester 2y ago
> Who would have thought that vectorized linear algebra will be at the center of so much financial speculation? "vectorized linear algebra" is at the root of most of modern Physics. Specifically, the laws of Physics are represented by the Lie groups U(1), SU(2), SU(3) and SO(3,1). While the manifolds that Physics act on are curved, they're "locally flat". That is why local operations are tensor operations. Or linear algebra, if you prefer. It's not all that surprising to me that "intelligence" is represented by similar math. In fact, there is active work being done on making sense of deep learning using Lie algebra [1] (and Algebraic Topology, which generalizes the Lie algebra). This math can be a bit hard, though, so the learning curve can be steep. However, when we're creating AI models to be ML scientists, I suspect that this kind of math may be a source of "unhobbling", as meant in Situational Awareness [2]. Because if we can understand the symmetries at play in a problem domain, it's generally a lot easier to find a mathematical architecture (like in the algebras above) that effectively describe the domain, which allows us to potentially reduce the degrees of freedom by many OOM. > Heck, maybe we will even eventually see the famously absent productivity gains from digital tech? I think it's a mistake to think of AI as "digital tech", especially so to assume that the development of the Internet, Social Media or crypto that we've seen over the last generation. AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero). [1] https://arxiv.org/pdf/2402.08871 https://arxiv.org/pdf/2402.08871 [2] https://situational-awareness.ai https://situational-awareness.ai
- baq 2y ago
  > AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero). time for a Culture re-read I guess.
- openrisk 2y ago
  > "vectorized linear algebra" is at the root of most of modern Physics. the "vectorized" adjective was meant to imply implementing linear algebra in digital computers that can operate concurrently on large-dimensional vectors/tensors. In this sense (and despite Wolfram's diligence and dearest wishes) modern physics theories have exactly 0% digital underpinning :-) > It's not all that surprising to me that "intelligence" is represented by similar math. yes, the state of the art of our modeling ability in pretty much any domain is to conceive of a non-linear system description and "solve it" by linearization. Me thinks this is primary reason we haven't really cracked "complexity": We can only solve the problems we have the proverbial hammer to apply to. > AI fundamentally comes with the potential to do anything a human can do That goes into wild speculation territory. In any case the economy is always about organizing human relationships. Technology artifacts only change the decor, not the substance of our social relations. Unless we completely cease to have dependencies on each other (what a dystopic world!) there will always be the question of an individual's ability to provide others with something of value.

gizajob 2y ago

“Who knows what the actual (financial) winners of this brave new era will be” Nvidia

openrisk 2y ago
They are now, but will they be winning in 10 (or even 3-5) years? Their shtick is a GPU on steroids. In the bigger picture its a well positioned hack that has ridden two successive speculative bubbles (crypto mining and AI) but its unclear how far this can go. Currently this approach is wildly successful cause nobody else bothered to toil a serious vision about the post-Moore's law era. But make no mistake, people's minds will get focused.
- JonChesterfield 2y ago
  It's not a graphics card hacked to do math any more. It's a general purpose computer with some legacy cruft added to handle graphics work if necessary. Lots of people are working very hard to find something better and have been for at least a decade, probably several.
__loam 2y ago
Maybe I'm naive but there seems to be way too much financial incentive in this space for CUDA to continue to be the top dog. Just like microprocessors, these devices are going to get commodified, standardized, open sourced, etc. Nvidia making massive profits is a sign of huge market inefficiency and potential opportunities for competition.
- robertlagrant 2y ago
  > Nvidia making massive profits is a sign of huge market inefficiency and potential opportunities for competition. Yep, but just as the first reading glasses were only available to the wealthy, and now anyone can have them, the inefficiency takes time to work out. It'll take a long time, especially given how vertically integrated Nvidia are.
  - Obscurity4340 2y ago
    How could people who actually like NEEDED glasses be productive or get around without them prior to glasses being developed and commodified?
    - robertlagrant 2y ago
      > who actually like NEEDED glasses I don't understand this phrasing - are you implying I'm not aware of these people? People...had bad eyesight before. Now they have bad eyesight with corrective lenses.
    - satvikpendem 2y ago
      Bad eyesight, especially myopia, was much rarer than today, with the leading hypothesis being that not enough sunlight exposure over one's growing years leads to it.
  - __loam 2y ago
    Weird analogy considering alternatives already exist in this case, it's just a matter of making projects like opencl or metal better.
trashtester 2y ago
My guess is that whoever develops superintelligence first will not release it to the public, but rather use it for their own purposes to gain an edge. They may still release AI products to the public that are good enough and cheap enough to prevent competitors from being profitable or receive funding (to prevent them from catching up), but that's not where the value would come from. Just as an example, let's say xAI is first. Instead of releasing the full capability as GROK 7, they would use the ASI to create a perfected version of their self driving software, to power their Optimus robots. And to speed up the development of future manufacturing products (including, but not limited to cars and humanoid robots) And as such winners may be challenged by anti-trust regulations, the ASI may also be utilized to gain leverage over the political system. Twitter/X could be one arena that would allow this. Eventually, Tesla robots might even be used to replace police officers and military personnel. If so, the company might be a single software update away from total control.
- robertlagrant 2y ago
  > whoever develops superintelligence first will not release it to the public, but rather use it for their own purposes to gain an edge We have no evidence that superintelligence will be developed. There's no "first". There's only "remote possibility".
  - trashtester 2y ago
    > We have no evidence that superintelligence will be developed. Fundamentally, we have no evidence of anything that will happen in the future. All we do is to extrapolate from the past through the present, typically using some kind of theory of how the world operates. The belief that we will eventually (whether it's this year or in 1000+ years), really only hinges on the following 3 assumptions: 1) The human brain is fully material (no divine souls is necessary for intelligence) 2) The human brain does not represent a global optimum for how intelligent a material intelligence-having-object can be. 3) We will eventually have the ability to build intelligence-having-objects (similar or different from the brain) that not only can do the same as a brain (that would be mere AGI), but also surpass it in many ways. Assumptions 1 and 2 have a lot of support in the current scientific consensus. Those who reject them either do not know the science or they have a belief system that would be invalidated if one of those assumptions were true. (That could be anything from a Christian belief in the soul to an ideological reliance of a "Tabula Rasa" brain). Assumption 3 is mostly techno-optimism, or an extrapolation of the trend that we are able to build ever more advanced devices. As for WHEN we get there, there is a fourth assumption required for it to happens soon: 4. For intelligence-having-objects to do their thing, they don't need some exotic mechanisms we don't yet need to build. For instance, there is no need to build large quantum computers for this. This assumption is mostly about belief, and we really don't know. Yet, given the current rate of progress, and if we accept assumptions 1-3, I don't think assumption 4 is unreasonable. If so, it's not unreasonable to assume that our synthetic brains reach roughly human level intelligence when their size/complexity becomes similar to that of human brains. Human brains have ~200 trillion synapses. That's about 100x-1000x more than the latest neural nets that we're building. Based only on scale, current nets (GPT-4 generation) should have total capabilities similar or slightly better than a rat. I think that's not very far off from what we're seeing, even if the nets tend to have those capabilities linked to text/images rather than the physical world that a rat navigates. In other words, I think we DO have SOME evidence (not conclusive) that the capabilities of a neural net can reach similar "intelligence" to animals with a similar number of synapses. So IF that hypothesis holds true, and given assumptions 1-3 above, there is a fair possibility that human level intelligence will be reached when we scale up to about 200 trillion weights (and have the ability to train such nets). And currently, several of the largest and most valuable companies in the world are making a huge gamble on this being the case, with plans to scale up nets by 100x over the next few years, which will be enough to get very close to human brain sized nets.
js8 2y ago
Computronium is the new gold.

lobochrome 2y ago

This is a fantastic comment. Very insightful.

hcta 2y ago
[flagged]

pennaMan 2y ago

>Who would have thought that vectorized linear algebra will be at the center of so much financial speculation? Wait till you hear that a bunch of meat[1] is behind all said speculation. [1]https://stuff.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html https://stuff.mit.edu/people/dpolicar/writing/prose/text/thi...

trashtester 2y ago

> Who would have thought that vectorized linear algebra will be at the center of so much financial speculation? "vectorized linear algebra" is at the root of most of modern Physics. Specifically, the laws of Physics are represented by the Lie groups U(1), SU(2), SU(3) and SO(3,1). While the manifolds that Physics act on are curved, they're "locally flat". That is why local operations are tensor operations. Or linear algebra, if you prefer. It's not all that surprising to me that "intelligence" is represented by similar math. In fact, there is active work being done on making sense of deep learning using Lie algebra [1] (and Algebraic Topology, which generalizes the Lie algebra). This math can be a bit hard, though, so the learning curve can be steep. However, when we're creating AI models to be ML scientists, I suspect that this kind of math may be a source of "unhobbling", as meant in Situational Awareness [2]. Because if we can understand the symmetries at play in a problem domain, it's generally a lot easier to find a mathematical architecture (like in the algebras above) that effectively describe the domain, which allows us to potentially reduce the degrees of freedom by many OOM. > Heck, maybe we will even eventually see the famously absent productivity gains from digital tech? I think it's a mistake to think of AI as "digital tech", especially so to assume that the development of the Internet, Social Media or crypto that we've seen over the last generation. AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero). [1] https://arxiv.org/pdf/2402.08871 https://arxiv.org/pdf/2402.08871 [2] https://situational-awareness.ai https://situational-awareness.ai

baq 2y ago
> AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero). time for a Culture re-read I guess.
openrisk 2y ago
> "vectorized linear algebra" is at the root of most of modern Physics. the "vectorized" adjective was meant to imply implementing linear algebra in digital computers that can operate concurrently on large-dimensional vectors/tensors. In this sense (and despite Wolfram's diligence and dearest wishes) modern physics theories have exactly 0% digital underpinning :-) > It's not all that surprising to me that "intelligence" is represented by similar math. yes, the state of the art of our modeling ability in pretty much any domain is to conceive of a non-linear system description and "solve it" by linearization. Me thinks this is primary reason we haven't really cracked "complexity": We can only solve the problems we have the proverbial hammer to apply to. > AI fundamentally comes with the potential to do anything a human can do That goes into wild speculation territory. In any case the economy is always about organizing human relationships. Technology artifacts only change the decor, not the substance of our social relations. Unless we completely cease to have dependencies on each other (what a dystopic world!) there will always be the question of an individual's ability to provide others with something of value.