7 ms·

I've been following the whole thing low key since the 2nd wave of neural networks in the mid 90s - and made a very very minor contribution to the field which ha

I've been following the whole thing low key since the 2nd wave of neural networks in the mid 90s - and made a very very minor contribution to the field which has applications these days back then too. My observation is that every wave of neural networks has resulted in a dead end. In my view, this is in large part caused by the (inevitable) brute force mathematical approach used and the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition. Or as put in the article "Current AI systems have no internal structure that relates meaningfully to their functionality". This is the most important thing. Maybe layers of indirection can fix that, but I kind of doubt it. I am however quite excited about what LLMs can do to make semantic search much easier, and impressed at how much better they've made the tooling around natural language processing. Nonetheless, I feel I can already see the dead end pretty close ahead.

ninetyninenine 2y ago
If we ever hit agi this overall point: “ the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition.” Will remain true imho. We will never fully intuit AI or understand it outside of some brute force abstraction like a token predictor or best fit curve.
steve_adams_86 2y ago
I didn’t see this at first, and I was fairly shaken by the potential impact on the world if their progress didn’t stop. A couple generations showed meaningful improvements, but now it seems like you’re probably correct. I’ve used these for years quite intensively to aid my work and while it’s a useful rubber duck, it doesn’t seem to yield much more beyond that. I worry a lot less about my career now. It really is a tool that creates more work for me rather than less.
- iman453 2y ago
  Would this still hold true in your opinion if models like O3 become super cheap and bit better over time? I don't know much about the AI space, but as a vanilla backend dev also worry about the future :)
  - root_axis 2y ago
    Let's see how O3 pans out in practice before we start setting it as the standard for the future.
    - varelse 2y ago
      Mamba-ish models are the breakthrough to cheap inference if they pan out. Calling a dead-end already is just silly.
  - sydd 2y ago
    We know that OpenAI is verz good at least in one thing: generating hype. When Sora was announced everyone thought that this will be revolutionary. Look at how it looks like in production. Same when they started floating rumours that they have some AGI prototype in their labs. They are the Tesla of the IT world, overpromise and under deliver.
    - WhyOhWhyQ 2y ago
      It's a brilliant marketing model. Humans are inherently highly interested in anything which could be a threat to their well-being. Everything they put out is a tacit promise that the viewer will soon be economically valueless.
      - bilbyx 2y ago
        I hope people will come to the realisation that we have created a good plagiarizer at best. The "intelligence" originates from the human beings who created the training data for these LLMs. The hype will die when reality hits.
    - gom_jabbar 2y ago
      Hype is very interesting. The concept of Hyperstition describes fictions that make themselves real. In this sense, hype is an essential part of capitalism: "Capitalization is [...] indistinguishable from a commercialization of potentials, through which modern history is slanted (teleoplexically) in the direction of ever greater virtualization, operationalizing science fiction scenarios as integral components of production systems." [0] "Within capitalist futures markets, the non-actual has effective currency. It is not an "imaginary" but an integral part of the virtual body of capital, an operationalized realization of the future." [1] This corresponds to the idea that virtual is opposed to actual, not real. [0] https://retrochronic.com/#teleoplexy-12 https://retrochronic.com/#teleoplexy-12 [1] https://retrochronic.com/#on-accelerate-2b https://retrochronic.com/#on-accelerate-2b
- trhway 2y ago
  >I worry a lot less about my career now. It really is a tool that creates more work for me rather than less. when i was a team/project leader the largest part of my work was talking to the reports on what needs to be implemented and how they are going to implement it and the current progress of the implementation, how to interface the stuff, what are the issues and how to approach the troubleshooting, what are the next steps, etc. with occasional looking into/reviewing the code - it looks to me what working with coding LLM would soon be quite similar to that.
- trod1234 2y ago
  Many of the major harms of these things were neglected, and downplayed even to this day people don't recognize just how changed the world has become. The mere delusion that AI will replace work has been used to justify mass layoffs. The persistence of indistinct ghost jobs that are generated by computer for pennies to flood and bind with prospective job seekers (similar to RNA interference), has resulted in severe brain drain in many fields. Worse, the fact these people have often been forced into poverty as a result will have a lasting impact. You might have planned for up to a year out of work pre-AI and had the financial resources, but now how long does it take? Conversion ratios for the first step have changed by two magnitudes or order (from x100 to x10,000). What are the odds of these people finding a job given their finite time, and requirements that are un-automatable for submission (nil). The media keeps claiming that everything is getting better, the stats say so (while neglecting the fact that the stats are being manipulated to the point of uselessness/fabricated), but you have 1/3 of welfare payouts now going to these people in California (in the US), just for basic food. When you can't find work, you go where the work is abandoning the bad economic investment and choice you made regardless of how competent you were. It is a psychologically sticky decision. When there is no chance at finding work, you get desperate, and many desperate people turn to crime and unrest. This was foreseen by a number of very intelligent people many decades ago, and ignored following business as usual. The mere demonstration that we are unable to react in time is what gave engineers such great pause to write about these things, as far back as in the 70s. Hysteresis is a lagging time problem where you can't react fast enough to avert catastrophic failure given chaotic conditions, leaving survival up to chance. Its the worst type of engineering problem with real consequences. Given how western society is structured dependently on labor exchange, its a perfect weapon of chaos and debasement in the value of labor, that effectively destroys half of its underlying economic structure (factor markets). This forces sieving conditions of wealth that become spinodal, and eventually falter under their constraints and spiral into deflationary trends over time. Business wins so much that they lose everything. Its quite a disadvantaged environment and the general trend is that everyone is ignoring the pink elephant. Actions (and inaction) have consequences. When people don't listen and take appropriate action, consequences get dire, it hits the fan.
  - cgio 2y ago
    I agree that we are often missing in our analyses the true materialised impact of expectations by focusing on the validity of said expectations instead. Organisations, even if not laying off, are pausing hiring plans with a conviction that AI will replace some of the workers. It then becomes a self fulfilling prophecy to some extent. It doesn’t matter if it can, what matters is if it will. And to assume that people won’t place a bet is futile, as everyone does, and even if it’s wrong the market will allocate the losses to the baseline.
hammock 2y ago
What are your thoughts on neuro-symbolic integration (combining the pattern-recognition capabilities of neural networks with the reasoning and knowledge representation of symbolic AI) ?
- bionhoward 2y ago
  Seems like the symbolic aspect is poorly defined and it’s too unclear to be useful. Always sounds cool, but what exactly are we talking about?
  - hammock 2y ago
    I’m not an AI expert, but from my armchair I might draw a comparison between functional (symbolic rule- and logic-based AI) and declarative (LLM) programming languages
gizmo 2y ago
Previous generations of neural nets were kind of useless. Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered. Machine learning had a couple of niche applications but for most things it didn't work. This time it's different. The naysayers are wrong. LLMs today can already automate many desk jobs. They already massively boost productivity for people like us on HN. LLMs will certainly get better, faster and cheaper in the coming years. It will take time for society to adapt and for people to realize how to take advantage of AI, but this will happen. It doesn't matter whether you can "test AI in part" or whether you can do "exhaustive whole system testing". It doesn't matter whether AIs are capable of real reasoning or are just good enough at faking it. AI is already incredibly powerful and with improved tooling the limitations will matter much less.
- jfengel 2y ago
  From what I have seen, most of the jobs that LLMs can do are jobs that didn't need to be done at all. We should turn them over to computers, and then turn the computers off.
  - kube-system 2y ago
    They're good at processing text. Processing text is a valuable thing that sometimes needs to be done. We still use calculators even though the profession we used to call "computer" was replaced by them.
    - jonasced 2y ago
      But here reliability comes in again. Calculators are different since the output is correct as long as the input is correct. LLMs do not guarantee any quality in the output even when processing text, and should in my opinion be verified before used in any serious applications.
      - kube-system 2y ago
        > Calculators are different since the output is correct as long as the input is correct. That isn't really true.[0] The application of calculators to a subject matter is something that does need to be considered in some use cases. LLMs also have accuracy considerations, and although it may be to a different degree, the subject matter to which they're applicable has a broad range of acceptable accuracies. While some textual subject matter demands a very specific answer, some doesn't: For example, there may be hundreds or thousands of various ways to summarize a text that could be accurate for a particular application. 0: example: https://www.reddit.com/r/calculus/comments/upjdn4/why_do_all_calculators_not_get_the_exact_same/ https://www.reddit.com/r/calculus/comments/upjdn4/why_do_all...
dlkf 2y ago
> Current AI systems have no internal structure that relates meaningfully to their functionality In what sense is the relationship between neurons and human function more “meaningful” than the relationship between matrices and LLM function? You’re correct that LLMs are probably a dead end with respect to AGI, but this is completely the wrong reason.
- mmcnl 2y ago
  Human intelligence has a track record of being useful for thousands of years.
- imtringued 2y ago
  The neurons are always learning whereas the matrices don't change.
  - dlkf 2y ago
    I mean, the matrices obviously change during training. I take it your point is that LLMs are trained once and then frozen, whereas humans continuously learn and adapt to their environment. I agree that this is a critical distinction. But it has nothing to do with “meaningful internal structure.”
- singingfish 2y ago
  Yeah the internal representation of organic neural networks are also weird - check out the signal processing that occurs between the retina and the various parts of the visual cortex before any decent information can emerge from the signal - David Marr's 1980s book Vision is a mathematically chewy treatise on this. This leads me to start thinking that human intuition may well caused by different neural network subsystems feeding processed data into other subsystems where consciousness and thus intuition and explanation emerges. Organic neural networks are pretty energy efficient in comparison- although still decently inefficient compared to other body systems - so there is the capacity to build things out to the scale required, assuming my read on what's going on there is correct, that is. So it's not clear to me that the energy inefficiency of ANNs can be sufficiently resolved to enable these multiple quasi-independent subsystems to be built at the scale required. Not even if these interesting looking trinomial neural nets which are matrix addition based rather than multiplication come to dominate the ANN scene. While I was thinking this comment through I realised there's a possible interpretation wherin human activity induced climate change is an emergent property of the relative energy inefficiency of neural architecture.
ftlisnotftl 2y ago
[dead]
busyant 2y ago
> "Current AI systems have no internal structure that relates meaningfully to their functionality". I'm curious as to why you feel this needs to be true? Or to put it another way, what would an AI structure look like to be more meaningfully connected to its function? Not trying to flame. I always feel that I can't think quite deeply enough about these issues, so I'm worried that I'm missing something 'obvious'.
- fragmede 2y ago
  AlphaGo had an artificial neural network that was specifically trained in best moves and winning percentages. An LLM trained on text has some data on what constitutes winning at go, but internally doesn't have a ANN specifically for the game of go.
  - deleted 2y ago
    [deleted]
  - busyant 2y ago
    > AlphaGo had an artificial neural network that was specifically trained in best moves and winning percentages. An LLM trained on text has some data on what constitutes winning at go, but internally doesn't have a ANN specifically for the game of go. This isn't addressing what the original commenter was referring to.
- singingfish 2y ago
  The reasoning is quite subtle, and because I'm not a very coherent guy I have problems expressing it. In the LLM space there are a whole bunch of pitfalls around overfit (largely solvable with pretty standard statistical methods) and inherent bias in training material which is a much harder to problem to solve. The fact that the internal representation gives you zero information on how to handle this bias means the tool can itself not be used to detect or resolve the problem. I found this episode of the nature podcast - "How AI works is often a mystery — that's a problem": https://www.nature.com/articles/d41586-023-04154-4 https://www.nature.com/articles/d41586-023-04154-4 - very useful in a 'thank goodness someone else has done the work of being coherent so I don't have to' way.
aoeusnth1 2y ago
Do your kids have internal structures which relate meaningfully to their functionality, which allow a mechanistic explanation of what they learned in school?
- intended 2y ago
  Not sure if this is satirical, but absolutely yes. Heck we have everything from fields of study, to professions that cover this. Neurology, psychology, counseling, teaching, amongst a few. All things being equal, If a kid didn’t pick up a concept, I can sit with them and figure out what happened, and we can both work towards making sure its cleared up.
pat64 2y ago
Given you just mentioned semantic search (a term I haven’t heard in over 15 years) and the other breadcrumbs in this comment, you wouldn’t by chance be an English lecturer living in Ireland would you?
- singingfish 2y ago
  me? No. Ex trainee neruopsychologist and failed academic who was in the right place at the right time back in the mid 90s who didn't pick up computers for professional interest until the mid-late 2000s after getting excited by Neil Stephenson's Cryptonomicon when I was looking for a career change. These days I identify as an international computer hacker, but mainly to take the piss (due to the tiny element of truth sitting underneath)

ninetyninenine 2y ago

If we ever hit agi this overall point: “ the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition.” Will remain true imho. We will never fully intuit AI or understand it outside of some brute force abstraction like a token predictor or best fit curve.

steve_adams_86 2y ago

I didn’t see this at first, and I was fairly shaken by the potential impact on the world if their progress didn’t stop. A couple generations showed meaningful improvements, but now it seems like you’re probably correct. I’ve used these for years quite intensively to aid my work and while it’s a useful rubber duck, it doesn’t seem to yield much more beyond that. I worry a lot less about my career now. It really is a tool that creates more work for me rather than less.

iman453 2y ago
Would this still hold true in your opinion if models like O3 become super cheap and bit better over time? I don't know much about the AI space, but as a vanilla backend dev also worry about the future :)
- root_axis 2y ago
  Let's see how O3 pans out in practice before we start setting it as the standard for the future.
  - varelse 2y ago
    Mamba-ish models are the breakthrough to cheap inference if they pan out. Calling a dead-end already is just silly.
- sydd 2y ago
  We know that OpenAI is verz good at least in one thing: generating hype. When Sora was announced everyone thought that this will be revolutionary. Look at how it looks like in production. Same when they started floating rumours that they have some AGI prototype in their labs. They are the Tesla of the IT world, overpromise and under deliver.
  - WhyOhWhyQ 2y ago
    It's a brilliant marketing model. Humans are inherently highly interested in anything which could be a threat to their well-being. Everything they put out is a tacit promise that the viewer will soon be economically valueless.
    - bilbyx 2y ago
      I hope people will come to the realisation that we have created a good plagiarizer at best. The "intelligence" originates from the human beings who created the training data for these LLMs. The hype will die when reality hits.
  - gom_jabbar 2y ago
    Hype is very interesting. The concept of Hyperstition describes fictions that make themselves real. In this sense, hype is an essential part of capitalism: "Capitalization is [...] indistinguishable from a commercialization of potentials, through which modern history is slanted (teleoplexically) in the direction of ever greater virtualization, operationalizing science fiction scenarios as integral components of production systems." [0] "Within capitalist futures markets, the non-actual has effective currency. It is not an "imaginary" but an integral part of the virtual body of capital, an operationalized realization of the future." [1] This corresponds to the idea that virtual is opposed to actual, not real. [0] https://retrochronic.com/#teleoplexy-12 https://retrochronic.com/#teleoplexy-12 [1] https://retrochronic.com/#on-accelerate-2b https://retrochronic.com/#on-accelerate-2b
trhway 2y ago
>I worry a lot less about my career now. It really is a tool that creates more work for me rather than less. when i was a team/project leader the largest part of my work was talking to the reports on what needs to be implemented and how they are going to implement it and the current progress of the implementation, how to interface the stuff, what are the issues and how to approach the troubleshooting, what are the next steps, etc. with occasional looking into/reviewing the code - it looks to me what working with coding LLM would soon be quite similar to that.
trod1234 2y ago
Many of the major harms of these things were neglected, and downplayed even to this day people don't recognize just how changed the world has become. The mere delusion that AI will replace work has been used to justify mass layoffs. The persistence of indistinct ghost jobs that are generated by computer for pennies to flood and bind with prospective job seekers (similar to RNA interference), has resulted in severe brain drain in many fields. Worse, the fact these people have often been forced into poverty as a result will have a lasting impact. You might have planned for up to a year out of work pre-AI and had the financial resources, but now how long does it take? Conversion ratios for the first step have changed by two magnitudes or order (from x100 to x10,000). What are the odds of these people finding a job given their finite time, and requirements that are un-automatable for submission (nil). The media keeps claiming that everything is getting better, the stats say so (while neglecting the fact that the stats are being manipulated to the point of uselessness/fabricated), but you have 1/3 of welfare payouts now going to these people in California (in the US), just for basic food. When you can't find work, you go where the work is abandoning the bad economic investment and choice you made regardless of how competent you were. It is a psychologically sticky decision. When there is no chance at finding work, you get desperate, and many desperate people turn to crime and unrest. This was foreseen by a number of very intelligent people many decades ago, and ignored following business as usual. The mere demonstration that we are unable to react in time is what gave engineers such great pause to write about these things, as far back as in the 70s. Hysteresis is a lagging time problem where you can't react fast enough to avert catastrophic failure given chaotic conditions, leaving survival up to chance. Its the worst type of engineering problem with real consequences. Given how western society is structured dependently on labor exchange, its a perfect weapon of chaos and debasement in the value of labor, that effectively destroys half of its underlying economic structure (factor markets). This forces sieving conditions of wealth that become spinodal, and eventually falter under their constraints and spiral into deflationary trends over time. Business wins so much that they lose everything. Its quite a disadvantaged environment and the general trend is that everyone is ignoring the pink elephant. Actions (and inaction) have consequences. When people don't listen and take appropriate action, consequences get dire, it hits the fan.
- cgio 2y ago
  I agree that we are often missing in our analyses the true materialised impact of expectations by focusing on the validity of said expectations instead. Organisations, even if not laying off, are pausing hiring plans with a conviction that AI will replace some of the workers. It then becomes a self fulfilling prophecy to some extent. It doesn’t matter if it can, what matters is if it will. And to assume that people won’t place a bet is futile, as everyone does, and even if it’s wrong the market will allocate the losses to the baseline.

hammock 2y ago

What are your thoughts on neuro-symbolic integration (combining the pattern-recognition capabilities of neural networks with the reasoning and knowledge representation of symbolic AI) ?

bionhoward 2y ago
Seems like the symbolic aspect is poorly defined and it’s too unclear to be useful. Always sounds cool, but what exactly are we talking about?
- hammock 2y ago
  I’m not an AI expert, but from my armchair I might draw a comparison between functional (symbolic rule- and logic-based AI) and declarative (LLM) programming languages

gizmo 2y ago

Previous generations of neural nets were kind of useless. Spotify ended up replacing their machine learning recommender with a simple system that would just recommend tracks that power listeners had already discovered. Machine learning had a couple of niche applications but for most things it didn't work. This time it's different. The naysayers are wrong. LLMs today can already automate many desk jobs. They already massively boost productivity for people like us on HN. LLMs will certainly get better, faster and cheaper in the coming years. It will take time for society to adapt and for people to realize how to take advantage of AI, but this will happen. It doesn't matter whether you can "test AI in part" or whether you can do "exhaustive whole system testing". It doesn't matter whether AIs are capable of real reasoning or are just good enough at faking it. AI is already incredibly powerful and with improved tooling the limitations will matter much less.

jfengel 2y ago
From what I have seen, most of the jobs that LLMs can do are jobs that didn't need to be done at all. We should turn them over to computers, and then turn the computers off.
- kube-system 2y ago
  They're good at processing text. Processing text is a valuable thing that sometimes needs to be done. We still use calculators even though the profession we used to call "computer" was replaced by them.
  - jonasced 2y ago
    But here reliability comes in again. Calculators are different since the output is correct as long as the input is correct. LLMs do not guarantee any quality in the output even when processing text, and should in my opinion be verified before used in any serious applications.
    - kube-system 2y ago
      > Calculators are different since the output is correct as long as the input is correct. That isn't really true.[0] The application of calculators to a subject matter is something that does need to be considered in some use cases. LLMs also have accuracy considerations, and although it may be to a different degree, the subject matter to which they're applicable has a broad range of acceptable accuracies. While some textual subject matter demands a very specific answer, some doesn't: For example, there may be hundreds or thousands of various ways to summarize a text that could be accurate for a particular application. 0: example: https://www.reddit.com/r/calculus/comments/upjdn4/why_do_all_calculators_not_get_the_exact_same/ https://www.reddit.com/r/calculus/comments/upjdn4/why_do_all...

dlkf 2y ago

> Current AI systems have no internal structure that relates meaningfully to their functionality In what sense is the relationship between neurons and human function more “meaningful” than the relationship between matrices and LLM function? You’re correct that LLMs are probably a dead end with respect to AGI, but this is completely the wrong reason.

mmcnl 2y ago
Human intelligence has a track record of being useful for thousands of years.
imtringued 2y ago
The neurons are always learning whereas the matrices don't change.
- dlkf 2y ago
  I mean, the matrices obviously change during training. I take it your point is that LLMs are trained once and then frozen, whereas humans continuously learn and adapt to their environment. I agree that this is a critical distinction. But it has nothing to do with “meaningful internal structure.”
singingfish 2y ago
Yeah the internal representation of organic neural networks are also weird - check out the signal processing that occurs between the retina and the various parts of the visual cortex before any decent information can emerge from the signal - David Marr's 1980s book Vision is a mathematically chewy treatise on this. This leads me to start thinking that human intuition may well caused by different neural network subsystems feeding processed data into other subsystems where consciousness and thus intuition and explanation emerges. Organic neural networks are pretty energy efficient in comparison- although still decently inefficient compared to other body systems - so there is the capacity to build things out to the scale required, assuming my read on what's going on there is correct, that is. So it's not clear to me that the energy inefficiency of ANNs can be sufficiently resolved to enable these multiple quasi-independent subsystems to be built at the scale required. Not even if these interesting looking trinomial neural nets which are matrix addition based rather than multiplication come to dominate the ANN scene. While I was thinking this comment through I realised there's a possible interpretation wherin human activity induced climate change is an emergent property of the relative energy inefficiency of neural architecture.

ftlisnotftl 2y ago

[dead]

busyant 2y ago

> "Current AI systems have no internal structure that relates meaningfully to their functionality". I'm curious as to why you feel this needs to be true? Or to put it another way, what would an AI structure look like to be more meaningfully connected to its function? Not trying to flame. I always feel that I can't think quite deeply enough about these issues, so I'm worried that I'm missing something 'obvious'.

fragmede 2y ago
AlphaGo had an artificial neural network that was specifically trained in best moves and winning percentages. An LLM trained on text has some data on what constitutes winning at go, but internally doesn't have a ANN specifically for the game of go.
- deleted 2y ago
  [deleted]
- busyant 2y ago
  > AlphaGo had an artificial neural network that was specifically trained in best moves and winning percentages. An LLM trained on text has some data on what constitutes winning at go, but internally doesn't have a ANN specifically for the game of go. This isn't addressing what the original commenter was referring to.
singingfish 2y ago
The reasoning is quite subtle, and because I'm not a very coherent guy I have problems expressing it. In the LLM space there are a whole bunch of pitfalls around overfit (largely solvable with pretty standard statistical methods) and inherent bias in training material which is a much harder to problem to solve. The fact that the internal representation gives you zero information on how to handle this bias means the tool can itself not be used to detect or resolve the problem. I found this episode of the nature podcast - "How AI works is often a mystery — that's a problem": https://www.nature.com/articles/d41586-023-04154-4 https://www.nature.com/articles/d41586-023-04154-4 - very useful in a 'thank goodness someone else has done the work of being coherent so I don't have to' way.

aoeusnth1 2y ago

Do your kids have internal structures which relate meaningfully to their functionality, which allow a mechanistic explanation of what they learned in school?

intended 2y ago
Not sure if this is satirical, but absolutely yes. Heck we have everything from fields of study, to professions that cover this. Neurology, psychology, counseling, teaching, amongst a few. All things being equal, If a kid didn’t pick up a concept, I can sit with them and figure out what happened, and we can both work towards making sure its cleared up.

pat64 2y ago

Given you just mentioned semantic search (a term I haven’t heard in over 15 years) and the other breadcrumbs in this comment, you wouldn’t by chance be an English lecturer living in Ireland would you?

singingfish 2y ago
me? No. Ex trainee neruopsychologist and failed academic who was in the right place at the right time back in the mid 90s who didn't pick up computers for professional interest until the mid-late 2000s after getting excited by Neil Stephenson's Cryptonomicon when I was looking for a career change. These days I identify as an international computer hacker, but mainly to take the piss (due to the tiny element of truth sitting underneath)