3 ms·

Don't you think, though, that each boom and bust cycle leaves us closer to real accomplishments? We now have protein folders and superhuman Go players -- that'

Don't you think, though, that each boom and bust cycle leaves us closer to real accomplishments? We now have protein folders and superhuman Go players -- that's new. I agree that ML ("AI") is currently at the alchemy stage. And, guess what? A neural network isn't even Turing complete! [citation needed - correct me if I am wrong.] So ML can only compute SOME functions. AGI, when it comes, and believe me, it will, will have ML as part of its structure, but only a small part.

ashtonkem 5y ago
I have no idea if each boom brings us closer. Scientific discovery is not exactly a linear process; we can’t observe before the fact if we’re getting closer or are on a dead end path, and that’s even assuming we can ever get there at all. But it’s hard to deny that each boom doesn’t give us something useful. Neural Nets might not exactly make an AGI, but they do have uses.
6gvONxR4sf7o 5y ago
I mean, each cycle does leave us with real accomplishments. If the question is whether it leaves us closer to AGI, then it's an open question. Like, when AGI happens, it will certainly trace its roots back to things happening in each period, but its roots will also go back to Gauss and Newton and co, so nobody knows whether it gets us closer in the way you probably mean.
Dr_Birdbrain 5y ago
ML is Turing complete in the sense that every computable function can be approximated to arbitrary precision by a 3-layer neural network. Classic result from the 90s, the paper in question (iirc) is titled something like “neural networks are universal approximators” Turing machines also can only approximate to arbitrary precision, so the computation models are equivalent.
randcraw 5y ago
Of course that depends on what you mean by "real accomplishments". It seems to me that that deep nets have effectively maximized the potential of using gradient pursuit to model patterns. But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall. Luckily human perception makes extensive use of gradients, as does most search, so problems in those arenas have been unsurprisingly amenable to solution using deep nets (vision, speech, game play, etc). But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. Will deep nets eventually fill that void? I doubt it. You can convert only so many problems with big data into gradients to pursue them efficiently with DL before that transformation trick runs out of steam. Personally I think deep net language modeling is one of those areas, and soon we'll encounter the limit of their generalizable contextual phrase association. Then because deep nets are so difficult to selectively revise or extend the specifics that they have learned, the vanguard of ever more complex deep nets (transformers) will eventually sink beneath their own weight, taking the last best hope for DL-based general AI with them.
- nicolapede 5y ago
  > But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. This seems to me quite a deep insight. But how would you formally define a process that is gradient-based?
- whimsicalism 5y ago
  > But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall, We're making some progress on the convexity bit (heat functions, RL, etc.), but yes, there are other areas of statistical research all involved in trying to solve those sorts of problems. ML is not necessarily a panacea, but just because you can point to problems that it doesn't solve doesn't mean it has no "real accomplishments." > DL, like human cognition, are NOT driven by gradients. ???
deleted 5y ago
[deleted]

ashtonkem 5y ago

I have no idea if each boom brings us closer. Scientific discovery is not exactly a linear process; we can’t observe before the fact if we’re getting closer or are on a dead end path, and that’s even assuming we can ever get there at all. But it’s hard to deny that each boom doesn’t give us something useful. Neural Nets might not exactly make an AGI, but they do have uses.

6gvONxR4sf7o 5y ago

I mean, each cycle does leave us with real accomplishments. If the question is whether it leaves us closer to AGI, then it's an open question. Like, when AGI happens, it will certainly trace its roots back to things happening in each period, but its roots will also go back to Gauss and Newton and co, so nobody knows whether it gets us closer in the way you probably mean.

Dr_Birdbrain 5y ago

ML is Turing complete in the sense that every computable function can be approximated to arbitrary precision by a 3-layer neural network. Classic result from the 90s, the paper in question (iirc) is titled something like “neural networks are universal approximators” Turing machines also can only approximate to arbitrary precision, so the computation models are equivalent.

randcraw 5y ago

Of course that depends on what you mean by "real accomplishments". It seems to me that that deep nets have effectively maximized the potential of using gradient pursuit to model patterns. But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall. Luckily human perception makes extensive use of gradients, as does most search, so problems in those arenas have been unsurprisingly amenable to solution using deep nets (vision, speech, game play, etc). But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. Will deep nets eventually fill that void? I doubt it. You can convert only so many problems with big data into gradients to pursue them efficiently with DL before that transformation trick runs out of steam. Personally I think deep net language modeling is one of those areas, and soon we'll encounter the limit of their generalizable contextual phrase association. Then because deep nets are so difficult to selectively revise or extend the specifics that they have learned, the vanguard of ever more complex deep nets (transformers) will eventually sink beneath their own weight, taking the last best hope for DL-based general AI with them.

nicolapede 5y ago
> But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. This seems to me quite a deep insight. But how would you formally define a process that is gradient-based?
whimsicalism 5y ago
> But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall, We're making some progress on the convexity bit (heat functions, RL, etc.), but yes, there are other areas of statistical research all involved in trying to solve those sorts of problems. ML is not necessarily a panacea, but just because you can point to problems that it doesn't solve doesn't mean it has no "real accomplishments." > DL, like human cognition, are NOT driven by gradients. ???

deleted 5y ago

[deleted]