7 ms·
I appeciate how Schmidhuber puts focus on non-US-centric history of Deep Learning and modern AI. He sometimes overdoes these. Even when something is not relate
by truth_ 5y ago
I appeciate how Schmidhuber puts focus on non-US-centric history of Deep Learning and modern AI.
He sometimes overdoes these. Even when something is not related to Deep Learning and modern AI. He tries to push very narrow and one-sided views without much room for nuance.
For example- "Turing used his (quite inefficient) model only to rephrase the results of Gödel and Church on the limits of computability."
He is trying to say that Turing did nothing new, his work is a mere extension of Godel's (!), and he only accounted for practical computational reality. That is not true.
He goes on further by saying, "Later, however, the simplicity of these machines made them a convenient tool for theoretical studies of complexity." Like, Turing and Church's work was only good for that. I digress. I believe that Turing's work is more seminal and paved the way for modern general computers.
Another bit here says "The formal models of Gödel (1931-34), Church (1935), Turing (1936), and Post (1936) were theoretical pen & paper constructs that cannot directly serve as a foundation for practical computers."
It is putting the works of Turing, Post, Church, and Godel all together. Like they are the same. I see the logic as they are all impractical, but they are different levels of impractical. Putting them all under one bracket like that makes no sense.
I respect Schmidhuber a lot, but some of his behaviour is borderline childish which are more pronounced in his claims about AI history. You should be skeptical about what he says (and what anyone says).
Despite all this, I did not know many things written in this article. I learned a lot and had fun reading it.
I first read about Zuse in Isaacson's Innovators book. Did not know that he used a high-level language for a chess program. Fascinating. He is very underrated.
- maweki 5y agoBoth the discovery of Lambda Calculus and the Turing Machine as models of computation are quite important and immediately useful, I'd say. Though very formal, just a little bit of syntactic sugar and eye-squinting make for useful programming constructs and more or less direct lines to early as well as modern programming languages.
- ProfHewitt 5y agoDevelopment of Lambda Calculus [Church 1931] and Turing Machine [Turing 1936] were fundamental achievements. However, they are inadequate models of computation because there are digital computations they cannot implement. See the following for an example: https://papers.ssrn.com/abstract=3603021 https://papers.ssrn.com/abstract=3603021
- kamray23 5y agoOnly true for nondeterministic systems. Which all computers are as there is physical nondeterminism. However, it can (mostly) be ignored in any practical application of these theories, because the chances of you ending up in a nondeterministic computation at all are minute, never mind an unbounded one. And we are talking about practical applications here. Also, isn't that your own paper?
- ProfHewitt 5y agoMany-core computer systems and networked computer systems depend crucially on indeterminacy in order to gain performance. See the following https://papers.ssrn.com/abstract=3459566 https://papers.ssrn.com/abstract=3459566
- wizzwizz4 5y agoMany-core and networked computer systems fight indeterminacy. My networking algorithms would be a lot simpler if I could guarantee that everything operated in lockstep; the fact that I have to discard lockstep to get performance is because lockstep is a high-cost abstraction over a high-entropy (so, basically non-deterministic) underlying reality, not because indeterminacy is somehow inherently better. It's a concession.
- ProfHewitt 5y agoA concession to the physical world?
- sn41 5y agoThis is classic Schmidhubris, a mixture of truth and exaggeration. Turing's paper is a landmark, and includes even an equivalence proof of the power of the lambda calculus and his universal machine. In doing so, he came up with a fixed-point combinator, an applicative combinator, now called the Turing fixed point combinator [1]. Especially appropriate on a website run by the Y combinator. Turing's paper should run as a TV ad "But wait, there's more..." Many applications of topology (he argues why the alphabet should be finite, rather than simply saying that it is finite, by using the Bolzano-Weierstrass theorem, essentially), combinatorics, logic, functional programming - when FP was hardly a decade old and largely unknown outside Princeton - and all from an undergraduate course project report! [1] https://en.wikipedia.org/wiki/Fixed-point_combinator#Other_fixed-point_combinators https://en.wikipedia.org/wiki/Fixed-point_combinator#Other_f...
- P-NP 5y agoI see neither hubris nor exaggeration. Before Turing, it was Church who proved the equivalence of the power of his lambda calculus and Gödel's universal model.
- ProfHewitt 5y ago[Church 1935] proved the computational undecidability of the halting problem before [Turing 1936], which was written up in a hurry after [Church 1935]. The Y-combinator does not work for strongly-typed systems. Instead recursion must be explicitly added as an additional primitive to the lambda calculus. See following for more information: https://papers.ssrn.com/abstract=3418003 https://papers.ssrn.com/abstract=3418003
- sn41 5y agoIt's an honor to receive a response from you. I agree on all points, especially the lack of combinators for strongly typed systems. (As I understand it, it is impossible to assign types to such terms.) My main point was that disrespect for Turing's paper is largely unjustified.
- 1024core 5y agoUS and EU researchers have been on parallel and competing tracks for a long time, ever since the days of Prolog and Lisp.
- bo1024 5y agoAgree, the quote lumping the models of computation together is very misleading. Turing’s machines are immediately obvious how to mechanize and automate, even with very old technology. That was the key difference. Then proving them equivalent to mu-recursive functions and lambda calculus showed that these models of computation actually deserved to be called such. It proved that they could be implemented on a machine with no human ingenuity or creativity required to carry out the steps, which is the heart of what it means to be computable.
- P-NP 5y agoNeither did Gödel's automatic theorem prover require "human ingenuity or creativity required to carry out the steps" to enumerate all the theorems from the axioms, which are also enumerable. Gödel really described "the heart of what it means to be computable."
- slipframe 5y agoDe-emphasizing Turing as an instance of focusing on non-US-centric history? Turing was not American! Furthermore, Church was American, but that quote is aggrandizing to Church while diminishing of Turing. I don't think that quote is an example of what you claim.
- mudlus 5y agoThe author hasn't read any David Deutsch, I can tell.
- YeGoblynQueenne 5y agoThe problem is that if you don't like Schmidhuber's interpretation of the history of logic, computer science and AI, and you want to propose a better interpretation, you need to know that history at least as well as he does.
- truth_ 5y agoThere are already better interpretation. And "know that history as well as he does" approach is not suited for knowledge fields. The notion that you need to know as much as a creator to critic their works is childish and immature. I might not know as much as him about the field, but I am sufficiently well versed about the topics discussed in this article to point out his biases and fallacies. I am a fan of people like LeCun and Schmidhuber, but I don't worship them. I have read many articles and listened to many talks by JS, and the kind of biases present in this article is nothing new.
- YeGoblynQueenne 5y ago>> The notion that you need to know as much as a creator to critic their works is childish and immature. Schmidhuber is not a "creator" in the sense that Pekinpah, or Coppola, are "creators" and his works are not works of art that can be experienced and criticised by anyone based on aesthetics alone (even though aesthetics also can take a lot of honing). In any case my comment was that, to propose a better interpretation of the work described by Schmidhuber (not "to point out their biases and fallacies", as you say), you need to know that work at least as well as him. Otherwise, you find yourself criticising the work of someone who knows more than you know. I don't know about you, but I've done that in the past and it made me look and feel like a fool. You may think you are "sufficiently well versed about the topics discussed in this article" but if you so dismiss the need for general knowledge, beyond the contents of one article, then there may easily be any amount of knowledge that you're missing and that makes you think you know all you need, simply because you don't know what you don't know. Remember what Socrates, the wisest of men, said: "I know one thing, that I know nothing, Jon Snow". I may have garbled that a bit there. But you get the picture. Arrogance cannot replace knowledge. Never assume that you know more than you need to know to prove wrong an expert in a field of knowledge as deep and with as long a history as AI, just because you have an internet connection. Finally, nobody "worships" LeCun or Schmidhuber. What people admire is their knowledge and the hard work they put into acquiring that knowledge. And the reason they admire it is because anyone who's ever try to get to grips with a scientific subject understands how much hard work it takes to acquire knowledge.
- P-NP 5y agoYou write that he is "trying to say that Turing did nothing new, his work is a mere extension of Godel's (!), and he only accounted for practical computational reality. That is not true." However, Schmidhuber's text on Gödel/Church/Turing/Post/Zuse is much more nuanced than that: "In 1935, Alonzo Church derived a corollary / extension of Gödel's result by showing that Hilbert & Ackermann's famous Entscheidungsproblem (decision problem) does not have a general solution.[CHU] To do this, he used his alternative universal coding language called Untyped Lambda Calculus, which forms the basis of the highly influential programming language LISP. In 1936, Alan Turing introduced yet another universal model which has become perhaps the most well-known of them all (at least in computer science): the Turing Machine.[TUR] He rederived the above-mentioned result.[T20](Sec. IV) Of course, he cited both Gödel and Church in his 1936 paper[TUR] (whose corrections appeared in 1937). In the same year of 1936, Emil Post published yet another independent universal model of computing,[POS] also citing Gödel and Church. Today we know many such models. Nevertheless, according to Wang,[WA74-96] it was Turing's work (1936) that convinced Gödel of the universality of both his own approach (1931-34) and Church's (1935). What exactly did Post[POS] and Turing[TUR] do in 1936 that hadn't been done earlier by Gödel[GOD][GOD34] (1931-34) and Church[CHU] (1935)? There is a seemingly minor difference whose significance emerged only later. Many of Gödel's instruction sequences were series of multiplications of number-coded storage contents by integers. Gödel did not care that the computational complexity of such multiplications tends to increase with storage size. Similarly, Church also ignored the spatio-temporal complexity of the basic instructions in his algorithms. Turing and Post, however, adopted a traditional, reductionist, minimalist, binary view of computing—just like Konrad Zuse (1936).[ZU36] Their machine models permitted only very simple elementary instructions with constant complexity, like the early binary machine model of Leibniz (1679).[L79][LA14][HO66] They did not exploit this back then—for example, in 1936, Turing used his (quite inefficient) model only to rephrase the results of Gödel and Church on the limits of computability. Later, however, the simplicity of these machines made them a convenient tool for theoretical studies of complexity. (I also happily used and generalized them for the case of never-ending computations.[ALL2])"
- P-NP 5y agoYou also write that Turing's work "paved the way for modern general computers." According to the text, however, this practical part really started with Konrad Zuse's patent application of 1936: "The formal models of Gödel (1931-34), Church (1935), Turing (1936), and Post (1936) were theoretical pen & paper constructs that cannot directly serve as a foundation for practical computers. Remarkably, Konrad Zuse's patent application[ZU36-38][Z36][RO98] for the first practical general-purpose program-controlled computer also dates back to 1936. It describes general digital circuits (and predates Claude Shannon's 1937 thesis on digital circuit design[SHA37]). Then, in 1941, Zuse completed Z3, the world's first practical, working, programmable computer (based on the 1936 application). Ignoring the inevitable storage limitations of any physical computer, the physical hardware of Z3 was indeed universal in the "modern" sense of Gödel, Church, Turing, and Post—simple arithmetic tricks can compensate for Z3's lack of an explicit conditional jump instruction.[RO98]"
- musicale 5y agoI'm puzzled as to why the author seems to think that the incompleteness theorem says anything important about AI (rather than just pointing out that self-contradictions can be a potentially irritating problem in logical systems - something that was known since antiquity.)
- P-NP 5y agoHe writes: "Thus he identified fundamental limits of algorithmic theorem proving, computing, and any type of computation-based AI." An automatic theorem prover is a kind of AI. Gödel showed its limitations. There have been entire conferences dedicated to Gödel and AI.
- musicale 5y ago> An automatic theorem prover is a kind of AI. Gödel showed its limitations. Which of those limitations apply only to AI and not to human intelligence and mathematical reasoning? Is it somehow surprising that AI can't prove mathematical contradictions to be true?