8 ms·

There's a famous paper by Dijkstra where he claims that using anthropomorphic terms for computers is a sign of immaturity of the discipline. I used to think tha

• ogogmad 6y ago
  > Machine learning is linear algebra, nothing more nothing less Neural nets are non-linear. If they were linear, that would be linear regression. I actually don't get what your point is.
  • qsort 6y ago
    I'm not claiming it's literally a linear model, obviously. The point is that they are relatively simple mathematical models (that actually use linear algebra inside, https://en.wikipedia.org/wiki/Artificial_neuron https://en.wikipedia.org/wiki/Artificial_neuron does the output formula remind you of anything?) different in scope/use but not conceptually from linear regressions. Attaching (meta)physical meanings ("neurons", "intelligence") to any of that is dubious at best.
    • mbeex 6y ago
      Without non-linearity you are mathematically provable not able to get solutions for most contemporary problems. Refusing the general AI hype bullshit and over-simplifying the hard foundations are quite different things, IMO.
      • qsort 6y ago
        I don't disagree with this. It was a misuse of terminology on my part, my broader point has absolutely nothing to do with the mathematical details of neural networks.
        • mbeex 6y ago
          Reason for the nitpicking is, that for me - as a trained mathematician - and seemingly some other commenters, the gap between linearity and non-linearity is yet inside mathematics quite substantial. This is true also in general. Non-linearity might be not sufficient, but it is certainly necessary for any definition and/or serious discussion of AGI.
        • unishark 6y ago
          Just insert the words "these days" and "mostly", and you're pretty much covered. Also a honorable mention for calculus perhaps. Even decision trees use gradient boosting very commonly.
    • ogogmad 6y ago
      How is this different from saying that a brain is "just" a bunch of atoms, which individually are quite simple? It's when you put the individually simple pieces together and run an optimiser that you get something clever. The result of that optimiser is a complex and very large array of weights.
      • qsort 6y ago
        The difference is that we have neither the scientific knowledge nor the computational power to make an exact model of the human brain (nor we are anywhere close to either requirement, AFAIK). In other words, we cannot pinpoint the exact sequence of operations and events that produce a certain behavior, while we absolutely can do that with ML models (you could in principle run machine learning models with pen and paper, obviously). If you could show me a program that emulates my brain to a certain degree of precision (it's unlikely that such a program would be a deterministic one, by the way), I would have no problem accepting that my brain is not meaningfully different from that program.
        • malkarouri 6y ago
          So you are defining intelligence to be whatever the brain does? That is certainly a philosophical position.
          • qsort 6y ago
            "Intelligence" is a word that, etymologically and semantically, is related to human or human-like capabilities. You wouldn't say that a leaf floating on a lake is swimming, and likewise, claiming that computers are "learning" or "intelligent" is at best a thin analogy and at worst a mischaracterization of the process. What's happening in my brain is something we don't have full scientific knowledge of, but we know it's not x86 machine code. While the two processes may be in many ways similar, conflating the two into this ill-defined concept of "intelligence" is a discussion about semantics more than anything else.
            • 6y ago
  • mbeex 6y ago
    > I actually don't get what your point is. I do, there is a difference between an universal approximator and intelligence. But I share the objection. And there are more parts, like graph theory, calculus, statistics or more advanced stochastic approaches.
• scottlocklin 6y ago
  I hate to be that guy, but most ML is definitely not linear algebra. The most popular and useful ML algorithms are boosted trees and random forests; they're more or less geometric approximators (which are all nonlinear). And as ANN nerds like to say, there's plenty of nonlinearity in neural approaches. But mostly I agree with you: modeling is horse shit, worshipping models (linear and otherwise) is childish idolatry and Dijkstra was completely correct about just about everything (speaking as an APL bum).
  • qsort 6y ago
    > I hate to be that guy, but most ML is definitely not linear algebra... Yes, I'm not disagreeing with any of that (see also my other reply), it was a misuse of terminology on my part. > But mostly I agree with you: modeling is horse shit, worshipping models... Absolutely. The way I see it is that models are like theorems: they are useful and worthy of being studied, but if you forget to validate that your hypotheses are correct, you get wrong results: garbage in, garbage out. An alarming amount of stuff these days is the equivalent of people trying to apply the pythagorean theorem to a non-rectangle triangle and then complaining the results don't match.
    • sebastianconcpt 6y ago
      Good ponits you guys. I think it would be fair to say that "AI" is nothing but algebra ran by an automata. The rest is a mix of hype and nerds taking advantage to manipulate people that has this concept out of their intellectual reach. Depending on the case, this can enable evil at global scale. As always with humanity, ideology is always a problem.
      • ericd 6y ago
        This kind of argument is common in this sort of discussion, and more than a bit reductionistic. Extreme complexity often comes from very simple processes at scale. Your computer's abilities are the result of little switches flipping between two states.
• versteegen 6y ago
  And here's a famous old paper by ‎McDermott that makes the same point but specifically for AI. Using terms like "understanding" to mean the effect of a certain algorithm is fooling yourself into thinking you've accomplished something you haven't. Drew ‎McDermott, 1976, Artificial intelligence meets natural stupidity https://www.csee.umbc.edu/courses/graduate/691/fall19/07/papers/AI_Meets_Natural_Stupidity.pdf https://www.csee.umbc.edu/courses/graduate/691/fall19/07/pap...
  • qsort 6y ago
    Thanks, I hadn't heard of it. Excellent read. "However, to say anything good about anyone is beyond the scope of this paper." This is the best conclusion for a paper I've ever seen :D
    • st1x7 6y ago
      And the opening: > As a field, artificial intelligence has always been on the border of respectability, and therefore on the border of crackpottery. This was written in 1976.
  • auggierose 6y ago
    I don't know the paper yet, but I love the title
• kerng 6y ago
  I think ML is a bit too complex (different approaches trees, DNNs,...) to oversimplify it like that. One could say, it's just math - but even experts can't really tell some of the why it works behind back propagation in DNNs. A lot of research is empirical, like switching to ReLU... ML is a commodity now, even popular with artists to create art. Being a commodity means you dont have to understand the inner workings to benefit from it. I don't see anything wrong with that, but a lack of proof for biases/backdoors/bugs in models is an issue that I'm worried about when its applied too enthusiastically in critical areas. Article is weak though, and hard to follow conclusion.
• ericjang 6y ago
  I searched for the "famous paper" in question and could not find it - can you point us to a link? So far I find two of Dijkstra's talks/writing on anthropomorphism [1] https://www.cs.utexas.edu/users/EWD/transcriptions/EWD05xx/EWD540.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD05xx/E... [2] https://www.cs.utexas.edu/users/EWD/transcriptions/EWD09xx/EWD936.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD09xx/E...
  • qsort 6y ago
    I was thinking about the second one you linked, even though it's not exactly what I'm saying, one of the points being made is similar. Also the first half of this: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD898.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E... is related.
• gumby 6y ago
  > There's a famous paper by Dijkstra where he claims that using anthropomorphic terms for computers is a sign of immaturity of the discipline. If you take the word "computer" out of the sentence this statement is likely backwards. Putting the word computer back in doesn't change that. Look, for example, at Dennett's seminal book, "The Intentional Stance" for a philosophical discussion of the expressive and reasoning power behind statements like "The thermostat tries to keep the temperature at 60 degrees".
  • seanhunter 6y ago
    Exactly. Anthropomorphic language is such a useful and normal way of thinking and expressing ourselves. If you say "the classifier thinks this roadsign is a truck" everyone knows you mean "the classifier has labeled this roadsign as a probably a truck with a high confidence". No-one seriously would take you to mean the classifier thinks in the same way a person thinks any more than if you say "The toaster wants to pop up but it's jammed," they would take you to mean the toaster has a yearning like unrequited love. If we remove our ability to use language in this way we give up on a mental model that is incredibly useful for simplifying lots of explanations and reduce ourselves to a very clumsy literalism instead. As long as we understand the limitations of this model and are clear that it's not literally what is going on it's very useful.
    • qsort 6y ago
      My problem is not with natural language in and of itself, it's obvious that natural language is extremely useful. I'm arguing one should be extremely careful, however. Natural language doesn't have deduction. It doesn't have definitions. It doesnt't have ways to restrict us from making vague statements. It is subject to non-unique interpretations. This is particularly relevant when technicians and scientists speak to the general public. Let me quote one of your examples: > "the classifier thinks this roadsign is a truck" everyone knows you mean "the classifier has labeled this roadsign as a probably a truck with a high confidence". The fact that the classification is probabilistic is obvious to me and to you, but nothing in the natural wording suggests that, and it's a pretty important caveat. It's not enough for us to understand the limitations of natural language, everyone in the conversation has to, for it to be a useful tool.
• sgt101 6y ago
  There's more to AI than Machine Learning; even if you only use Machine Learning you need some sort of system to embed it. Unless you are doing academic research.
• aeturnum 6y ago
  I knew an older programmer who used the phrase "parasitic interpretation" to describe the human tendency to assume a things' name related to its function. This is the idea that something called `loop_index` is the index for a loop, when it could actually be anything. He strongly felt that semantic content in variable names should be minimized. I haven't run into the phrase anywhere else, and Google doesn't turn up any programming-related results that I can tell, but I've found the idea endlessly useful. In talking about programs and the goal of programs, we often use language that describes things we have not yet been able to construct. The language has a parasitic relationship to the thing it names - misdirecting our understanding of the machine it sits on. I try to practice a lot of skepticism around language and computers. It is very easy to name a thing and much harder to make the machine behave in the way the language suggests.
  • bonoboTP 6y ago
    You may like the article "Troubling Trends in Machine Learning Scholarship", Lipton & Steinhardt, 2018 https://arxiv.org/abs/1807.03341 https://arxiv.org/abs/1807.03341 Especially the trend of "misuse of language, e.g., by choosing terms of art with colloquial connotations or by overloading established technical terms"
  • qsort 6y ago
    Yeah, this is an interesting take. Broadly speaking I agree, but you have to be pragmatic about that. There exist some accepted conventions it's just senseless to break just to make a point, for example, if you are defining a ring, making + distributive over \* rather than the opposite as is usual is not inherently wrong, but I would classify that as bad notation. Similarly in OOP, you could technically name getters in any way you want, but why would you not conform to the accepted "getValue()" standard? Of course fooling yourself into thinking you have achieved things you actually haven't by clever usage of natural language is the opposite end of the spectrum, and it's exactly what I was ranting against. I guess "skepticism" is the correct word, indeed.
• umvi 6y ago
  I thought gradient descent was mostly calculus, not linear algebra. I was under the impression linear algebra was used to frame calculations so that GPUs could be utilized (since GPUs are very good at LA operations)
  • mr_toad 6y ago
    It is. Even in models that are actually linear (e.g. linear regression), calculus is used to characterise the function used to minimise the errors.