4 ms·

People say this a lot - that we don't understand this or that, but I'm not really sure what they mean. We know exactly how these algorithms work. We know every

People say this a lot - that we don't understand this or that, but I'm not really sure what they mean. We know exactly how these algorithms work. We know every calculation - the maths is not particularly difficult, we understand how the training process leads to information being stored in the weights, we know how inference works. What more would you want to understand before you would agree we understood it?

blagie 3y ago
> We know exactly how these algorithms work. We have no idea how these algorithms work. > We know every calculation - the maths is not particularly difficult, We do know that. > we understand how the training process leads to information being stored in the weights, we know how inference works. We do not know that. > What more would you want to understand before you would agree we understood it? Let me give an analogy: We have an almost perfect understanding of transistors. If you hand me a Qualcomm mobile chipset in a black box, I'll have little or no understanding of how that allows me to make phone calls. Back in the day, I understood the x86 instruction set very well. However, if you gave me the binary of a video game, I'd have no idea how it worked. Neuroscientists understand the mathematics of how neurons work, imperfectly (but pretty well). For the sake of argument, we can pretend the models are perfect. We understand the neural wiring of simple organisms perfectly. We still have very little idea of how the human brain works. The algorithms in deep learning are evolved and have billions of parameters. We understand the general topology, and the math of individual neurons, but we have absolutely no idea how the things work as a system. Anyone who tells you they do is lying (very likely with no ill intent; they're probably deluding themselves as well). The people doing deep learning are, by and large, not brilliant mathematicians, of the type who did earlier AI. The math is simple compared to most of the convex optimization algorithms which came before (and could probably be made much better if those were applied). Even at the human level, a lot of work in deep learning is - randomly tweaking parameters, topologies, and algorithms - developing intuition (NOT theory) for which ones work better, and - bullshitting explanations for why that might be (which would, at best, pass for hypothesis in any scientific process) It's hard for me to emphasize how little we know about how or why these things work. We just set up a general framing which evolves well, and evolved it. An analogy would be if we set up a random number generator to write a piece of code, ran it 10^10^10 times, and picked the result which made the best wavelet transforms. We'd have no clue how it works. The only difference is (1) we have algorithms which are more tractable than randomly picking algorithms (2) we set up neural networks which evolve better than code, largely by virtue of being continuous rather than discrete.
- kybernetikos 3y ago
  I'm still struggling to understand what counts as 'knowing how it works' for you. In my view, if you randomly generated a piece of code to do a task, you know how it works - you can see the algorithm right there in front of you. If you randomly generated it and checked each instance until you found a good one, you even know how you got it and why it's good at the task (because you checked it and threw away the ones that weren't). Obviously if you start not knowing what the code is, you have a simple, surface level lack of understanding which you can resolve by tracing it. Once you've done that, you understand everything there is to understand about it. The fact that it produces nice wavelet transforms is a simple product of how it was found. What more do you want to understand about it? What more is there to understand about it?
  - famouswaffles 3y ago
    >you know how it works - you can see the algorithm right there in front of you. Seeing the algorithm in front of you doesn't mean you know how it works. It's gibberish code. If you'd never learnt C or any other programming language in your life, i could show you the C Code for a popular application. You could inspect it all you like. You will still understand nothing. The best you can do is, "this code is running this application". In the real world, you can just pick up a C book and start learning. In this instance, no one on earth has learnt C and there are no books on it. and neural network calculations are not just one unvarying "algorithm"
    - kybernetikos 3y ago
      Knowing the algorithm literally means that you could theoretically reproduce it yourself, step by step (absent practical worries about longevity). Each of the steps is simple and well understood. What to do at each step is simple and well understood. What more is there to understand?
      - famouswaffles 3y ago
        By that definition, i know COBOL too since i can look up programs coded in it. I can reproduce the program too by hand-typing it and running it elsewhere. Banks should hire me asap /s. I genuinely don't get what is so hard to understand here. You don't know the algorithm. You can see it. That's all. You don't suddenly understand information just because you can see and copy it. Would certainly be nice though.

blagie 3y ago

> We know exactly how these algorithms work. We have no idea how these algorithms work. > We know every calculation - the maths is not particularly difficult, We do know that. > we understand how the training process leads to information being stored in the weights, we know how inference works. We do not know that. > What more would you want to understand before you would agree we understood it? Let me give an analogy: We have an almost perfect understanding of transistors. If you hand me a Qualcomm mobile chipset in a black box, I'll have little or no understanding of how that allows me to make phone calls. Back in the day, I understood the x86 instruction set very well. However, if you gave me the binary of a video game, I'd have no idea how it worked. Neuroscientists understand the mathematics of how neurons work, imperfectly (but pretty well). For the sake of argument, we can pretend the models are perfect. We understand the neural wiring of simple organisms perfectly. We still have very little idea of how the human brain works. The algorithms in deep learning are evolved and have billions of parameters. We understand the general topology, and the math of individual neurons, but we have absolutely no idea how the things work as a system. Anyone who tells you they do is lying (very likely with no ill intent; they're probably deluding themselves as well). The people doing deep learning are, by and large, not brilliant mathematicians, of the type who did earlier AI. The math is simple compared to most of the convex optimization algorithms which came before (and could probably be made much better if those were applied). Even at the human level, a lot of work in deep learning is - randomly tweaking parameters, topologies, and algorithms - developing intuition (NOT theory) for which ones work better, and - bullshitting explanations for why that might be (which would, at best, pass for hypothesis in any scientific process) It's hard for me to emphasize how little we know about how or why these things work. We just set up a general framing which evolves well, and evolved it. An analogy would be if we set up a random number generator to write a piece of code, ran it 10^10^10 times, and picked the result which made the best wavelet transforms. We'd have no clue how it works. The only difference is (1) we have algorithms which are more tractable than randomly picking algorithms (2) we set up neural networks which evolve better than code, largely by virtue of being continuous rather than discrete.

kybernetikos 3y ago
I'm still struggling to understand what counts as 'knowing how it works' for you. In my view, if you randomly generated a piece of code to do a task, you know how it works - you can see the algorithm right there in front of you. If you randomly generated it and checked each instance until you found a good one, you even know how you got it and why it's good at the task (because you checked it and threw away the ones that weren't). Obviously if you start not knowing what the code is, you have a simple, surface level lack of understanding which you can resolve by tracing it. Once you've done that, you understand everything there is to understand about it. The fact that it produces nice wavelet transforms is a simple product of how it was found. What more do you want to understand about it? What more is there to understand about it?
- famouswaffles 3y ago
  >you know how it works - you can see the algorithm right there in front of you. Seeing the algorithm in front of you doesn't mean you know how it works. It's gibberish code. If you'd never learnt C or any other programming language in your life, i could show you the C Code for a popular application. You could inspect it all you like. You will still understand nothing. The best you can do is, "this code is running this application". In the real world, you can just pick up a C book and start learning. In this instance, no one on earth has learnt C and there are no books on it. and neural network calculations are not just one unvarying "algorithm"
  - kybernetikos 3y ago
    Knowing the algorithm literally means that you could theoretically reproduce it yourself, step by step (absent practical worries about longevity). Each of the steps is simple and well understood. What to do at each step is simple and well understood. What more is there to understand?
    - famouswaffles 3y ago
      By that definition, i know COBOL too since i can look up programs coded in it. I can reproduce the program too by hand-typing it and running it elsewhere. Banks should hire me asap /s. I genuinely don't get what is so hard to understand here. You don't know the algorithm. You can see it. That's all. You don't suddenly understand information just because you can see and copy it. Would certainly be nice though.