8 ms·

Is deep learning a new kind of programming?

94 points by jpcooper 6y ago

teruakohatu 6y ago
Is linear regression programming? A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves, but they could be used to built analog computers (where cogs, cams etc. Are like lines of code or procedures). Building a neutral network to get a result is not at all like programming. There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network.
- qayxc 6y ago
  > There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network. The same can be said about complex simulations. The difference lies in knowing and being able to determine the limits of the system and the verification process. In a simulation we can derive the accuracy of our model from parameters like numerical precision and -stability, coarseness and model used. In other words we know the function we want to model because we state it explicitly. Neural nets can model any function and the challenge is to extract the learned function from the trained network, to examine its limits and correctness. This is what I understood the author meant by the "operational" viewpoint. We can verify that a given network architecture combined with a given optimisation function will find a local minimum w.r.t a given set of training data. This can be verified and tested. What's not so easy to verify and test, however, are the properties of the modelled function as well as the function itself. That's why we still have to rely on proxies like error metrics on fixed datasets or failure cases. With a simulation on the other hand, we can easily control and predict the (quality of the-) outcome by manipulating well understood parameters (number of iterations, coarseness of the simulation, numerical precision, etc.). I picked simulations as an example, because many other classes of program can be verified using formal methods since the desired results are usually known beforehand. Again, just another reason why the author talks about a distinction in terms of operations, not the fundamental type of programming. I find this to be a very interesting and thought provoking idea.
- mr_toad 6y ago
  > Is linear regression programming? I’d argue not. Mostly. If you fit a model using lm() in R, and then apply that model it’s not the same as hand selecting the weights of a linear equation and coding that equation. You could in theory select the same weights, and code it by hand. But no one ever does.
  - disgruntledphd2 6y ago
    Doesn't that imply that if one can fit an NN in a line of code, then it's not programming? That seems like a weird distinction to make.
deleted 6y ago
[deleted]
hprotagonist 6y ago
I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot." there's really no other magic in it than that.
- ipsum2 6y ago
  How well does "linear least squares to some nonlinear methods, simulated annealing, bayesian methods" work when you're doing speech to text, text generation or object detection? Deep learning is different in that it's a huge leap closer to human capabilities compared to the other methods you've listed.
  - f6v 6y ago
    Deep learning is closer to linear regression than it’s to human capabilities.
    - mr_toad 6y ago
      > Deep learning is closer to linear regression than it’s to human capabilities. Quantitatively perhaps, in that deep learning is equivalent to many neurons, whereas a least squares model is only equivalent to a single neuron. Other than that there is not much evidence that a human brain is qualitatively different.
      - f6v 6y ago
        Deep learning is much more than a "long" network nowadays. Most of DL success comes from clever architectures, like Inception Net. AFAIK there's no evidence human brain is anything like those fancy deep NNs.
  - fock 6y ago
    well, your neural network is just a very complicated - ideally continously differentiable - function. In principle you could just fit this function to data with nonlinear least squares (e.g. Levenberg-Marquardt), we use backpropagation and SGD, because it's faster. Concerning simulated annealing: have a look at the old Alphafold and check how they optimize their neural networks... So, could you explain again how the concepts and foundations of deep learning differs from plain old regression techniques?
chriszhang 6y ago
We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced the model with a single line of code with one boolean expression made with those 4 parameters connected with logical operators.
- 29athrowaway 6y ago
  Have you tried training the network using dropout?
  - chriszhang 6y ago
    yes, dropout and other regularization techniques were used
- teruakohatu 6y ago
  That kind of problem, with such a limited number of parameters, really shouldn't be thrown into a neutral network. A decision tree (or varient) might have been the ideal ML technique, and you may have been able quickly see what parameters mattered and reduce the four parameters to code if needed. Neural networks make sense with huge number of input parameters where feature selection is really tricky to reason about and decision boundaries are very non-linear such as image classification. Edited: slight clarification
  - tluyben2 6y ago
    When I studied AI in uni in the cold cold (cold) winter of AI and this kind of input was really significant, but most problems we consider ML now are vastly more complex and other problems can be addressed by things that are no longer considered AI at all (while they were back then). It is funny how my uni top research (on 1m$ computers) neural nets are considered to make no sense anymore. That went a lot faster than programming.
pmohun 6y ago
I would go a step further, and say that prompt design will become an important sector of programming. Modern language models (eg GPT-3 et al) offer the capability to take a natural language input, match it against the context of the sentence, then propose a query that is understandable to the layperson. This abstraction allows us to understand the problem better, rather than just analyzing the way the problem manifests itself in code. Having a programming language that mirrors our everyday communication is an important step forward in making the innovations from software broadly available. The next wave of programmers will need to understand how human language can be used to efficiently guide models to solve problems that can’t be solved by human-written code. This is a big challenge, and we’re just at the very beginning of it, but I think it will open up new and undiscovered ways to create value in the world. I have quite a few additional thoughts on the topic which I’ve captured here: https://sundayscaries.substack.com/p/whos-the-real-expert https://sundayscaries.substack.com/p/whos-the-real-expert
- jpcooper 6y ago
  I don't know why you were downvoted, but I am interested in maybe the simpler case of computer-assisted programming. It doesn't necessarily have to be natural language input. Anything that can help debug compiler errors or debug programmes. I went to a talk years ago on someone's PhD project involving a certain interactive debugger for Haskell, where the user could traverse the graph, making claims about nodes and eliminating possibilities. I wish I could remember its name. uu-parsinglib [1] is a parser combinator library that provides error correction. [1] https://hackage.haskell.org/package/uu-parsinglib-2.3.0 https://hackage.haskell.org/package/uu-parsinglib-2.3.0 Maybe these algorithms could combine with AI to create something better.
- qayxc 6y ago
  > Having a programming language that mirrors our everyday communication is an important step forward in making the innovations from software broadly available. I don't think that's the case at all. Mathematics developed a formalised non-natural language precisely because human language is completely unsuitable for expressing abstract concepts in a concise and unambiguous fashion. You will find that even in non-technical fields language will quickly converge to a well-defined, coarse and highly coded subset of regular human language when efficiency and correctness are key. You can observe this in the different branches of military, medicine, and trades. We use programming to formalise algorithms, processes, and models. Those are abstract concepts and the difficulty doesn't lie in expressing them verbally. This has been shown time and again by fruitless efforts to create localised dialects of more accessible programming languages like BASIC or Pascal. Turns out it doesn't matter whether keywords are written in your native language or if you could write natural language-like sentences: the difficult part remained formalising the abstract concept and ideas in a meaningful, logical and sound way. What I do think will help tremendously, however, is using system such as GPT-3 to create another level of abstraction. There are many descriptive tasks that don't need to be put into code manually. The structure and behaviour of UIs comes to mind.
Barrin92 6y ago
I wouldn't say that deep learning is programming. I think the key feature of programming is legibility. A program is something that is clear enough to read and understand, to be decomposed in its constituent parts, and that has understandable semantics. For example, writing an algorithm that has precise steps and procedures is programming. Putting my input into a box, shaking the box, and taking the result out is not programming, even if the box somehow solved the problem. Merely describing a problem and then having it solved is not enough to delineate programming, because that actually does apply to almost anything.
- jpcooper 6y ago
  Going by the operationalistic idea of the article, which I interpret as duck typing, I think the key operations that can be applied to programmes are building them, running them, and measuring the build and results of the programme with respect to a certain goal. This applies both to a traditional programme, and to deep learning. Programmes are after all written to satisfy human needs, even if it is an obfuscated C contest. For instance, imagine you have a black box that observes the horse races, Twitbook, the betting market and so on, and based on those observations executes bets for you with a bookmaker. The execution of the orders has a measurable effect on your net worth. You might write a traditional programme which takes all of this data, and based on some ETL, statistical models and probability calculations, executes orders. You might do some ETL, plug it all into a neural network, tune it and execute orders based on the results. Your traditional programme is very complex, and combinations of small bugs may have large effects on the results. Your unit and integration tests may themselves be wrong. Formal testing possibly reduces the expected value of the system and is an arse to carry out for any large system. The expected value of the system itself becomes harder to reason about as the system grows, based on the operation of reading and understanding the code. The internals of your neural network are also difficult to reason about in some ways. It is difficult to understand the workings of your neural network and specific parts' effects on the measured effects of its output. It will take time to tune it and build the most profitable model. Both implementations of the black box may be backtested, and some sort of trust can be established over the expected value of each implementation. Both implementations allow the operations of running, and measuring the results of running. Both implementations are difficult to reason about in various ways. We are perfectly happy to give money to people for them to do things without fully understanding their inner thoughts and the processes behind those thoughts. Which is the golden duck?
jokoon 6y ago
As I've understood it, normal programming is transforming an input with a program to get an output. Machine learning is giving the input and the output to get a program. Problem is, it's too difficult to summarize or understand the resulting program, while the program you get is tied to the output data which is never really accurate. I'm still curious how ML specialists are approaching the task of analyzing a resulting deep neural network, and squeeze some science from it (meaning putting words on things they understand and are able to explain). I've also read that google was using ML to test different learning models, to easily find the best model to use for a given problem. I'm not sure but it sounded like they were feeding the training model and the data into another learning model. I can't remember the details or the article or the reddit comment but it sounded quite interesting.
- selestify 6y ago
  You're probably talking about Google AutoML. FWIW there seems to also be Amazon SageMaker and Azure Machine Learning AutoML
sriku 6y ago
Having thought along these lines before, I realized that i didn't get any new useful insights by throwing neutral networks and conventional programming into the same bucket. Life went on as usual in both the worlds and I stopped thinking about it. Any insights worth learning about?
- sriku 6y ago
  About the only useful genralization has been differentiable programming - I.e. AD embedded in a programming language to help mix learning some functions from data. But this is not a consequence of clubbing the aforesaid two.
programmerslave 6y ago
Why can’t there be a discussion on machine learning without everyone on HN trying to prove how unnecessary it is. Queue the anecdotes on simpler regression based methods, over paid scientists, and how much superior some other simpler method is.
- fxtentacle 6y ago
  It's an expensive (hardware, time, complexity) technique that is rarely the best. Why wouldn't people discuss cheaper, faster, understandable alternatives?
  - gauku 6y ago
    As a computer vision engineer, I feel "rarely" quickly changes to "most probably" for a lot of the problems we work on.
  - deleted 6y ago
    [deleted]
  - programmerslave 6y ago
    Just because you aren’t employed in a job that can make use of deep learning doesn’t mean it isn’t profitable. It’s extremely profitable. I’m tired of reading about some dumb alternatives that aren’t even relevant to the topic. Deep learning works very well for a certain class of problems. Continuing on with the trope that it doesn’t work just shows you aren’t educated
    - mrmonkeyman 6y ago
      Profitable != useful (and interesting and the right solution etc). Of course it is profitable. One poster showed us a 20 parameter model. He was getting paid to apply deep learning on it. Damnit.
    - adrian_b 6y ago
      While I agree with you that deep learning works very well for a certain class of problems and that some criticism directed towards it is misguided, there are also some arguments for the opposite side. I normally do not feel the need to comment about deep learning, because it is only tangentially related to some of my past projects, but I can also understand those who might want to comment negatively, because I have seen many cases of managers who did not understand at all how exactly certain problems can be solved, but nevertheless they pushed vigorously for the use of deep learning to replace other better suited solutions, because they believed it to be a modern and universally better method. So there were times when I was tired of seeing one more attempt to misuse deep learning and to have to explain and demonstrate once more which solution is better. Obviously, any such opinions, about which method is better in a certain case, should be proved with numerical results from tests or simulations, not with guesses, but some times that requires a lot of work to implement both methods, even if you are pretty sure about which will be the result.
KingOfCoders 6y ago
Deep learning is very different to, but feels to me like the way you work with Prolog.
raitucarp 6y ago
Somehow remind me of Software 2.0[1] [1] https://medium.com/@karpathy/software-2-0-a64152b37c35 https://medium.com/@karpathy/software-2-0-a64152b37c35
jacquesm 6y ago
Over time the words 'teaching' and 'programming' will converge and at some point you likely won't be able to tell the difference between the two anymore (in a computer context). Deep learning isn't programming per-se, but it definitely creates results that are of the same kind that programming would be able to create as well (in principle, at least, in many cases).
- SiebenHeaven 6y ago
  If you squint hard enough, deep learning can be seen as programming. A computer (human) is used to train a model (write and compile a program). The generated model (compiled program) is then used for inference (ran) on target machine with new input data.
0thgen 6y ago
for anyone interested, chris olah has a good blog post providing further insight on this topic: https://colah.github.io/posts/2015-09-NN-Types-FP/ https://colah.github.io/posts/2015-09-NN-Types-FP/

94 points by jpcooper 6y ago

Is linear regression programming? A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves, but they could be used to built analog computers (where cogs, cams etc. Are like lines of code or procedures). Building a neutral network to get a result is not at all like programming. There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network.

qayxc 6y ago
> There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network. The same can be said about complex simulations. The difference lies in knowing and being able to determine the limits of the system and the verification process. In a simulation we can derive the accuracy of our model from parameters like numerical precision and -stability, coarseness and model used. In other words we know the function we want to model because we state it explicitly. Neural nets can model any function and the challenge is to extract the learned function from the trained network, to examine its limits and correctness. This is what I understood the author meant by the "operational" viewpoint. We can verify that a given network architecture combined with a given optimisation function will find a local minimum w.r.t a given set of training data. This can be verified and tested. What's not so easy to verify and test, however, are the properties of the modelled function as well as the function itself. That's why we still have to rely on proxies like error metrics on fixed datasets or failure cases. With a simulation on the other hand, we can easily control and predict the (quality of the-) outcome by manipulating well understood parameters (number of iterations, coarseness of the simulation, numerical precision, etc.). I picked simulations as an example, because many other classes of program can be verified using formal methods since the desired results are usually known beforehand. Again, just another reason why the author talks about a distinction in terms of operations, not the fundamental type of programming. I find this to be a very interesting and thought provoking idea.
mr_toad 6y ago
> Is linear regression programming? I’d argue not. Mostly. If you fit a model using lm() in R, and then apply that model it’s not the same as hand selecting the weights of a linear equation and coding that equation. You could in theory select the same weights, and code it by hand. But no one ever does.
- disgruntledphd2 6y ago
  Doesn't that imply that if one can fit an NN in a line of code, then it's not programming? That seems like a weird distinction to make.

deleted 6y ago

[deleted]

hprotagonist 6y ago

I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot." there's really no other magic in it than that.

ipsum2 6y ago
How well does "linear least squares to some nonlinear methods, simulated annealing, bayesian methods" work when you're doing speech to text, text generation or object detection? Deep learning is different in that it's a huge leap closer to human capabilities compared to the other methods you've listed.
- f6v 6y ago
  Deep learning is closer to linear regression than it’s to human capabilities.
  - mr_toad 6y ago
    > Deep learning is closer to linear regression than it’s to human capabilities. Quantitatively perhaps, in that deep learning is equivalent to many neurons, whereas a least squares model is only equivalent to a single neuron. Other than that there is not much evidence that a human brain is qualitatively different.
    - f6v 6y ago
      Deep learning is much more than a "long" network nowadays. Most of DL success comes from clever architectures, like Inception Net. AFAIK there's no evidence human brain is anything like those fancy deep NNs.
- fock 6y ago
  well, your neural network is just a very complicated - ideally continously differentiable - function. In principle you could just fit this function to data with nonlinear least squares (e.g. Levenberg-Marquardt), we use backpropagation and SGD, because it's faster. Concerning simulated annealing: have a look at the old Alphafold and check how they optimize their neural networks... So, could you explain again how the concepts and foundations of deep learning differs from plain old regression techniques?

chriszhang 6y ago

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced the model with a single line of code with one boolean expression made with those 4 parameters connected with logical operators.

29athrowaway 6y ago
Have you tried training the network using dropout?
- chriszhang 6y ago
  yes, dropout and other regularization techniques were used
teruakohatu 6y ago
That kind of problem, with such a limited number of parameters, really shouldn't be thrown into a neutral network. A decision tree (or varient) might have been the ideal ML technique, and you may have been able quickly see what parameters mattered and reduce the four parameters to code if needed. Neural networks make sense with huge number of input parameters where feature selection is really tricky to reason about and decision boundaries are very non-linear such as image classification. Edited: slight clarification
- tluyben2 6y ago
  When I studied AI in uni in the cold cold (cold) winter of AI and this kind of input was really significant, but most problems we consider ML now are vastly more complex and other problems can be addressed by things that are no longer considered AI at all (while they were back then). It is funny how my uni top research (on 1m$ computers) neural nets are considered to make no sense anymore. That went a lot faster than programming.

pmohun 6y ago

I would go a step further, and say that prompt design will become an important sector of programming. Modern language models (eg GPT-3 et al) offer the capability to take a natural language input, match it against the context of the sentence, then propose a query that is understandable to the layperson. This abstraction allows us to understand the problem better, rather than just analyzing the way the problem manifests itself in code. Having a programming language that mirrors our everyday communication is an important step forward in making the innovations from software broadly available. The next wave of programmers will need to understand how human language can be used to efficiently guide models to solve problems that can’t be solved by human-written code. This is a big challenge, and we’re just at the very beginning of it, but I think it will open up new and undiscovered ways to create value in the world. I have quite a few additional thoughts on the topic which I’ve captured here: https://sundayscaries.substack.com/p/whos-the-real-expert https://sundayscaries.substack.com/p/whos-the-real-expert

jpcooper 6y ago
I don't know why you were downvoted, but I am interested in maybe the simpler case of computer-assisted programming. It doesn't necessarily have to be natural language input. Anything that can help debug compiler errors or debug programmes. I went to a talk years ago on someone's PhD project involving a certain interactive debugger for Haskell, where the user could traverse the graph, making claims about nodes and eliminating possibilities. I wish I could remember its name. uu-parsinglib [1] is a parser combinator library that provides error correction. [1] https://hackage.haskell.org/package/uu-parsinglib-2.3.0 https://hackage.haskell.org/package/uu-parsinglib-2.3.0 Maybe these algorithms could combine with AI to create something better.
qayxc 6y ago
> Having a programming language that mirrors our everyday communication is an important step forward in making the innovations from software broadly available. I don't think that's the case at all. Mathematics developed a formalised non-natural language precisely because human language is completely unsuitable for expressing abstract concepts in a concise and unambiguous fashion. You will find that even in non-technical fields language will quickly converge to a well-defined, coarse and highly coded subset of regular human language when efficiency and correctness are key. You can observe this in the different branches of military, medicine, and trades. We use programming to formalise algorithms, processes, and models. Those are abstract concepts and the difficulty doesn't lie in expressing them verbally. This has been shown time and again by fruitless efforts to create localised dialects of more accessible programming languages like BASIC or Pascal. Turns out it doesn't matter whether keywords are written in your native language or if you could write natural language-like sentences: the difficult part remained formalising the abstract concept and ideas in a meaningful, logical and sound way. What I do think will help tremendously, however, is using system such as GPT-3 to create another level of abstraction. There are many descriptive tasks that don't need to be put into code manually. The structure and behaviour of UIs comes to mind.

Barrin92 6y ago

I wouldn't say that deep learning is programming. I think the key feature of programming is legibility. A program is something that is clear enough to read and understand, to be decomposed in its constituent parts, and that has understandable semantics. For example, writing an algorithm that has precise steps and procedures is programming. Putting my input into a box, shaking the box, and taking the result out is not programming, even if the box somehow solved the problem. Merely describing a problem and then having it solved is not enough to delineate programming, because that actually does apply to almost anything.

jpcooper 6y ago
Going by the operationalistic idea of the article, which I interpret as duck typing, I think the key operations that can be applied to programmes are building them, running them, and measuring the build and results of the programme with respect to a certain goal. This applies both to a traditional programme, and to deep learning. Programmes are after all written to satisfy human needs, even if it is an obfuscated C contest. For instance, imagine you have a black box that observes the horse races, Twitbook, the betting market and so on, and based on those observations executes bets for you with a bookmaker. The execution of the orders has a measurable effect on your net worth. You might write a traditional programme which takes all of this data, and based on some ETL, statistical models and probability calculations, executes orders. You might do some ETL, plug it all into a neural network, tune it and execute orders based on the results. Your traditional programme is very complex, and combinations of small bugs may have large effects on the results. Your unit and integration tests may themselves be wrong. Formal testing possibly reduces the expected value of the system and is an arse to carry out for any large system. The expected value of the system itself becomes harder to reason about as the system grows, based on the operation of reading and understanding the code. The internals of your neural network are also difficult to reason about in some ways. It is difficult to understand the workings of your neural network and specific parts' effects on the measured effects of its output. It will take time to tune it and build the most profitable model. Both implementations of the black box may be backtested, and some sort of trust can be established over the expected value of each implementation. Both implementations allow the operations of running, and measuring the results of running. Both implementations are difficult to reason about in various ways. We are perfectly happy to give money to people for them to do things without fully understanding their inner thoughts and the processes behind those thoughts. Which is the golden duck?

jokoon 6y ago

As I've understood it, normal programming is transforming an input with a program to get an output. Machine learning is giving the input and the output to get a program. Problem is, it's too difficult to summarize or understand the resulting program, while the program you get is tied to the output data which is never really accurate. I'm still curious how ML specialists are approaching the task of analyzing a resulting deep neural network, and squeeze some science from it (meaning putting words on things they understand and are able to explain). I've also read that google was using ML to test different learning models, to easily find the best model to use for a given problem. I'm not sure but it sounded like they were feeding the training model and the data into another learning model. I can't remember the details or the article or the reddit comment but it sounded quite interesting.

selestify 6y ago
You're probably talking about Google AutoML. FWIW there seems to also be Amazon SageMaker and Azure Machine Learning AutoML

sriku 6y ago

Having thought along these lines before, I realized that i didn't get any new useful insights by throwing neutral networks and conventional programming into the same bucket. Life went on as usual in both the worlds and I stopped thinking about it. Any insights worth learning about?

sriku 6y ago
About the only useful genralization has been differentiable programming - I.e. AD embedded in a programming language to help mix learning some functions from data. But this is not a consequence of clubbing the aforesaid two.

programmerslave 6y ago

Why can’t there be a discussion on machine learning without everyone on HN trying to prove how unnecessary it is. Queue the anecdotes on simpler regression based methods, over paid scientists, and how much superior some other simpler method is.

fxtentacle 6y ago
It's an expensive (hardware, time, complexity) technique that is rarely the best. Why wouldn't people discuss cheaper, faster, understandable alternatives?
- gauku 6y ago
  As a computer vision engineer, I feel "rarely" quickly changes to "most probably" for a lot of the problems we work on.
- deleted 6y ago
  [deleted]
- programmerslave 6y ago
  Just because you aren’t employed in a job that can make use of deep learning doesn’t mean it isn’t profitable. It’s extremely profitable. I’m tired of reading about some dumb alternatives that aren’t even relevant to the topic. Deep learning works very well for a certain class of problems. Continuing on with the trope that it doesn’t work just shows you aren’t educated
  - mrmonkeyman 6y ago
    Profitable != useful (and interesting and the right solution etc). Of course it is profitable. One poster showed us a 20 parameter model. He was getting paid to apply deep learning on it. Damnit.
  - adrian_b 6y ago
    While I agree with you that deep learning works very well for a certain class of problems and that some criticism directed towards it is misguided, there are also some arguments for the opposite side. I normally do not feel the need to comment about deep learning, because it is only tangentially related to some of my past projects, but I can also understand those who might want to comment negatively, because I have seen many cases of managers who did not understand at all how exactly certain problems can be solved, but nevertheless they pushed vigorously for the use of deep learning to replace other better suited solutions, because they believed it to be a modern and universally better method. So there were times when I was tired of seeing one more attempt to misuse deep learning and to have to explain and demonstrate once more which solution is better. Obviously, any such opinions, about which method is better in a certain case, should be proved with numerical results from tests or simulations, not with guesses, but some times that requires a lot of work to implement both methods, even if you are pretty sure about which will be the result.

KingOfCoders 6y ago

Deep learning is very different to, but feels to me like the way you work with Prolog.

raitucarp 6y ago

Somehow remind me of Software 2.0[1] [1] https://medium.com/@karpathy/software-2-0-a64152b37c35 https://medium.com/@karpathy/software-2-0-a64152b37c35

jacquesm 6y ago

Over time the words 'teaching' and 'programming' will converge and at some point you likely won't be able to tell the difference between the two anymore (in a computer context). Deep learning isn't programming per-se, but it definitely creates results that are of the same kind that programming would be able to create as well (in principle, at least, in many cases).

SiebenHeaven 6y ago
If you squint hard enough, deep learning can be seen as programming. A computer (human) is used to train a model (write and compile a program). The generated model (compiled program) is then used for inference (ran) on target machine with new input data.

0thgen 6y ago

for anyone interested, chris olah has a good blog post providing further insight on this topic: https://colah.github.io/posts/2015-09-NN-Types-FP/ https://colah.github.io/posts/2015-09-NN-Types-FP/