6 ms·

> More worrisome, perhaps, are the algorithms that are not written by humans, algorithms written by the machine, as it learns. Can anyone show even one example

> More worrisome, perhaps, are the algorithms that are not written by humans, algorithms written by the machine, as it learns. Can anyone show even one example of a computer writing an original and useful algorithm on it's own? Total misunderstanding of deep learning and "AI" is so common in today's PopSci.

mav3rick 8y ago
When weights are tuned you are technically left with a different behavior of an algorithm than when it started. When this tuning happened automatically, it's not really a misunderstanding.
- jcoffland 8y ago
  Weights are just input data so you are still left with the same algorithm.
currymj 8y ago
I actually think this is a pretty literal description of automatically learned decision trees, which are used in practice (directly or as sub parts of state-of-the-art techniques like gradient boosting). It is basically learning a series of complex nested if statements. Edit: also, there is actual work done in so-called “program synthesis”, especially employing Bayesian techniques, although it’s way more experimental and I haven’t heard of it being used “in the wild”.
- modriano 8y ago
  Decision tree models are trained by a machine learning algorithm that a human implemented. DTrees don't create new algorithms, they are calculated/trained through the application of an algorithm on data. You can find these DTree-generating algorithms in any decent machine learning algorithm textbook, and you can execute the steps by hand. For example, on page 56 (or pdf page 68) of [0], you can step through the ID3 decision tree algorithm. At no point is a new algorithm generated. [0]: https://www.cs.ubbcluj.ro/~gabis/ml/ml-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf https://www.cs.ubbcluj.ro/~gabis/ml/ml-books/McGrawHill%20-%...
  - currymj 8y ago
    to my understanding, a bunch of nested if-else statements is an algorithm, and if you change all the conditions in the if statements, you get a different algorithm. so e.g. ID3 is an algorithm, designed by a human, that produces new algorithms, only indirectly designed by a human. we may be talking past each other, though. i just think it's reasonable to apply the word "algorithm" to things that are simple/stupid enough to be generated automatically, and so it's no big deal to talk about algorithms writing algorithms.
    - jcoffland 8y ago
      > to my understanding, a bunch of nested if-else statements is an algorithm A bunch of nested if-else statements can be reduced to a boolean expression and a boolean expression is hardly an algorithm. If you change the conditions you simply get a different result not a new algorithm.
- jcoffland 8y ago
  https://en.wikipedia.org/wiki/Program_synthesis https://en.wikipedia.org/wiki/Program_synthesis It's quite clear from the Wikipedia page that program synthesis is not producing new algorithms. If I'm wrong, it should be easy to show. Simply point out one concrete example. Not research or articles discussing the idea but an example of actual original computer generated algorithm.
  - currymj 8y ago
    I really think we're talking past each other, and that you must have some more restrictive definition of "algorithm" than me, because I'm reading that page and it walks you through an example in excruciating detail of how given a specification for "maximum", program synthesis produces an algorithm for computing "maximum".

mav3rick 8y ago

When weights are tuned you are technically left with a different behavior of an algorithm than when it started. When this tuning happened automatically, it's not really a misunderstanding.

jcoffland 8y ago
Weights are just input data so you are still left with the same algorithm.

currymj 8y ago

I actually think this is a pretty literal description of automatically learned decision trees, which are used in practice (directly or as sub parts of state-of-the-art techniques like gradient boosting). It is basically learning a series of complex nested if statements. Edit: also, there is actual work done in so-called “program synthesis”, especially employing Bayesian techniques, although it’s way more experimental and I haven’t heard of it being used “in the wild”.

modriano 8y ago
Decision tree models are trained by a machine learning algorithm that a human implemented. DTrees don't create new algorithms, they are calculated/trained through the application of an algorithm on data. You can find these DTree-generating algorithms in any decent machine learning algorithm textbook, and you can execute the steps by hand. For example, on page 56 (or pdf page 68) of [0], you can step through the ID3 decision tree algorithm. At no point is a new algorithm generated. [0]: https://www.cs.ubbcluj.ro/~gabis/ml/ml-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf https://www.cs.ubbcluj.ro/~gabis/ml/ml-books/McGrawHill%20-%...
- currymj 8y ago
  to my understanding, a bunch of nested if-else statements is an algorithm, and if you change all the conditions in the if statements, you get a different algorithm. so e.g. ID3 is an algorithm, designed by a human, that produces new algorithms, only indirectly designed by a human. we may be talking past each other, though. i just think it's reasonable to apply the word "algorithm" to things that are simple/stupid enough to be generated automatically, and so it's no big deal to talk about algorithms writing algorithms.
  - jcoffland 8y ago
    > to my understanding, a bunch of nested if-else statements is an algorithm A bunch of nested if-else statements can be reduced to a boolean expression and a boolean expression is hardly an algorithm. If you change the conditions you simply get a different result not a new algorithm.
jcoffland 8y ago
https://en.wikipedia.org/wiki/Program_synthesis https://en.wikipedia.org/wiki/Program_synthesis It's quite clear from the Wikipedia page that program synthesis is not producing new algorithms. If I'm wrong, it should be easy to show. Simply point out one concrete example. Not research or articles discussing the idea but an example of actual original computer generated algorithm.
- currymj 8y ago
  I really think we're talking past each other, and that you must have some more restrictive definition of "algorithm" than me, because I'm reading that page and it walks you through an example in excruciating detail of how given a specification for "maximum", program synthesis produces an algorithm for computing "maximum".