4 ms·

Can all algorithms then be cast as learning problems and their optimal versions produced this way? Seems like amazing work, but I don't know enough to confirm.

dangerlibrary 10y ago
Gradient descent has local maxima problems, so it's not always going to produce an "optimal" result.
- quotemstr 10y ago
  Humans also have local maxima problems
  - jacobush 10y ago
    Just this one last cigarette - in the pursuit of happinnes
- merraksh 10y ago
  Gradient ascent reaches a local maximum eventually, but gradient descent is guaranteed to find local minima only.
  - no_flags 10y ago
    They're the same thing, give or take a minus sign... right?
    - merraksh 10y ago
      Indeed max {f(x): x in X} = - min {-f(x): x in X} However, gradient ascent on a convex minimization problem will get stuck in a local maximum (as a convex minimization problem has f(x) convex, hence with local minima = global minima), and viceversa for gradient descent algorithms on concave maximization problems.
tlarkworthy 10y ago
There are a ton of auto tuning PID algorithms out there, which is somewhat analogous. Unfortunately they only work for certain regions. No free lunch (theorem)!
scythe 10y ago
The resolution of the Entscheidungsproblem would seem to answer this question in the negative: http://en.wikipedia.org/wiki/Entscheidungsproblem http://en.wikipedia.org/wiki/Entscheidungsproblem However, it does not place limits on the quality of approximation. This suggests that the development of AI will proceed gradually rather than suddenly (I've always disliked the term "singularity"; exponentials don't have those) since the infinite case is not going to be solved.
marcosdumay 10y ago
The name is "automatic programming", or at least it was by the times the books available at the library in my undergrad were written (what was much earlier than my undergrad). Genetic programming in special is very interesting, and worth reading about.

dangerlibrary 10y ago

Gradient descent has local maxima problems, so it's not always going to produce an "optimal" result.

quotemstr 10y ago
Humans also have local maxima problems
- jacobush 10y ago
  Just this one last cigarette - in the pursuit of happinnes
merraksh 10y ago
Gradient ascent reaches a local maximum eventually, but gradient descent is guaranteed to find local minima only.
- no_flags 10y ago
  They're the same thing, give or take a minus sign... right?
  - merraksh 10y ago
    Indeed max {f(x): x in X} = - min {-f(x): x in X} However, gradient ascent on a convex minimization problem will get stuck in a local maximum (as a convex minimization problem has f(x) convex, hence with local minima = global minima), and viceversa for gradient descent algorithms on concave maximization problems.

tlarkworthy 10y ago

There are a ton of auto tuning PID algorithms out there, which is somewhat analogous. Unfortunately they only work for certain regions. No free lunch (theorem)!

scythe 10y ago

The resolution of the Entscheidungsproblem would seem to answer this question in the negative: http://en.wikipedia.org/wiki/Entscheidungsproblem http://en.wikipedia.org/wiki/Entscheidungsproblem However, it does not place limits on the quality of approximation. This suggests that the development of AI will proceed gradually rather than suddenly (I've always disliked the term "singularity"; exponentials don't have those) since the infinite case is not going to be solved.

marcosdumay 10y ago

The name is "automatic programming", or at least it was by the times the books available at the library in my undergrad were written (what was much earlier than my undergrad). Genetic programming in special is very interesting, and worth reading about.