4 ms·

Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techn

by Kaizyn 18y ago

Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? If so, then sure this is a problem. Otherwise, then the 'deep math' is perhaps a bit superfluous to the problem of programming applications that make use of AI techniques?

plinkplonk 18y ago
"Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? " That is exactly what it means. "Deep math" is not learned because people have nothing else to do with their time. The moment you try to apply or extend the techniques in PCI beyond the toy examples in the book you will see the need for the "deep math". Just one example. BackPropogation Neural Networks are covered very superficially PCI. To get a real understanding of BPNNs, read Chris Bishop's Neural Networks for Patttern Recognition. That's a whole book on one type of neural network. I quote Peter Norvig's Amazon Review of Bishop's book. (emphasis mine) "To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is palstered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere, but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef." "If you want understanding of the techniques, you have to understand the math". Without understanding you can't (a) judge the appropriateness of a particular algorithm to a data set (b) know which variant of the algorithm to apply (c) understand the results, especially when they don't make sense (d) debug the implementation if required. In short at best you are making calls into a black box library and praying the results make sense. PCI skips the "understanding" part of AI aIgorithms (which does need math as Norvig points out. Most AI is applied math)and provides a superficial outline and many blackboxes. E.g: the "use libSVM for Support Vector Machines" idea PCI propagates (after a very superficial overview of SVMs) . LibSVM is a great library,but trying to use it to "SVM ize" your code without understanding exactly how exactly SVMs work (which does need "deep math" ;-) ) , will give you ... ummm.. suboptimal ... results. One of my friends tried to use algorithms from PCI to add "intelligence" to some code for his startup (he first ported the PCI code to Ruby) and ended up getting nonsensical results. Eventually I looked at the data, threw away his code and replaced it with a different algorithm which was appropriate. But, hey, I got some equity in return for the AI code so that turned out all right :-P All that said, I have no problems with folks using/liking PCI. I was just reacting to the "deep tech" phrase in the original post.
- Kaizyn 18y ago
  Thank you. That's the sort of answer to the question I was looking for.
brent 18y ago
I agree with plinkplonk, but if you'd like examples: 1) I mentioned multidimensional scaling elsewhere. In the book there is implementation and it doesn't even say what the goal is (in terms of objective function) and/or how well it performs. There are many different goals of multidimensional scaling. If one understand even the objective function they are minimizing then the algorithm to do so will make more sense. 2) The book grazes over SVM's and makes a few blanket statements that I haven't heard from serious researchers. E.g., "[t]he one that is usually recommended ... is called the radial-basis function. The radial-basis function is like the dot-product in that it takes two vectors and returns a value. Unlike the dot-product, it is not linear and can thus map more complex spaces." Most of the kernel/svm folks I know would never be so confident in saying such a blanket statement as it entirely relies on the data/situation.

by Kaizyn 18y ago

plinkplonk 18y ago

"Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? " That is exactly what it means. "Deep math" is not learned because people have nothing else to do with their time. The moment you try to apply or extend the techniques in PCI beyond the toy examples in the book you will see the need for the "deep math". Just one example. BackPropogation Neural Networks are covered very superficially PCI. To get a real understanding of BPNNs, read Chris Bishop's Neural Networks for Patttern Recognition. That's a whole book on one type of neural network. I quote Peter Norvig's Amazon Review of Bishop's book. (emphasis mine) "To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is palstered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere, but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef." "If you want understanding of the techniques, you have to understand the math". Without understanding you can't (a) judge the appropriateness of a particular algorithm to a data set (b) know which variant of the algorithm to apply (c) understand the results, especially when they don't make sense (d) debug the implementation if required. In short at best you are making calls into a black box library and praying the results make sense. PCI skips the "understanding" part of AI aIgorithms (which does need math as Norvig points out. Most AI is applied math)and provides a superficial outline and many blackboxes. E.g: the "use libSVM for Support Vector Machines" idea PCI propagates (after a very superficial overview of SVMs) . LibSVM is a great library,but trying to use it to "SVM ize" your code without understanding exactly how exactly SVMs work (which does need "deep math" ;-) ) , will give you ... ummm.. suboptimal ... results. One of my friends tried to use algorithms from PCI to add "intelligence" to some code for his startup (he first ported the PCI code to Ruby) and ended up getting nonsensical results. Eventually I looked at the data, threw away his code and replaced it with a different algorithm which was appropriate. But, hey, I got some equity in return for the AI code so that turned out all right :-P All that said, I have no problems with folks using/liking PCI. I was just reacting to the "deep tech" phrase in the original post.

Kaizyn 18y ago
Thank you. That's the sort of answer to the question I was looking for.

brent 18y ago

I agree with plinkplonk, but if you'd like examples: 1) I mentioned multidimensional scaling elsewhere. In the book there is implementation and it doesn't even say what the goal is (in terms of objective function) and/or how well it performs. There are many different goals of multidimensional scaling. If one understand even the objective function they are minimizing then the algorithm to do so will make more sense. 2) The book grazes over SVM's and makes a few blanket statements that I haven't heard from serious researchers. E.g., "[t]he one that is usually recommended ... is called the radial-basis function. The radial-basis function is like the dot-product in that it takes two vectors and returns a value. Unlike the dot-product, it is not linear and can thus map more complex spaces." Most of the kernel/svm folks I know would never be so confident in saying such a blanket statement as it entirely relies on the data/situation.