8 ms·

"At the same time, it shows how much most web applications being developed today lack deep technology" Just my 2 cents but I find PCI a very shallow book witho

"At the same time, it shows how much most web applications being developed today lack deep technology" Just my 2 cents but I find PCI a very shallow book without much "deep technology" in it either. AI cannot be divorced from the underlying math. PCI takes (in my opinion, feel free to differ) a math-lite, "dummies guide" approach to AI algorithms. I realize that my opinion is in the minority, and a lot of people think that the book is very cool. So take it with a grain of salt.

bockris 18y ago
While I agree with your assessment, IMO you don't seem to be the target audience. I loved PCI because it was math-lite. I'm a business applications programmer by trade and fun stuff like this isn't anywhere on our radar. I also have a family and the associated obligations like T-Ball and dance recitals so I don't have the time to dig deeply into the algorithms and math. PCI was a good balance to me between showing the theory and also providing concrete working code. My only quibble with the book was that a lot of the code was not idiomatic Python. (un-Pythonic, if you will.) Just my $0.02.
brent 18y ago
It is certainly a "dummie's guide" approach and probably the best book applying this framework to machine learning related problems. It introduces a lot of ideas to people where the math would simply not be approachable. That said I have three complaints: 1 I think it tries to stay too high level on some areas where it is truly unavoidable and wouldn't scare anyone off to add a little depth. 2 I think it too often labels their implementation with a name that is associated with a family of techniques. This is simply going to embarrass/mislead the reader. In the interest of maintaining simplicity the book states several things that are flat out wrong. 3 No references. If someone wants to take the next step there is zero guidance. This is important because someone learning about a deep topic could read the book and a) think they understand it (not likely) and b) not have a clue where to look next.
- j2d2 18y ago
  Can you elaborate on the things it simply gets wrong?
  - brent 18y ago
    One example, simply because I opened the book to this page after reading your comment and happen to know a bit about the subject (p49): "[t]he algorithm takes the difference between every pair of items and tries to make a chart in which the distances between the items match those differences" This is referring to multidimensional scaling which is a large family of techniques, some of which attempt to achieve this goal (delta ~= d), but not necessarily. Granted, in this case the algorithm presented DOES attempt to do this. BUT, the algorithm presented is not "multidimensional scaling". It is an implementation (not even a good one) attempting to minimize one of many possible objective functions. I noticed several more when I flipped through it initially, but this is the type of thing I'm referring to.
- me2i81 18y ago
  I thought the lack of references was a major failing of the book. Some of the algorithms barely scratch the surface, and it does a disservice to the reader to provide noplace to go. Here's a few to get started: 1. Russel and Norvig's AI text, 2. Elements of Statistical Learning by Hastie et. al., 3. Pattern Recognition and Machine Learning by Chris Bishop. On the other hand going right into code examples is useful, including jumping right into getting real data downloaded and worked on.
  - brent 18y ago
    I agree about #2. I have Russell and Norvig and agree that it is an excellent book, but I am not sure how much overlap there truly is here. I also do not have PRML, but Neural Networks for Pattern Recognition by Bishop is excellent (and includes many non-NN related items). ESL is excellent and, to me, the best modern text in machine learning. It covers many of the topics in PCI both at a reasonable level and in much more depth and provides MANY references (100s?) to dig deeper.
mcroydon 18y ago
The biggest thing I got from PCI was a basic discussion of a lot of algorithms that I was able to research further on my own. I had no idea that I wanted to know more about K-means clustering until I read the book. If that's all you get out of it great, but there's a lot of interesting stuff in there, covered in enough detail to get you started, which is more than enough to make me happy.
Kaizyn 18y ago
Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? If so, then sure this is a problem. Otherwise, then the 'deep math' is perhaps a bit superfluous to the problem of programming applications that make use of AI techniques?
- plinkplonk 18y ago
  "Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? " That is exactly what it means. "Deep math" is not learned because people have nothing else to do with their time. The moment you try to apply or extend the techniques in PCI beyond the toy examples in the book you will see the need for the "deep math". Just one example. BackPropogation Neural Networks are covered very superficially PCI. To get a real understanding of BPNNs, read Chris Bishop's Neural Networks for Patttern Recognition. That's a whole book on one type of neural network. I quote Peter Norvig's Amazon Review of Bishop's book. (emphasis mine) "To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is palstered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere, but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef." "If you want understanding of the techniques, you have to understand the math". Without understanding you can't (a) judge the appropriateness of a particular algorithm to a data set (b) know which variant of the algorithm to apply (c) understand the results, especially when they don't make sense (d) debug the implementation if required. In short at best you are making calls into a black box library and praying the results make sense. PCI skips the "understanding" part of AI aIgorithms (which does need math as Norvig points out. Most AI is applied math)and provides a superficial outline and many blackboxes. E.g: the "use libSVM for Support Vector Machines" idea PCI propagates (after a very superficial overview of SVMs) . LibSVM is a great library,but trying to use it to "SVM ize" your code without understanding exactly how exactly SVMs work (which does need "deep math" ;-) ) , will give you ... ummm.. suboptimal ... results. One of my friends tried to use algorithms from PCI to add "intelligence" to some code for his startup (he first ported the PCI code to Ruby) and ended up getting nonsensical results. Eventually I looked at the data, threw away his code and replaced it with a different algorithm which was appropriate. But, hey, I got some equity in return for the AI code so that turned out all right :-P All that said, I have no problems with folks using/liking PCI. I was just reacting to the "deep tech" phrase in the original post.
jmatt 18y ago
what would you recommend then?
- brent 18y ago
  I am not the 'parent', but as I said elsewhere I would recommend The Elements of Statistical Learning. It gets quite a bit deeper than PCI, but I'm fairly confident you could learn almost everything in PCI from ESL. It doesn't come with pre-canned python, but honestly almost everything (in terms of code) in PCI is available somewhere on the web and/or already built into python libraries, matlab, and/or R.
toby 18y ago
I really appreciate the feedback (I'm the author), I definitely expected this sort of response as I was writing it. I think I got a lot right when I set the level but having seen it in the wild there are things I would have done differently too. Mostly I wanted people to get excited and see that they could do this right now and that there were good reasons to. I never intended to compete with Norvig, more to create something that would inspire them to want to read it.

bockris 18y ago

While I agree with your assessment, IMO you don't seem to be the target audience. I loved PCI because it was math-lite. I'm a business applications programmer by trade and fun stuff like this isn't anywhere on our radar. I also have a family and the associated obligations like T-Ball and dance recitals so I don't have the time to dig deeply into the algorithms and math. PCI was a good balance to me between showing the theory and also providing concrete working code. My only quibble with the book was that a lot of the code was not idiomatic Python. (un-Pythonic, if you will.) Just my $0.02.

brent 18y ago

It is certainly a "dummie's guide" approach and probably the best book applying this framework to machine learning related problems. It introduces a lot of ideas to people where the math would simply not be approachable. That said I have three complaints: 1 I think it tries to stay too high level on some areas where it is truly unavoidable and wouldn't scare anyone off to add a little depth. 2 I think it too often labels their implementation with a name that is associated with a family of techniques. This is simply going to embarrass/mislead the reader. In the interest of maintaining simplicity the book states several things that are flat out wrong. 3 No references. If someone wants to take the next step there is zero guidance. This is important because someone learning about a deep topic could read the book and a) think they understand it (not likely) and b) not have a clue where to look next.

j2d2 18y ago
Can you elaborate on the things it simply gets wrong?
- brent 18y ago
  One example, simply because I opened the book to this page after reading your comment and happen to know a bit about the subject (p49): "[t]he algorithm takes the difference between every pair of items and tries to make a chart in which the distances between the items match those differences" This is referring to multidimensional scaling which is a large family of techniques, some of which attempt to achieve this goal (delta ~= d), but not necessarily. Granted, in this case the algorithm presented DOES attempt to do this. BUT, the algorithm presented is not "multidimensional scaling". It is an implementation (not even a good one) attempting to minimize one of many possible objective functions. I noticed several more when I flipped through it initially, but this is the type of thing I'm referring to.
me2i81 18y ago
I thought the lack of references was a major failing of the book. Some of the algorithms barely scratch the surface, and it does a disservice to the reader to provide noplace to go. Here's a few to get started: 1. Russel and Norvig's AI text, 2. Elements of Statistical Learning by Hastie et. al., 3. Pattern Recognition and Machine Learning by Chris Bishop. On the other hand going right into code examples is useful, including jumping right into getting real data downloaded and worked on.
- brent 18y ago
  I agree about #2. I have Russell and Norvig and agree that it is an excellent book, but I am not sure how much overlap there truly is here. I also do not have PRML, but Neural Networks for Pattern Recognition by Bishop is excellent (and includes many non-NN related items). ESL is excellent and, to me, the best modern text in machine learning. It covers many of the topics in PCI both at a reasonable level and in much more depth and provides MANY references (100s?) to dig deeper.

mcroydon 18y ago

The biggest thing I got from PCI was a basic discussion of a lot of algorithms that I was able to research further on my own. I had no idea that I wanted to know more about K-means clustering until I read the book. If that's all you get out of it great, but there's a lot of interesting stuff in there, covered in enough detail to get you started, which is more than enough to make me happy.

Kaizyn 18y ago

Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? If so, then sure this is a problem. Otherwise, then the 'deep math' is perhaps a bit superfluous to the problem of programming applications that make use of AI techniques?

plinkplonk 18y ago
"Does this mean that implementations of the algorithms in PCI won't work as well as something one would write if they first learned all the deep math those techniques are built on? " That is exactly what it means. "Deep math" is not learned because people have nothing else to do with their time. The moment you try to apply or extend the techniques in PCI beyond the toy examples in the book you will see the need for the "deep math". Just one example. BackPropogation Neural Networks are covered very superficially PCI. To get a real understanding of BPNNs, read Chris Bishop's Neural Networks for Patttern Recognition. That's a whole book on one type of neural network. I quote Peter Norvig's Amazon Review of Bishop's book. (emphasis mine) "To the reviewer who said "I was looking forward to a detailed insight into neural networks in this book. Instead, almost every page is plastered up with sigma notation", that's like saying about a book on music theory "Instead, almost every page is palstered with black-and-white ovals (some with sticks on the edge)." Or to the reviewer who complains this book is limited to the mathematical side of neural nets, that's like complaining about a cookbook on beef being limited to the carnivore side. If you want a non-technical overview, you can get that elsewhere, but if you want understanding of the techniques, you have to understand the math. Otherwise, there's no beef." "If you want understanding of the techniques, you have to understand the math". Without understanding you can't (a) judge the appropriateness of a particular algorithm to a data set (b) know which variant of the algorithm to apply (c) understand the results, especially when they don't make sense (d) debug the implementation if required. In short at best you are making calls into a black box library and praying the results make sense. PCI skips the "understanding" part of AI aIgorithms (which does need math as Norvig points out. Most AI is applied math)and provides a superficial outline and many blackboxes. E.g: the "use libSVM for Support Vector Machines" idea PCI propagates (after a very superficial overview of SVMs) . LibSVM is a great library,but trying to use it to "SVM ize" your code without understanding exactly how exactly SVMs work (which does need "deep math" ;-) ) , will give you ... ummm.. suboptimal ... results. One of my friends tried to use algorithms from PCI to add "intelligence" to some code for his startup (he first ported the PCI code to Ruby) and ended up getting nonsensical results. Eventually I looked at the data, threw away his code and replaced it with a different algorithm which was appropriate. But, hey, I got some equity in return for the AI code so that turned out all right :-P All that said, I have no problems with folks using/liking PCI. I was just reacting to the "deep tech" phrase in the original post.

jmatt 18y ago

what would you recommend then?

brent 18y ago
I am not the 'parent', but as I said elsewhere I would recommend The Elements of Statistical Learning. It gets quite a bit deeper than PCI, but I'm fairly confident you could learn almost everything in PCI from ESL. It doesn't come with pre-canned python, but honestly almost everything (in terms of code) in PCI is available somewhere on the web and/or already built into python libraries, matlab, and/or R.

toby 18y ago

I really appreciate the feedback (I'm the author), I definitely expected this sort of response as I was writing it. I think I got a lot right when I set the level but having seen it in the wild there are things I would have done differently too. Mostly I wanted people to get excited and see that they could do this right now and that there were good reasons to. I never intended to compete with Norvig, more to create something that would inspire them to want to read it.