9 ms·

Ask HN: Best books on AI?

375 points by mavmak 10y ago

Which are the best books on AI, for somoone who doesnt have much knowledge on the topic. Mostly practical, and not academic.

ju-st 10y ago
"Python for Data Science For Dummies" by Luca Massaron & John Paul Mueller is a very practical book on machine learning. In a real world scenario your first obstacles will be learning how to use the programming language as well as preparing data. Both topics are well covered in this book. The "learning from data" chapter contains an introduction into basic machine learning algorithms as well as ensemble learning. The book contains minor inaccuracies but I think it's a good, practical start for a novice. It doesn't include anything on neural nets however.
cowpig 10y ago
AI: A Modern Approach (Russell, Norvig) and Deep Learning (Goodfellow, Bengio) have been mentioned already. I'd also recommend: Godel, Escher Bach: an Eternal Golden Braid by Douglas Hofstadter. Might not be exactly what you're looking for (it's all over the place, touching music theory, math, art, philosophy...), but it's fun and enjoyable to read. Also very dense.
brogrammer2 10y ago
Though it is an academic book, Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig. The first chapter in the book provides a detailed analysis of how other disciplines contribute to the idea of AI - from Philosophy to Psychology, Biology to Computer Science. Makes for an interesting read, even for a non-tech reader.
- iamlucaswolf 10y ago
  This. If you're also looking for a course that goes alongside the book, I highly recommend UC Berkley's CS188 (you can find it at http://ai.berkeley.edu http://ai.berkeley.edu). The lecturer Pieter Abbeel does such a good job explaining stuff and the programming exercises are really neat. Edit: Formatting
  - flor1s 10y ago
    The course is also quite easy to follow without buying the book. I love the exercises in which you are programming an intelligent agent to move through a maze. It reminded me of how we learned programming in university using Karel The Robot. This alongside Andrew Ng's Machine Learning course was my first exposure to the field. https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning I can also recommend Sebastian Thrun's Artificial Ingelligence for Robotics course: https://www.udacity.com/course/artificial-intelligence-for-robotics--cs373 https://www.udacity.com/course/artificial-intelligence-for-r...
- deleted 10y ago
  [deleted]
- danielam 10y ago
  I wouldn't exaggerate the "detailed analysis" here. For example, I found the philosophical parts quite weak and superficial.
curuinor 10y ago
AI is not a field where the practitioners can safely ignore the academic. A huge number of people got into rabbit-holes in its history and basically failed completely and wasted decades of their lives, in some cases.
- palad1n 10y ago
  Anecdotal. Do you have specific records of such failures?
  - curuinor 10y ago
    Well, it's not very nice to talk about people spending decades on failure, but I guess I can give some examples. Cycorp still exists, from 1984. However, D. Lenat's approach to AI by ontology engineering has basically been completely infertile after about the early 90's. Feigenbaum's expert systems stuff was a basic bust, it led to the Japanese just throwing that stuff away. People spent an incredible amount of effort and time systematizing expert knowledge and making expert systems and it was not a happy time. Much of that knowledge went into probabilistic forms, culminating in the Bayes net. The most famous application of Bayes net: Clippy (there are a lot more successful applications, but still...) It was believed shortly after the AI conference that computer vision could be solved by a summer project in the 50's. That didn't happen.
  - curuinor 10y ago
    More failures. Minsky and Papert gave a criticism on single-layer perceptrons in '86 where they proved that they could only make linear discriminators and therefore were useless for any real practical purposes harder than the XOR problem. They were wrong, given that we call multi-layer perceptrons neural networks. Simon and Newell made their model and thought that models like theirs with production rules would point the way towards the way that humans could systematize thought. That didn't happen, although they had some cool papers. People saw ELIZA and SHRDLU and thought that good NLP was coming in only a decade or so.... in the 60's. Beveridge report. "The spirit was willing, but the flesh was weak" to "The vodka was good, but the meat was rotten." (that last one's a bit apocryphal, but still)
  - 10y ago
hpvic03 10y ago
If you want to get into actually implementing deep neural nets (what everyone is calling "AI" these days), then look at TensorFlow and Keras. This is a good getting started book for TensorFlow: https://www.amazon.com/TensorFlow-Machine-Learning-Cookbook-McClure-ebook/dp/B01HY3TC54 https://www.amazon.com/TensorFlow-Machine-Learning-Cookbook-...
kruhft 10y ago
Paradigms of Artificial Intelligence Programming (PAIP) One of the best books on AI and Programming ever.
- curuinor 10y ago
  Unfortunately, there is not a drop of numerics in that book. It's a good book for learning about symbolic AI. There is, to a solid first order approximation, zero symbolic AI in a system like, say, Google speech recognition.
  - kleiba 10y ago
    Not surprisingly, I would say, since I wouldn't count speech recognition as an AI task.
    - Scarblac 10y ago
      Nothing is counted as an AI task after someone finally manages to do it.
      - kleiba 10y ago
        I anticipated that argument, but I wouldn't have considered ASR an AI task even before it was first attempted.
        bstamour 10y ago
        Any particular reason why? I feel that the problem of mapping sensory inputs like sound etc onto internal concepts for reasoning is an important part of AI.
        kleiba 10y ago
        My personal reason is that I don't consider the output of speech recognition to be "internal concepts for reasoning". In a Spoken Dialog System, this task is typically performed by a subsequent component that does natural language understanding (rather than recognition).
shoshin23 10y ago
Superintelligence by nick bostrom. It's a book that explores how superintelligence could emerge, the different ways it can take off and what it means to us as humans. More importantly, the book takes on the difficult task of figuring out ways to make sure the AI is safe and not land up in the wrong hands. Pretty interesting read that doesnt really require technical know-how.
- curuinor 10y ago
  There is lots of content in this book but there is no practical technical content in this book. Interesting philosophy. Much of AI philosophy is done by extremely non-practitioners. John Searle can't code. Nick Bostrom came to coding extremely late in life. Geoffrey Hinton and the other ex-PDP folks wrote some philosophy papers, though, which are of interest if you like the philosophy.
- adastra 10y ago
  The book is interesting, but the level of detail he goes into in some of his speculation is completely unjustified. It's kind of ridiculous. "There's no sense in being precise when you don't even know what you're talking about." - von Neumann
davidhunter 10y ago
The Quest for Artificial Intelligence: A History of Ideas and Achievements by Nils Nilsson Gives a great run through of the history of AI research. Understanding the approaches that have been tried before gives you a sense of why the state of the field is what it is today. It is worth bearing in mind that AI research expands far beyond computer science into psychology, philosophy, linguistics etc.
- flor1s 10y ago
  I think there is a pdf draft of the book legally available from http://ai.stanford.edu/~nilsson/QAI/qai.pdf http://ai.stanford.edu/~nilsson/QAI/qai.pdf
iamlucaswolf 10y ago
To iterate on what others said, but what was not emphasized enough from my point of few: AI is academic (as a synonym for 'theoretical' and 'math-intensive'). Once you look beyond purely symbolic AI, which proved to be infeasible as @curuinor pointed out somewhere here, you will need to build up at least basic knowledge in probability theory and linear algebra. The path I'm following at the moment is a quite rigorous one and is outlined here (http://www.deeplearningweekly.com/pages/open_source_deep_learning_curriculum http://www.deeplearningweekly.com/pages/open_source_deep_lea...). If you've never had any exposure to probability theory or statistics, I recommend having a look at the course "MIT 6.041 Probabilistic Systems Analysis and Applied Probability" taught by John Tsitsiklis at MIT (video lectures are available through YouTube and MIT OpenCourseWare for free). Both the course and Tsitsiklis' book are superb learning materials to get into probabilisitc thinking. Edit: Link was broken. Thanks to @blauditore.
- flor1s 10y ago
  I think to be successful at machine learning you also need a good understanding of calculus, besides probability and linear algebra.
  - gugagore 10y ago
    but perhaps not too much: https://news.ycombinator.com/item?id=9189553 https://news.ycombinator.com/item?id=9189553
    - curuinor 10y ago
      Strang's complaint is that there's too little linear algebra. This is true. This doesn't overshadow the fact that you're not going to get out of using some partial derivatives in neural net land (and many other AI subfields).
    - flor1s 10y ago
      What is too much though? Backpropagation uses derivatives, some filters in Computer Vision use multivariate calculus. If you want to have a thorough understanding then calculus is necessary. That said, Andrew Ng was quite good at avoiding calculus in his Machine Learning MOOC, and for applied machine learning I guess calculus is not that important. A great place to study about math is www.khanacademy.org, they have courses on calculus, probability/statistics and linear algebra.
blojayble 10y ago
Any recommendations for a book focused on AI in game development?
- fapjacks 10y ago
  I've got a book called "Game Programming Patterns" by Robert Nystrom which goes into this. It's not advanced, but it's very good.
vasira 10y ago
The society of mind and the diamond age !
arihant 10y ago
You cannot really go practical in AI without academic rigor. You can do recipes of what's already been done using a TensorFlow book, but that's how far one can go. If one is serious in getting in AI today, a great way to do is read the following books, in order: 1. AI: A Modern Approach by Stuart Russell and Peter Norvig. 2. Deep Learning by Ian Goodfellow and Yoshua Bengio. It is amazing how approachable both books are for beginners, but you will be diving a lot into academic stuff as you go along.
- deepnet 10y ago
  Sutton & Barto's Reinforment Learning complete this triumvirate https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.html https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.htm... David Silver's Reinforcement Course is based on Sutton & Barto https://www.youtube.com/watch?v=2pWv7GOvuf0 https://www.youtube.com/watch?v=2pWv7GOvuf0
  - jumpCastle 10y ago
    I liked all three of them. I also feel Murphy's MLAPP provides additional useful material and is well written.
  - thedailymail 10y ago
    For whatever reason, Sutton's was the first serious book I read in any area of AI. The balance between explaining the history, the concepts and the code is handled really well.
- nextos 10y ago
  I love Paradigms of Artificial Intelligence by Peter Norvig much more than AIMA, which I found excessively encyclopedic and shallow. While some will argue it is dated, I think it presents many timeless ideas that will get in vogue soon with little tweaks to their inference schemes. Same for The Art of Prolog.
  - curuinor 10y ago
    If you're shipping anything after 2010, you're not going to get within an order of magnitude within state of the art with that book (PAI), unfortunately. There's basically no numerics in that book about anything that'll past muster at NIPS or ICML nowadays or would be shipped by one of the big corporate AI labs, I'm sorry to say.
payne92 10y ago
The challenge I've found with books is the space has been moving so quickly in the past 10 years. By the time the book is out, the methods described in it are no longer state of the art.
- endisukaj 10y ago
  That's why it's important to understand the underlying math.
krosaen 10y ago
Some on this thread have recommended Norvig's PAIP, but that's kind of an old school AI book in that it focuses on heuristic search and logic (implementing prolog in lisp at one point, very impressive stuff actually); but is lacking any coverage of statistical machine learning, which is the approach that underlies most of the cool stuff these days. It's still a great book, but I'd instead recommend a path that focuses on machine learning: - The Master Algorithm: made for a general audience, gives you a lay of the land - Python Machine Learning by Sebastian Raschka: gives you practical skills using python, scikit-learn, numpy, jupyter notebooks, pandas etc. From zero to kaggle in 4 chapters, goes deeper after that. Also goes into enough theory you aren't flying completely blind. After that, I'm afraid I think you do need to go "academic", if by that you mean learning some of the underlying math to approach AI / ML from a more rigorous probabilistic perspective. I'd recommend studying probability theory and then working your way through Bishop's Pattern Recognition and Machine Learning. After that a lot more doors open up too more specialized topics like computer vision, reinforcement learning etc. I've written up a lot more about this here: http://karlrosaen.com/ml/ http://karlrosaen.com/ml/
- JamilD 10y ago
  You don't need to focus on just one side or the other. AlphaGo wasn't made by just hacking together a couple neural nets, it built on heuristic search, MCTS, and the "old-school" AI. I agree that basic statistics + Bishop's book is a great way to start getting into machine learning -- but AI is a much broader field than that.
  - taktoa 10y ago
    "old-school" AI is just a pretentious name for algorithms/graph theory/combinatorics, plus a bit of (now very outdated) PLT mixed in. "new-school" AI (machine learning) is just a more pretentious name for statistics/control theory/randomized algorithms.
    - nojvek 10y ago
      What is to say the human brain doesn't engage in graph combinotrics with some learned heuristics. Some of the old ideas with enough computation power are actually pretty amazing.
JamilD 10y ago
Firstly, I don't think you can dive straight into coding without understanding the fundamentals. AI is such a broad and rich field, and there's a lot you need to know before you start. It also depends on what you're going to focus on. Are you looking to implement a game-playing agent? An object recognition algorithm? More of a logic focus? If you just want Deep Learning and statistical methods, then Bishop's Pattern Recognition and Machine Learning is a good start. Otherwise, Russel and Norvig's Artificial Intelligence or Patrick Winston's similarly titled book are great starting points. For more big-picture stuff, Marvin Minsky's Society of Mind is great, and Hofstader's Gödel, Escher, Bach is a classic too. Both are a lot less practical though, which seems to be what you're looking for.
pieterp 10y ago
The Second Machine Age
learningtool 10y ago
A good suggestion seems to be not to read a lot of books, but to put intro practice what you read and apply it to problems you need to solve, that way you learn a lot more effectively
akssri 10y ago
Graphical models might also be something folks might want to consider. Ideas from PGMs are often behind many advances in ML. The canonical text is by Daphne Koller; a course I took used Martin Wainwright's monograph though - the book is briefer and dives into the math quicker. https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_F...
mnault000 10y ago
Try this site: http://www.allitebooks.com/?s=artificial http://www.allitebooks.com/?s=artificial And the books are free and downloadable :)
visarga 10y ago
Imagine you want to learn the Roman alphabet. Which book should you use? Any book. There are so many good books and courses that it's almost useless to select. Don't worry about selecting the initial book, just use any course or book in the beginning, and later when you will know exactly what fits your needs, you will be able to fine tune.
danielmorozoff 10y ago
Once you get your feet wet, the first year PHD course book for good theory is Pattern Recognition and Machine Learning by Bishop. But it could be a bit too theoretical - it provides a foundational mathematical framework and got me thinking about problems in a better way.
halis 10y ago
Just watch the movie with Jude Law. We're close, we're so close!
sehugg 10y ago
You know, I have never found a "casual" book covering tree-search techniques, from minimax to Monte Carlo Tree Search. Still relevant for game/agent AI (AlphaGo used MCTS for example). You'd think there would have been 100 "How To Make a Computer Chess Engine in BASIC" books back in the 80s, and continuing to the present day, but I can't find them. Lots of papers and online tutorials, and some stuff in textbooks, but no accessible hands-on books.
j_s 10y ago
First, read fucking Hastie, Tibshirani, and whoever. Chapters 1-4 and 7. If you don't understand it, keep reading it until you do. You can read the rest of the book if you want. You probably should, but I'll assume you know all of it. Take Andrew Ng's Coursera. Do all the exercises in Matlab and python and R. Make sure you get the same answers with all of them. Now forget all of that and read the deep learning book. Put tensorflow or torch on a Linux box and run examples until you get it. Do stuff with CNNs and RNNs and just feed forward NNs. Once you do all of that, go on arXiv and read the most recent useful papers. The literature changes every few months, so keep up. There. Now you can probably be hired most places. If you need resume filler, so some Kaggle competitions. If you have debugging questions, use StackOverflow. If you have math questions, read more. If you have life questions, I have no idea. source: fizixer https://news.ycombinator.com/item?id=13890952 https://news.ycombinator.com/item?id=13890952 FWIW, a "super harsh" guide to (learning) ML [1] was posted on reddit a few days ago. [1] https://redd.it/5z8110 https://redd.it/5z8110 Edit: The entire Reddit discussion feels slightly similar to this one, if more snarky. The first reply there also links all the resources listed above. I don't really know enough to add anything.
- radicality 10y ago
  Statistical learning is only one part of AI. If the person wants to learn about AI, wouldn't starting with something like "AI: A modern approach" he a better start?
- cs702 10y ago
  I would add one more item to the list: replicate the results of at least a few "classic" deep learning papers from scratch in one of the popular frameworks (TensorFlow, Torch, Caffe, etc.), instead of downloading code written by others. For example, build and train Alexnet or one of the VGGnets, a Word2Vec model, an image captioning model (joint CNN and LSTM RNN), and a pong- or breakout-playing AI (CNN with reinforcement learning). It's possible to do all of this on a single machine with a relatively inexpensive GPU.
dongslol 10y ago
Get your feet wet: * http://cs231n.stanford.edu/ http://cs231n.stanford.edu/ (the course notes are excellent) * http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/ * http://www.deeplearningbook.org/ http://www.deeplearningbook.org/ Use Tensorflow to train a few small neural nets. Move on to CNNs and RNNs. Make sure you actually do this. By this point you'll have read a lot, and retain none of it if you don't put it to use. Look at reinforcement learning. Use the book by Sutton and Barto, the new edition: https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.html https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.htm... Read the first 4-5 chapters, then go online and read about Deep Q learning, policy gradients, DDPG, etc. Then try to solve some problems on OpenAI Gym. Once you have an idea of the kinds of problems you can solve, and have a couple you're interested in, go back and learn the foundational math, and start reading research papers. In general, start with modern books that mention deep learning. With older books or high-level-overview books, you'll get frustrated when you see something cool on /r/machinelearning and can't find any mention of it in the book.
mrdrozdov 10y ago
David Rosenberg's Machine Learning course is an excellent intro. Will give you the foundations that you need for anything else. Has a few links for additional resources, but the slides are mostly sufficient. https://davidrosenberg.github.io/ml2015 https://davidrosenberg.github.io/ml2015
tvalentius 10y ago
If you are developing AI for games, then Programming Game AI By Example by Mat Buckland are one of the best book for that : https://www.amazon.com/Programming-Example-Wordware-Developers-Library/dp/1556220782 https://www.amazon.com/Programming-Example-Wordware-Develope...
- LyutD 10y ago
  I can also vouch for this, it's a great introduction to AI in Games, however it is just an introduction as the topic is very broad.
MidoAssran 10y ago
Bishop, Hastie, Shai Shalev-Shwartz
tatoalo 10y ago
I'd spend my money on these two: - The Emotion Machine by Minsky - Superintelligence: Paths, Dangers, Strategies by Nick Bostrom
giardini 10y ago
Sometimes its best to answer a question with a question. Are you simply curious or is there something more pressing? For example, do you want some light reading or have you perhaps been asked to implement machine learning for your company? Most answers here assume you want to jump into the ML swamp and start analyzing your trove of "big data" ASAP. But is that so?
- mavmak 10y ago
  I want to have a basic understanding on the topic, and implement something small, maybe a side project.

375 points by mavmak 10y ago

Which are the best books on AI, for somoone who doesnt have much knowledge on the topic. Mostly practical, and not academic.

ju-st 10y ago

"Python for Data Science For Dummies" by Luca Massaron & John Paul Mueller is a very practical book on machine learning. In a real world scenario your first obstacles will be learning how to use the programming language as well as preparing data. Both topics are well covered in this book. The "learning from data" chapter contains an introduction into basic machine learning algorithms as well as ensemble learning. The book contains minor inaccuracies but I think it's a good, practical start for a novice. It doesn't include anything on neural nets however.

cowpig 10y ago

AI: A Modern Approach (Russell, Norvig) and Deep Learning (Goodfellow, Bengio) have been mentioned already. I'd also recommend: Godel, Escher Bach: an Eternal Golden Braid by Douglas Hofstadter. Might not be exactly what you're looking for (it's all over the place, touching music theory, math, art, philosophy...), but it's fun and enjoyable to read. Also very dense.

brogrammer2 10y ago

Though it is an academic book, Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig. The first chapter in the book provides a detailed analysis of how other disciplines contribute to the idea of AI - from Philosophy to Psychology, Biology to Computer Science. Makes for an interesting read, even for a non-tech reader.

iamlucaswolf 10y ago
This. If you're also looking for a course that goes alongside the book, I highly recommend UC Berkley's CS188 (you can find it at http://ai.berkeley.edu http://ai.berkeley.edu). The lecturer Pieter Abbeel does such a good job explaining stuff and the programming exercises are really neat. Edit: Formatting
- flor1s 10y ago
  The course is also quite easy to follow without buying the book. I love the exercises in which you are programming an intelligent agent to move through a maze. It reminded me of how we learned programming in university using Karel The Robot. This alongside Andrew Ng's Machine Learning course was my first exposure to the field. https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning I can also recommend Sebastian Thrun's Artificial Ingelligence for Robotics course: https://www.udacity.com/course/artificial-intelligence-for-robotics--cs373 https://www.udacity.com/course/artificial-intelligence-for-r...
deleted 10y ago
[deleted]
danielam 10y ago
I wouldn't exaggerate the "detailed analysis" here. For example, I found the philosophical parts quite weak and superficial.

curuinor 10y ago

AI is not a field where the practitioners can safely ignore the academic. A huge number of people got into rabbit-holes in its history and basically failed completely and wasted decades of their lives, in some cases.

palad1n 10y ago
Anecdotal. Do you have specific records of such failures?
- curuinor 10y ago
  Well, it's not very nice to talk about people spending decades on failure, but I guess I can give some examples. Cycorp still exists, from 1984. However, D. Lenat's approach to AI by ontology engineering has basically been completely infertile after about the early 90's. Feigenbaum's expert systems stuff was a basic bust, it led to the Japanese just throwing that stuff away. People spent an incredible amount of effort and time systematizing expert knowledge and making expert systems and it was not a happy time. Much of that knowledge went into probabilistic forms, culminating in the Bayes net. The most famous application of Bayes net: Clippy (there are a lot more successful applications, but still...) It was believed shortly after the AI conference that computer vision could be solved by a summer project in the 50's. That didn't happen.
- curuinor 10y ago
  More failures. Minsky and Papert gave a criticism on single-layer perceptrons in '86 where they proved that they could only make linear discriminators and therefore were useless for any real practical purposes harder than the XOR problem. They were wrong, given that we call multi-layer perceptrons neural networks. Simon and Newell made their model and thought that models like theirs with production rules would point the way towards the way that humans could systematize thought. That didn't happen, although they had some cool papers. People saw ELIZA and SHRDLU and thought that good NLP was coming in only a decade or so.... in the 60's. Beveridge report. "The spirit was willing, but the flesh was weak" to "The vodka was good, but the meat was rotten." (that last one's a bit apocryphal, but still)
- 10y ago

hpvic03 10y ago

If you want to get into actually implementing deep neural nets (what everyone is calling "AI" these days), then look at TensorFlow and Keras. This is a good getting started book for TensorFlow: https://www.amazon.com/TensorFlow-Machine-Learning-Cookbook-McClure-ebook/dp/B01HY3TC54 https://www.amazon.com/TensorFlow-Machine-Learning-Cookbook-...

kruhft 10y ago

Paradigms of Artificial Intelligence Programming (PAIP) One of the best books on AI and Programming ever.

curuinor 10y ago
Unfortunately, there is not a drop of numerics in that book. It's a good book for learning about symbolic AI. There is, to a solid first order approximation, zero symbolic AI in a system like, say, Google speech recognition.
- kleiba 10y ago
  Not surprisingly, I would say, since I wouldn't count speech recognition as an AI task.
  - Scarblac 10y ago
    Nothing is counted as an AI task after someone finally manages to do it.
    - kleiba 10y ago
      I anticipated that argument, but I wouldn't have considered ASR an AI task even before it was first attempted.
      - bstamour 10y ago
        Any particular reason why? I feel that the problem of mapping sensory inputs like sound etc onto internal concepts for reasoning is an important part of AI.
        kleiba 10y ago
        My personal reason is that I don't consider the output of speech recognition to be "internal concepts for reasoning". In a Spoken Dialog System, this task is typically performed by a subsequent component that does natural language understanding (rather than recognition).

shoshin23 10y ago

Superintelligence by nick bostrom. It's a book that explores how superintelligence could emerge, the different ways it can take off and what it means to us as humans. More importantly, the book takes on the difficult task of figuring out ways to make sure the AI is safe and not land up in the wrong hands. Pretty interesting read that doesnt really require technical know-how.

curuinor 10y ago
There is lots of content in this book but there is no practical technical content in this book. Interesting philosophy. Much of AI philosophy is done by extremely non-practitioners. John Searle can't code. Nick Bostrom came to coding extremely late in life. Geoffrey Hinton and the other ex-PDP folks wrote some philosophy papers, though, which are of interest if you like the philosophy.
adastra 10y ago
The book is interesting, but the level of detail he goes into in some of his speculation is completely unjustified. It's kind of ridiculous. "There's no sense in being precise when you don't even know what you're talking about." - von Neumann

davidhunter 10y ago

The Quest for Artificial Intelligence: A History of Ideas and Achievements by Nils Nilsson Gives a great run through of the history of AI research. Understanding the approaches that have been tried before gives you a sense of why the state of the field is what it is today. It is worth bearing in mind that AI research expands far beyond computer science into psychology, philosophy, linguistics etc.

flor1s 10y ago
I think there is a pdf draft of the book legally available from http://ai.stanford.edu/~nilsson/QAI/qai.pdf http://ai.stanford.edu/~nilsson/QAI/qai.pdf

iamlucaswolf 10y ago

To iterate on what others said, but what was not emphasized enough from my point of few: AI is academic (as a synonym for 'theoretical' and 'math-intensive'). Once you look beyond purely symbolic AI, which proved to be infeasible as @curuinor pointed out somewhere here, you will need to build up at least basic knowledge in probability theory and linear algebra. The path I'm following at the moment is a quite rigorous one and is outlined here (http://www.deeplearningweekly.com/pages/open_source_deep_learning_curriculum http://www.deeplearningweekly.com/pages/open_source_deep_lea...). If you've never had any exposure to probability theory or statistics, I recommend having a look at the course "MIT 6.041 Probabilistic Systems Analysis and Applied Probability" taught by John Tsitsiklis at MIT (video lectures are available through YouTube and MIT OpenCourseWare for free). Both the course and Tsitsiklis' book are superb learning materials to get into probabilisitc thinking. Edit: Link was broken. Thanks to @blauditore.

flor1s 10y ago
I think to be successful at machine learning you also need a good understanding of calculus, besides probability and linear algebra.
- gugagore 10y ago
  but perhaps not too much: https://news.ycombinator.com/item?id=9189553 https://news.ycombinator.com/item?id=9189553
  - curuinor 10y ago
    Strang's complaint is that there's too little linear algebra. This is true. This doesn't overshadow the fact that you're not going to get out of using some partial derivatives in neural net land (and many other AI subfields).
  - flor1s 10y ago
    What is too much though? Backpropagation uses derivatives, some filters in Computer Vision use multivariate calculus. If you want to have a thorough understanding then calculus is necessary. That said, Andrew Ng was quite good at avoiding calculus in his Machine Learning MOOC, and for applied machine learning I guess calculus is not that important. A great place to study about math is www.khanacademy.org, they have courses on calculus, probability/statistics and linear algebra.

blojayble 10y ago

Any recommendations for a book focused on AI in game development?

fapjacks 10y ago
I've got a book called "Game Programming Patterns" by Robert Nystrom which goes into this. It's not advanced, but it's very good.

vasira 10y ago

The society of mind and the diamond age !

arihant 10y ago

You cannot really go practical in AI without academic rigor. You can do recipes of what's already been done using a TensorFlow book, but that's how far one can go. If one is serious in getting in AI today, a great way to do is read the following books, in order: 1. AI: A Modern Approach by Stuart Russell and Peter Norvig. 2. Deep Learning by Ian Goodfellow and Yoshua Bengio. It is amazing how approachable both books are for beginners, but you will be diving a lot into academic stuff as you go along.

deepnet 10y ago
Sutton & Barto's Reinforment Learning complete this triumvirate https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.html https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.htm... David Silver's Reinforcement Course is based on Sutton & Barto https://www.youtube.com/watch?v=2pWv7GOvuf0 https://www.youtube.com/watch?v=2pWv7GOvuf0
- jumpCastle 10y ago
  I liked all three of them. I also feel Murphy's MLAPP provides additional useful material and is well written.
- thedailymail 10y ago
  For whatever reason, Sutton's was the first serious book I read in any area of AI. The balance between explaining the history, the concepts and the code is handled really well.
nextos 10y ago
I love Paradigms of Artificial Intelligence by Peter Norvig much more than AIMA, which I found excessively encyclopedic and shallow. While some will argue it is dated, I think it presents many timeless ideas that will get in vogue soon with little tweaks to their inference schemes. Same for The Art of Prolog.
- curuinor 10y ago
  If you're shipping anything after 2010, you're not going to get within an order of magnitude within state of the art with that book (PAI), unfortunately. There's basically no numerics in that book about anything that'll past muster at NIPS or ICML nowadays or would be shipped by one of the big corporate AI labs, I'm sorry to say.

payne92 10y ago

The challenge I've found with books is the space has been moving so quickly in the past 10 years. By the time the book is out, the methods described in it are no longer state of the art.

endisukaj 10y ago
That's why it's important to understand the underlying math.

krosaen 10y ago

Some on this thread have recommended Norvig's PAIP, but that's kind of an old school AI book in that it focuses on heuristic search and logic (implementing prolog in lisp at one point, very impressive stuff actually); but is lacking any coverage of statistical machine learning, which is the approach that underlies most of the cool stuff these days. It's still a great book, but I'd instead recommend a path that focuses on machine learning: - The Master Algorithm: made for a general audience, gives you a lay of the land - Python Machine Learning by Sebastian Raschka: gives you practical skills using python, scikit-learn, numpy, jupyter notebooks, pandas etc. From zero to kaggle in 4 chapters, goes deeper after that. Also goes into enough theory you aren't flying completely blind. After that, I'm afraid I think you do need to go "academic", if by that you mean learning some of the underlying math to approach AI / ML from a more rigorous probabilistic perspective. I'd recommend studying probability theory and then working your way through Bishop's Pattern Recognition and Machine Learning. After that a lot more doors open up too more specialized topics like computer vision, reinforcement learning etc. I've written up a lot more about this here: http://karlrosaen.com/ml/ http://karlrosaen.com/ml/

JamilD 10y ago
You don't need to focus on just one side or the other. AlphaGo wasn't made by just hacking together a couple neural nets, it built on heuristic search, MCTS, and the "old-school" AI. I agree that basic statistics + Bishop's book is a great way to start getting into machine learning -- but AI is a much broader field than that.
- taktoa 10y ago
  "old-school" AI is just a pretentious name for algorithms/graph theory/combinatorics, plus a bit of (now very outdated) PLT mixed in. "new-school" AI (machine learning) is just a more pretentious name for statistics/control theory/randomized algorithms.
  - nojvek 10y ago
    What is to say the human brain doesn't engage in graph combinotrics with some learned heuristics. Some of the old ideas with enough computation power are actually pretty amazing.

JamilD 10y ago

Firstly, I don't think you can dive straight into coding without understanding the fundamentals. AI is such a broad and rich field, and there's a lot you need to know before you start. It also depends on what you're going to focus on. Are you looking to implement a game-playing agent? An object recognition algorithm? More of a logic focus? If you just want Deep Learning and statistical methods, then Bishop's Pattern Recognition and Machine Learning is a good start. Otherwise, Russel and Norvig's Artificial Intelligence or Patrick Winston's similarly titled book are great starting points. For more big-picture stuff, Marvin Minsky's Society of Mind is great, and Hofstader's Gödel, Escher, Bach is a classic too. Both are a lot less practical though, which seems to be what you're looking for.

pieterp 10y ago

The Second Machine Age

learningtool 10y ago

A good suggestion seems to be not to read a lot of books, but to put intro practice what you read and apply it to problems you need to solve, that way you learn a lot more effectively

akssri 10y ago

Graphical models might also be something folks might want to consider. Ideas from PGMs are often behind many advances in ML. The canonical text is by Daphne Koller; a course I took used Martin Wainwright's monograph though - the book is briefer and dives into the math quicker. https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_F...

mnault000 10y ago

Try this site: http://www.allitebooks.com/?s=artificial http://www.allitebooks.com/?s=artificial And the books are free and downloadable :)

visarga 10y ago

Imagine you want to learn the Roman alphabet. Which book should you use? Any book. There are so many good books and courses that it's almost useless to select. Don't worry about selecting the initial book, just use any course or book in the beginning, and later when you will know exactly what fits your needs, you will be able to fine tune.

danielmorozoff 10y ago

Once you get your feet wet, the first year PHD course book for good theory is Pattern Recognition and Machine Learning by Bishop. But it could be a bit too theoretical - it provides a foundational mathematical framework and got me thinking about problems in a better way.

halis 10y ago

Just watch the movie with Jude Law. We're close, we're so close!

sehugg 10y ago

You know, I have never found a "casual" book covering tree-search techniques, from minimax to Monte Carlo Tree Search. Still relevant for game/agent AI (AlphaGo used MCTS for example). You'd think there would have been 100 "How To Make a Computer Chess Engine in BASIC" books back in the 80s, and continuing to the present day, but I can't find them. Lots of papers and online tutorials, and some stuff in textbooks, but no accessible hands-on books.

j_s 10y ago

First, read fucking Hastie, Tibshirani, and whoever. Chapters 1-4 and 7. If you don't understand it, keep reading it until you do. You can read the rest of the book if you want. You probably should, but I'll assume you know all of it. Take Andrew Ng's Coursera. Do all the exercises in Matlab and python and R. Make sure you get the same answers with all of them. Now forget all of that and read the deep learning book. Put tensorflow or torch on a Linux box and run examples until you get it. Do stuff with CNNs and RNNs and just feed forward NNs. Once you do all of that, go on arXiv and read the most recent useful papers. The literature changes every few months, so keep up. There. Now you can probably be hired most places. If you need resume filler, so some Kaggle competitions. If you have debugging questions, use StackOverflow. If you have math questions, read more. If you have life questions, I have no idea. source: fizixer https://news.ycombinator.com/item?id=13890952 https://news.ycombinator.com/item?id=13890952 FWIW, a "super harsh" guide to (learning) ML [1] was posted on reddit a few days ago. [1] https://redd.it/5z8110 https://redd.it/5z8110 Edit: The entire Reddit discussion feels slightly similar to this one, if more snarky. The first reply there also links all the resources listed above. I don't really know enough to add anything.

radicality 10y ago
Statistical learning is only one part of AI. If the person wants to learn about AI, wouldn't starting with something like "AI: A modern approach" he a better start?
cs702 10y ago
I would add one more item to the list: replicate the results of at least a few "classic" deep learning papers from scratch in one of the popular frameworks (TensorFlow, Torch, Caffe, etc.), instead of downloading code written by others. For example, build and train Alexnet or one of the VGGnets, a Word2Vec model, an image captioning model (joint CNN and LSTM RNN), and a pong- or breakout-playing AI (CNN with reinforcement learning). It's possible to do all of this on a single machine with a relatively inexpensive GPU.

dongslol 10y ago

Get your feet wet: * http://cs231n.stanford.edu/ http://cs231n.stanford.edu/ (the course notes are excellent) * http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/ * http://www.deeplearningbook.org/ http://www.deeplearningbook.org/ Use Tensorflow to train a few small neural nets. Move on to CNNs and RNNs. Make sure you actually do this. By this point you'll have read a lot, and retain none of it if you don't put it to use. Look at reinforcement learning. Use the book by Sutton and Barto, the new edition: https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.html https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.htm... Read the first 4-5 chapters, then go online and read about Deep Q learning, policy gradients, DDPG, etc. Then try to solve some problems on OpenAI Gym. Once you have an idea of the kinds of problems you can solve, and have a couple you're interested in, go back and learn the foundational math, and start reading research papers. In general, start with modern books that mention deep learning. With older books or high-level-overview books, you'll get frustrated when you see something cool on /r/machinelearning and can't find any mention of it in the book.

mrdrozdov 10y ago

David Rosenberg's Machine Learning course is an excellent intro. Will give you the foundations that you need for anything else. Has a few links for additional resources, but the slides are mostly sufficient. https://davidrosenberg.github.io/ml2015 https://davidrosenberg.github.io/ml2015

tvalentius 10y ago

If you are developing AI for games, then Programming Game AI By Example by Mat Buckland are one of the best book for that : https://www.amazon.com/Programming-Example-Wordware-Developers-Library/dp/1556220782 https://www.amazon.com/Programming-Example-Wordware-Develope...

LyutD 10y ago
I can also vouch for this, it's a great introduction to AI in Games, however it is just an introduction as the topic is very broad.

MidoAssran 10y ago

Bishop, Hastie, Shai Shalev-Shwartz

tatoalo 10y ago

I'd spend my money on these two: - The Emotion Machine by Minsky - Superintelligence: Paths, Dangers, Strategies by Nick Bostrom

giardini 10y ago

Sometimes its best to answer a question with a question. Are you simply curious or is there something more pressing? For example, do you want some light reading or have you perhaps been asked to implement machine learning for your company? Most answers here assume you want to jump into the ML swamp and start analyzing your trove of "big data" ASAP. But is that so?

mavmak 10y ago
I want to have a basic understanding on the topic, and implement something small, maybe a side project.