- I’m not a fan of Deep Learning by Goodfellow, I read it since it’s the best option available.
-
On the AI textbook side, there’s no avoiding “Artificial Intelligence: A Modern Approach”, still the standard for AI textbooks, I think. It surveys the discipline and capably introduces all the major concepts and tools of the trade. Just a word of warning: That book is thick as all get out and will require a real commitment to read. No TV. No radio. No nothing but you, that book, and several highlighters!
-
Also an excellent history of AI is “The Quest for Artificial Intelligence” by Nils Nilsson. It’s a bit of a dry read, but its coverage of AI’s past schools of thought is unparallelled.
-
Finally, in recognition of the revolution going on in AI right now, you’ll want to learn about Deep Learning and neural networks. Probably the best introduction is Michael Nielsen’s on-line work “Neural Networks and Deep Learning” http://neuralnetworksanddeeplearning.com .
- Having read that, the next step will be much more challenging — “Deep Learning”, the as-yet unpublished work by GoodFellow, Bengio, and Courville,
-
Once you’ve reached this point, I also recommend Juergen Schmidhuber’s 88 page survey paper on deep learning, “Deep Learning in Neural Networks: An Overview” which spans the subject’s concepts and history.
-
-
Imo Goodfellow kinda feels more like a reference book rather than something to learn from. Nielson’s free book is really good for beginners, however. I love the way he describes things, it’s so down to earth and simple to understand.
- Sutton’s RL book looks promising.
-
Amen. Sutton & barto is one of those really well done books that’s really trying to help you understand. I suggest that you don’t skip on any of the exercises as they help in demystifying everything.
-
- Machine Learning a Probabilistic Perspective by Murphy is ok too.
-
Introduction to Deep Learning: From Logical Calculus to Artificial Intelligence by Sandro Skansi. This is a great book, but as you may gather from reviews very subjective… some hate it, others think its great. I like it a lot, and introductory it may be, but definitely best if you approach it after a basic ML course/practice lab. Skansi is kind of like a mentor as he guides you along (with his views) – I learnt a lot from this not so well known book.
- My absolute fave is “Foundations of Machine Learning”
-
Neural Networks: A systematic introduction by Raul Rojas
-
Elements of Statistical Learning from Friedman et. al.
-
This book is a must-read: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning). It tackles the fundamentals of machine learning, like generalization and regularization, as well as more advanced topics like density estimation or generative models. It uses SVMs instead of neural networks (which makes things simpler and possible to analyze), but the principles apply to deep learning just as well.
- I thoroughly enjoyed David J. C. Mackay’s inference book. I liked his writing style so much that I went back and read a lot of his papers and his thesis
-
David Mackay, Information Theory, Inference and Learning Algorithms is still fro me the best book more on foundations side, especially the connections to information and communication theory. It uses information theory as a new lens to use to look at ML.
-
- I would also suggest Gaussian Process for Machine Learning by Rasmussen and Williams. GPs can be a powerful tool in your ML arsenal.
-
Bishop’s pattern recognition and machine learning is great, though it might potentially be a little elementary if your math is really solid.
-
Deep learning (DL); Goodfellow, Courville, Bengio: Covers deep learning from statistical / connnectionist perspective.
-
Pattern recognition and Machine Learning (PRML); Bishop: accessible introduction to machine learning. Bishop uses more examples and figures, where Murphy’s book is more equation heavy.
-
Machine learning, a probabilistic perspective (MLaPP); Murphy: More thorough survey of machine learning. I recognize more Bayesian flavours. There’s more focus on equations. However, consider this an encyclopedia rather than a book. Whereas one might read Bishop’s book end to end, I would not recommend it for this one.
-
Information theory, inference, learning algorithms(ITILA); Mackay. David Mackay is notoriously playful. Expect a new take on almost every model or algorithm you have encountered in the previous books. Being a curious mindself, this book is my favorite. I took this book on a mountain trip years ago and enjoyed each chapter.
-
Elements of statistical learning (ESL); Hastie Tibshirani: The statistical views on most of machine learning. Expect to dive into splines, RKHS, and many many tree-based models.
-
Bayesian Data Analysis (BDA); Gelman et all.: Imagine a perspective of machine learning books. On the left are Statistical books like ELS and ULM. In the middle are books like PRML and MLaPP. Well on that axis the BDA book is on the right. They take a Bayesian approach to nearly everything. Fortunately, they then also show the approximate inferences, so that you can actually apply the models to real world problems. In my opinion, this books suffers from too many authors. Reading the book, one feels that authors wrote different sections in their personal styles. I appreciate the book and hope one day somebody rewrites the book in one consistent story.
- Superintelligence by Nick Bostrom blew my mind.
- The best book for beginners, in my opinion, is Machine Learning with R. It doesn’t require any advanced math skills and gives an overview over the statistics.
History of ML
- This book of transcribed interviews is great. Talking Nets: An Oral History of Neural Networks
- The Quest for Artificial Intelligence: A History of Ideas and Achievements by Nilsson is a good history of AI written by one of the pioneers of AI research.
- I think any deep learning book would give you some good insight in the beginning chapters, have a look at the intro to Ian Goodfellow’s Deep Learning book
- the Lighthill report video feat. John Macarthy , Michie , Gregory vs Lighthill . I watched half of it and found to be really interesting and in some ways felt ironical as to how every company and government is literally pouring billions now .
-
The Master Algorithm.
Other sites
However if you plan on working with AI in the future you may as well learn some calculus and linear algebra right now. Khan Academy and Paul’s Online Math Notes are great sites for this.
Different kinds of AI. What specifically are you looking for?
Once you have the basics:
- There are genetic algorithms, which learn a solution to a stated problem by emulating the way evolution prunes species.
- There’s Bayesian classifiers that apply statistics to make probabilistically likely decisions.
- There’s straightforward decision-tree problems such as alpha-beta pruning, where you make a tree of all of the options available to you and make smart decisions on which branches your AI should follow.
- And, of course, there’s neural networks. They basically run derivatives on a huge number of nodes to determine which ones to change to produce the best answer they can. Neural nets in particular have led to some astounding results recently (see Andrey Karpathy’s blog titled “on the unreasonable effectiveness of recursive neural networks”).
CheckNotes 