Interesting on the one hand, but is anybody seriously going to go through those books one by one now? Personally I have troubles going through just one book (Pattern Recognition by Bishop atm), and even that might be useless without practical application. I managed to eventually read through MacKay (enjoyable book and available as a free PDF, too) and feel I have already forgotten most of it again :-(
Another way might be to just get going, and pick up knowledge on the way?
"but is anybody seriously going to go through those books one by one now? Personally I have troubles going through just one book (Pattern Recognition by Bishop atm), and even that might be useless without practical application"
Working through CLRS completely is a very time consuming task I think Bradford intended that book as a reference, but yes, you need to work through some of the stuff in order. For example, you need to be fairly conversant in Linear Algebra, probability, and proof technique before you can tackle Bishop, else you won't make much progress. Once you get some basics under you (especially the underlying math stuff) you'll end up being able to read through an ML book the way you can read through a moderately tough book in programming.
" I managed to eventually read through MacKay (enjoyable book and available as a free PDF, too) and feel I have already forgotten most of it again :-("
The best way to learn this stuff is to have an eventual project in mind. I ended up learning most of this stuff because I was working on a Robotics project for the fine folks in the Indian Defence Depts and was very much "thrown in at the deep end" - nothing like it to accelerate learning but I wouldn't wish to do it again. for the first few weeks I couldn't (literally) understand a single sentence in an hour long meeting. Very humbling.
Depending on what exactly you wish to do, you maybe able to avoid many of the books. If you think I can help you narrow down to a smaller list , please ask here or send me email (my email id is in my profile).
But yes, in the end Norvig's point applies here too (as Bradford points out. I have been working in ML for 8 years now so still 2 years to go :-P) .
OTOH I am just a programmer who got bored with enterprise software and have no formal training in math (or CS for that matter) and if I can do it anyone (certainly anyone on HN) can.
+1 I've been doing applied research for a decade. I think we need to double Norvig's 10 year rule for this field ... or maybe it is an infinite sequence.
I took a 3 year detour through pure engineering just because that part is so important and was causing me to experience a bottleneck.
I also agree that driving your studies with a real project helps tremendously.
Ultimately, you need to think about whether you really want to commit to this. It is very hard work, but also very fun and rewarding.
I wish there was a more hands on introduction, somehow. Having read through MacKay, I didn't feel as if I could just approach a company and suggest to do data analysis for them.
I suppose I should come up with my own projects, and I have some ideas, but they always have a huge question mark at the beginning.
To get the spirit of many of these techniques with practical examples in python using numpy/scipy check out the book "Machine Learning: An Algorithmic Perspective" by Stephen Marsland. It doesn't have the mathematical depth or proofs found in these other books, but the code is decent and will get you started doing some basic data analysis.
My advice is not to try to read through these books (like Elements or Bishop's book). If I were to learn the topic from scratch again * I would:
1. Learn basic terminology (basically, skim the chapters and understand roughly what the topics are)
2. Work on a problem in depth. You are probably interested in a certain area or type of problem.
a. Read the relevant chapters in detail.
b. Pick up the necessary math along the way using additional references. This way you are motivated to learn it (whether it be calculus, probability, or linear algebra). E.g., it would be hard to approach McDiarmid's Inequality and be able to imagine its use. However, if you run across it in a book/paper you'll understand the context.
c. Lastly, checkout recent NIPS, ICML, and JMLR papers on the topic (nips.cc, jmlr.org, and icml isn't centralized, but each conference can probably be found online).
* - I am a graduate student and have been studying statistical machine learning for the last 3 years.
Another way might be to just get going, and pick up knowledge on the way?