Self Study: Probability and Bayesian Statistics
An average 21-year-old with a college degree spends a little over 15000 hours in classroom lectures. This accounts for a roughly 8th portion of his / her active life on this planet. To me, this alone is a good enough reason to bring constant improvement in our education system. The idea is to make learning fun! Let me make my small contribution by suggesting a few strategies I feel might be useful while learning concepts in higher education. I am going to use these same strategies to recommend books to learn probability and Bayesian statistics.
Struggle of Learning
As I have mentioned before, my formal education is in physics. I hope you could appreciate that quantum mechanics or astrophysics can be really daunting to the young students of physics. Despite my love for physics, like most students, I struggled to grasp the concepts. My two biggest struggles were
1) I thought I was the only one struggling.
2) A major part of my training in physics was about learning highly complicated calculus without the knowledge of its applications in physics.
My Analysis of the Learning Struggle
I recently read a book by Simon Singh (yes I love his writing) called ‘Big Bang’. In this book he has presented the historical evolution of astrophysics. I wish I had read it while I was studying physics because I could see reasons for my struggles.
1) Even the greatest minds including Galileo, Newton and Einstein struggled with ideas – It is always good to learn about the struggle that the founding fathers / mothers of the theory have gone through before formulating the concepts. This is a similar struggle that a new learner faces and relates to.
2) The founders of the theory first looked at the big gigantic sky, got puzzled and asked questions. They did their calculus (a tool to answer the questions) much later. I don’t see a reason why the students should do it the other way round.
Using the above concepts, let me present two different styles of learning in the next segment using a pictorial representation.
Of the above two learning styles, I prefer the second one. Keeping this in mind, in this part of the article, let me suggest the big picture (popular science) books for you to learn probability and Bayesian Statistics. I have divided these books into two parts
1) Historic and conceptual perspective
In the next part I will suggest books that will provide you the tools and methods to implements Bayesian statistics.
1) Historic and Conceptual Perspective
The Theory That Would Not Die – How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy – by Sharon Bertsch McGrayne
YOU CANalytics Book Rating (3.7 / 5)
I had really high expectations from this book when I started reading it – and it fell a little short. The book did a fair job of recreating the 250 year history of Bayes’ rule. Ms. McGrayne has a friendly writing style. She explains quite nicely why Bayes’ rule should be called Bayes-Price-Laplace rule, if not Laplace’s rule. Thomas Bayes had discovered the rule, but the rule would have lost to obscurity had it not for Richard Price who publicized the rule after Bayes’ death. On the other hand, Pierre-Simon Laplace had not only independently discovered the rule but created the mathematical construct that we associate with Bayes’ rule. One place the book fell short is exploring some depth of logic of Bayes’ rule and also pinning down the specific scientific reasons why Bayes or Laplace actually conceptualized the rule.
The Drunkard’s Walk – How Randomness Rules Our lives – by Leonard Mlodinow
YOU CANalytics Book Rating (4.8 / 5)
This is a terrific book to get a taste of the conceptual framework of probability and Bayes’ rule (even for math haters). The writing is brilliantly easy, and the stories are wonderful and baffling at the same time. If you want to learn how probability does not come naturally to humans then read this book, a must read in my opinion.
The Signal and The Noise – Why So Many Predictions Fail — but Some Don’t – by Nate Silver
YOU CANalytics Book Rating (4.5 / 5)
Applications: Economics, Politics, Climate, and Games
Nate Silver, the author of this book, has predicted 2012 US-elections with a perfect accuracy 50 for 50 states. He has done everything right but could have done this book slightly better. Not that it is not fun to read, on the contrary, it gets you excited about prediction and Bayesian statistics so much that you want to do it yourself. Good books are written by smart authors but great books make the readers feel smart. For this book to be exceptional Mr. Silver could have revealed some basic tricks of his trade to his audience such as the creation of a simple election-result-prediction model that he has mastered. I hope he does that in his next book. Nevertheless, The Signal and The Noise is a must read.
Thinking, Fast and Slow – by Daniel Kahneman
YOU CANalytics Book Rating (5 / 5)
Applications: Psychology, and Sociology
Daniel Kahneman is a Noble Laureate. This book is a culmination of research and experiments he has done along with Amos Tversky. The brilliance of the book is in learning extraordinary results from simple experiments. The perfect rating for this book is for the original ideas, brilliant & friendly narration, and making the reader feel smart – read this without fail. Also, buy a physical copy, not the Kindle edition.
Math on Trial – How Numbers Get Used and Abused in the Courtroom – by Leila Schneps and Coralie Colmez
YOU CANalytics Book Rating (4.8 / 5)
Applications: Criminal Trials, and Law
If you like courtroom dramas and probability then this is the book for you. I read this book over the stretch of a few hours on my Kindle. The authors have packed one fascinating real life story after another in this brilliant book. The stories are arranged not in chronological order but in the incremental order of hardness of mathematical concepts. Most of the stories are about misrepresentation of information through dubious maths. The problem was not with mathematics but the lack of logical rigor. In the day and age of DNA fingerprinting, one cannot ignore usage of probability for courtroom trials. Hence, there is an imminent need to educate the general public about the logical construct of probability and Bayesian statistics. Read this book, you will enjoy it.
Reckoning with Risk – Learning to Live with Uncertainty – by Gerd Gigerenzer
YOU CANalytics Book Rating (4.2 / 5)
Applications: Medicine, Diagnostics, and DNA fingerprinting
This book is full of stories where doctors make mistakes while estimating the probability of disease after positive tests (HIV, cancer etc.) – and cause unnecessary confusion and anxiety for the patient. Gerd Gigerenzer presents an effective solution to this by representing probabilities in terms of natural frequency i.e (100 patient out of 1000) rather than 0.1 probability. He observed in his research that doctors and the general public understand and interpret frequencies much better than probabilities. A good case for medical industry to deal in frequencies. The writing is fun and the book is good.
Proving History – Bayes’s Theorem and the Quest for the Historical Jesus – by Richard C. Carrier
YOU CANalytics Book Rating (4 / 5)
Applications: Historic Investigation, and Religion
You may want to stay away from this book if you belong to the following two classes of people
(i) You don’t want to question gods and religion (this book is questioning existence of Jesus and Christianity)
(ii) You are indifferent to religion and religious history i.e. they don’t excite you much.
I partially belong to the second class of people but still I am enjoying this book (have not completed the book just yet). The author is a well-trained researcher and hence employs scholastic logic while arguing his points. Having said this, the tone of the book is fairly lively and the narration fun to read. Read this book with an open mind.
In this part of the article, we have discussed the books to get you excited about probability and Bayesian statistics. However, the above set of books is certainly not a complete list. I would love to hear about your favorite books on the topic.
In the second part of this article, we will get a bit serious and learn about the nuts and bolts of doing Bayesian statistics. I’ll divide the next set of books into two parts i.e.
1) The Theory
2) The Practice
See you soon.
Brilliant Roopam! Its always a pleasure reading your blog posts! I never miss even one. 😀
Thanks Roopam.. that is helpful introduction to these books with your opinion on them.
Very nice reference for later.
If I could suggest a book, check out http://www.amazon.com/Data-Science-Business-data-analytic-thinking/dp/1449361323
It’s used as a Masters level course book at NYU for data science.
For Application section….would also recommend Naseem Taleb’s books…starting with Fooled by Randomness and Black Swan to the latest Antifragile. its mostly driven by his experience with financial markets and related risks.Less on maths and more on concepts..though his website has all the stats material he uses. He makes no qualms of taking on the Bschools for preaching the erroneous portfolio risk theories…given he predicted crash of 2008 and made a killing…I would listen.
Swapnil, I have read both of the books you mentioned, but I beg to differ. Taleb’s writing style is pretty dry (Fooled by Randomness is a tad better than Black Swan which I thought in all honesty was a better book). I am an Engineer, but it’s sometimes hard to relate to his concepts, because he does assume that the reader has a fairly decent understanding of Math other than the spelling obviously and I don’t like that in an author – if I am to recommend it to any lay reader. I hope the above list isn’t anything like that – since I am yet to order these and start them. I don’t really know when, but I hope to get on this bucket list soon.
Hello Roopam but when is the second part coming.
Awesome blog. I enjoyed reading it! Thank you!