Self Study: Probability and Bayesian Statistics

Self Taught – by Roopam

An average 21-year-old with a college degree spends a little over 15000 hours in classroom lectures. This accounts for a roughly 8th portion of his / her active life on this planet. To me, this alone is a good enough reason to bring constant improvement in our education system. The idea is to make learning fun! Let me make my small contribution by suggesting a few strategies I feel might be useful while learning concepts in higher education. I am going to use these same strategies to recommend books to learn probability and Bayesian statistics.

Struggle of Learning

As I have mentioned before, my formal education is in physics. I hope you could appreciate that quantum mechanics or astrophysics can be really daunting to the young students of physics. Despite my love for physics, like most students, I struggled to grasp the concepts. My two biggest struggles were

1) I thought I was the only one struggling.

2) A major part of my training in physics was about learning highly complicated calculus without the knowledge of its applications in physics.

My Analysis of the Learning Struggle

I recently read a book by Simon Singh (yes I love his writing) called ‘Big Bang’. In this book he has presented the historical evolution of astrophysics. I wish I had read it while I was studying physics because I could see reasons for my struggles.

1) Even the greatest minds including Galileo, Newton and Einstein struggled with ideas – It is always good to learn about the struggle that the founding fathers / mothers of the theory have gone through before formulating the concepts. This is a similar struggle that a new learner faces and relates to.

2) The founders of the theory first looked at the big gigantic sky, got puzzled and asked questions. They did their calculus (a tool to answer the questions) much later. I don’t see a reason why the students should do it the other way round.

Using the above concepts, let me present two different styles of learning in the next segment using a pictorial representation.

Learning Styles

* This representation is partially inspire by betterexplained.com (a terrific site to learn mathematics)

Of the above two learning styles, I prefer the second one. Keeping this in mind, in this part of the article, let me suggest the big picture (popular science) books for you to learn probability and Bayesian Statistics. I have divided these books into two parts

1) Historic and conceptual perspective
2) Applications

In the next part I will suggest books that will provide you the tools and methods to implements Bayesian statistics.

1) Historic and Conceptual Perspective

The Theory That Would Not Die – How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy – by Sharon Bertsch McGrayne

YOU CANalytics Book Rating (3.7 / 5)

I had really high expectations from this book when I started reading it – and it fell a little short. The book did a fair job of recreating the 250 year history of Bayes’ rule. Ms. McGrayne has a friendly writing style. She explains quite nicely why Bayes’ rule should be called Bayes-Price-Laplace rule, if not Laplace’s rule. Thomas Bayes had discovered the rule, but the rule would have lost to obscurity had it not for Richard Price who publicized the rule after Bayes’ death. On the other hand, Pierre-Simon Laplace had not only independently discovered the rule but created the mathematical construct that we associate with Bayes’ rule. One place the book fell short is exploring some depth of logic of Bayes’ rule and also pinning down the specific scientific reasons why Bayes or Laplace actually conceptualized the rule.

The Drunkard’s Walk – How Randomness Rules Our lives – by Leonard Mlodinow

YOU CANalytics Book Rating (4.8 / 5)

This is a terrific book to get a taste of the conceptual framework of probability and Bayes’ rule (even for math haters). The writing is brilliantly easy, and the stories are wonderful and baffling at the same time. If you want to learn how probability does not come naturally to humans then read this book, a must read in my opinion.

2) Applications

The Signal and The Noise – Why So Many Predictions Fail — but Some Don’t – by Nate Silver

YOU CANalytics Book Rating (4.5 / 5)

Applications: Economics, Politics, Climate, and Games

Nate Silver, the author of this book, has predicted 2012 US-elections with a perfect accuracy 50 for 50 states. He has done everything right but could have done this book slightly better. Not that it is not fun to read, on the contrary, it gets you excited about prediction and Bayesian statistics so much that you want to do it yourself. Good books are written by smart authors but great books make the readers feel smart. For this book to be exceptional Mr. Silver could have revealed some basic tricks of his trade to his audience such as the creation of a simple election-result-prediction model that he has mastered. I hope he does that in his next book. Nevertheless, The Signal and The Noise is a must read.

Thinking, Fast and Slow – by Daniel Kahneman

YOU CANalytics Book Rating (5 / 5)

Applications: Psychology, and Sociology

Daniel Kahneman is a Noble Laureate. This book is a culmination of research and experiments he has done along with Amos Tversky. The brilliance of the book is in learning extraordinary results from simple experiments. The perfect rating for this book is for the original ideas, brilliant & friendly narration, and making the reader feel smart – read this without fail. Also, buy a physical copy, not the Kindle edition.

Math on Trial – How Numbers Get Used and Abused in the Courtroom – by Leila Schneps and Coralie Colmez

YOU CANalytics Book Rating (4.8 / 5)

Applications: Criminal Trials, and Law

If you like courtroom dramas and probability then this is the book for you. I read this book over the stretch of a few hours on my Kindle. The authors have packed one fascinating real life story after another in this brilliant book. The stories are arranged not in chronological order but in the incremental order of hardness of mathematical concepts. Most of the stories are about misrepresentation of information through dubious maths. The problem was not with mathematics but the lack of logical rigor. In the day and age of DNA fingerprinting, one cannot ignore usage of probability for courtroom trials. Hence, there is an imminent need to educate the general public about the logical construct of probability and Bayesian statistics. Read this book, you will enjoy it.

Reckoning with Risk – Learning to Live with Uncertainty – by Gerd Gigerenzer

YOU CANalytics Book Rating (4.2 / 5)

Applications: Medicine, Diagnostics, and DNA fingerprinting

This book is full of stories where doctors make mistakes while estimating the probability of disease after positive tests (HIV, cancer etc.) – and cause unnecessary confusion and anxiety for the patient. Gerd Gigerenzer presents an effective solution to this by representing probabilities in terms of natural frequency i.e (100 patient out of 1000) rather than 0.1 probability. He observed in his research that doctors and the general public understand and interpret frequencies much better than probabilities. A good case for medical industry to deal in frequencies. The writing is fun and the book is good.

Proving History – Bayes’s Theorem and the Quest for the Historical Jesus – by Richard C. Carrier

YOU CANalytics Book Rating (4 / 5)

Applications: Historic Investigation, and Religion

You may want to stay away from this book if you belong to the following two classes of people

(i) You don’t want to question gods and religion (this book is questioning existence of Jesus and Christianity)

(ii) You are indifferent to religion and religious history i.e. they don’t excite you much.

I partially belong to the second class of people but still I am enjoying this book (have not completed the book just yet). The author is a well-trained researcher and hence employs scholastic logic while arguing his points. Having said this, the tone of the book is fairly lively and the narration fun to read. Read this book with an open mind.

Sign-off Note

In this part of the article, we have discussed the books to get you excited about probability and Bayesian statistics. However, the above set of books is certainly not a complete list. I would love to hear about your favorite books on the topic.

In the second part of this article, we will get a bit serious and learn about the nuts and bolts of doing Bayesian statistics. I’ll divide the next set of books into two parts i.e.

1) The Theory

2) The Practice

See you soon.