This series of blog posts introduces probability and mathematical statistics. While I wrote these posts with a focus on machine learning and data science applications, they are kept sufficiently general for other readers.
Some familiarity with vectors and matrices, as well as differential and integral calculus, is necessary to fully understand all concepts. If these topics are new to you or you need a refresher, I suggest you check out my other series on linear algebra and calculus first.
Basic Statistics and Probability
- Introduction to Probability and Random Variables
- Probability Mass Function and Probability Density Function
- Conditional Probability and Independence
- The Law of Total Probability and Bayesian Inference
- Variance and the Expected Value
- Covariance and Correlation
- Bernoulli Random Variables and the Binomial Distribution
- Normal Distribution and Gaussian Random Variables
- Maximum Likelihood Estimation
- Multivariate Gaussian Distribution
- Maximum Likelihood Estimation for Gaussian Distributions
- The Law of Large Numbers
- The Central Limit Theorem
- Confidence Intervals and Z Score
- Factorization Theorem and the Exponential Family
- Poisson Distribution
- Exponential Distribution
- Gamma Distribution
- Beta Distribution
- Conjugate Priors
- Geometric Distribution
- Chi-Square Distribution and Degrees of Freedom
- Student’s T-Distribution
- Hypothesis Testing and P-Values
- Chi-Square Test for Independence and Goodness of Fit
Critical Value Tables
For your convenience, here are the most important tables for looking up critical values.
- Chi-Square Table
For writing these posts I’ve relied on the following books:
Introduction to Mathematical Statistics by Hogg, McKean, and Craig
Pattern Recognition and Machine Learning by Christopher Bishop
The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani, et al.
Mathematics for Machine Learning by Deisenroth, Faisal and Ong
Deep Learning by Ian Goodfellow, Yoshua Bengio, et al.
Machine Learning: A Probabilistic Perspective by Kevin Murphy
I’m grateful to the authors for writing these amazing books.