Probability and Statistics for Machine Learning and Data Science

This series of blog posts introduces probability and mathematical statistics. While I wrote these posts with a focus on machine learning and data science applications, they are kept sufficiently general for other readers.

Some familiarity with vectors and matrices, as well as differential and integral calculus, is necessary to fully understand all concepts. If these topics are new to you or you need a refresher, I suggest you check out my other series on linear algebra and calculus first.

Series Overview

Basic Statistics and Probability

1. Introduction to Probability and Random Variables
2. Probability Mass Function and Probability Density Function
3. Conditional Probability and Independence
4. The Law of Total Probability and Bayesian Inference
5. Variance and the Expected Value
6. Covariance and Correlation
7. Bernoulli Random Variables and the Binomial Distribution
8. Normal Distribution and Gaussian Random Variables
9. Maximum Likelihood Estimation
10. Multivariate Gaussian Distribution
11. Maximum Likelihood Estimation for Gaussian Distributions
12. The Law of Large Numbers
13. The Central Limit Theorem
14. Confidence Intervals and Z Score

Probability Distributions

1. Factorization Theorem and the Exponential Family
2. Poisson Distribution
3. Exponential Distribution
4. Gamma Distribution
5. Beta Distribution
6. Conjugate Priors
7. Geometric Distribution
8. Chi-Square Distribution and Degrees of Freedom
9. Student’s T-Distribution

Statistical Testing

1. Hypothesis Testing and P-Values
2. Chi-Square Test for Independence and Goodness of Fit

Critical Value Tables

For your convenience, here are the most important tables for looking up critical values.

1. Z-Table
2. T-Table
3. Chi-Square Table

Further Resources

For writing these posts I’ve relied on several textbooks, online courses, and blogs.

Here you find a comprehensive list of resources that I’ve used to write these posts and that I recommend for further study.