Statistical Machine Learning

Foundational probability theory and statistical concepts

1. 1. Probability spaces and random variables
2. 2. Distributions and limit theorems
3. 3. Estimation (MLE, MAP)
4. 4. Hypothesis tests
5. 5. Decision theory

Regression models and their extensions

1. 1. Linear regression
2. 2. Regularized regression
3. 3. Generalized linear models
4. 4. Model diagnostics and selection
5. 5. Multicollinearity and diagnostics

Bayesian inference and hierarchical models

1. 1. Bayes rule and conjugacy
2. 2. Priors and posteriors
3. 3. MCMC and variational inference
4. 4. Hierarchical models
5. 5. Bayesian model comparison

Kernel-based learning and nonparametric approaches

1. 1. k-nearest neighbors and kernel density estimation
2. 2. Support vector machines and kernel methods
3. 3. Gaussian processes
4. 4. Splines and basis expansions
5. 5. Model complexity control

Probabilistic graphical models and structured outputs

1. 1. Directed and undirected graphical models
2. 2. Inference methods
3. 3. Conditional random fields and hidden Markov models
4. 4. Latent variable models
5. 5. Structure learning

Temporal modeling and forecasting

1. 1. ARIMA and state-space models
2. 2. Kalman and particle filters
3. 3. Seasonality and regime switching
4. 4. Forecast evaluation
5. 5. Change-point detection

Optimization theory and algorithms

1. 1. Convexity and duality
2. 2. Gradient descent variants
3. 3. Stochastic optimization
4. 4. Constrained optimization
5. 5. Numerical stability

Statistical learning theory and generalization bounds

1. 1. Bias-variance tradeoff
2. 2. VC dimension and capacity
3. 3. Regularization theory
4. 4. Uniform convergence
5. 5. PAC-Bayes bounds

Statistical approaches to causal reasoning

1. 1. Directed acyclic graphs and d-separation
2. 2. Backdoor and frontdoor adjustment
3. 3. Instrumental variables
4. 4. Propensity scores
5. 5. Mediation analysis

Model evaluation and uncertainty quantification

1. 1. Resampling methods
2. 2. Predictive intervals
3. 3. Model averaging
4. 4. Residual diagnostics
5. 5. Robust statistics