04 Maximum Likelihood estimation - Frequentist Framework

Maximum Likelihood (ML) estimation is a method for estimating the parameter $\theta$ of a statistical model that makes the observed data $D$ as likely as possible, given the model. The idea is simple: you want to find the value of $\theta$ that maximises the likelihood of the observed data $D$.

$${\theta }_{ML}=\arg \underset{\theta }{\max }p(D|\theta )$$

• ML can overfit with small datasets; MAP incorporates priors to mitigate this.
• Both ML and MAP use optimisation to find their best parameter values. ML has no notion of probability distribution over parameter values - all uncertainty in parameters are due to randomness in the data - the frequentist perspective.