10 Likelihood function

Let $X_1,X_2,\dots X_n$ (denoted compactly using a vector $\mathbf{X}=(X_1,X_2,\dots ,X_n)$) be and i.i.d sample from a distribution with a parameter $\theta$. Suppose we have observed that $X_1=x_1,X_2=x_2,\dots X_n=x_n$, denoted as $\mathbf{x}=(x_1,x_2,\dots ,x_n)$.

• If the distribution is discrete

$$L(\theta |\mathbf{x})=L(\theta |x_1,x_2,\dots ,x_n)=P_{\mathbf{X}}(\mathbf{x};\theta )$$

where $P_{\mathbf{X}}(\mathbf{x};\theta )$ is the PMF of $\mathbf{X}$ parameterized by $\theta$.

• If the distribution is continuous,
  

$$L(\theta |\mathbf{x})=L(\theta |x_1,x_2,\dots ,x_n)=f_{\mathbf{X}}(\mathbf{x};\theta )$$

where $f_{\mathbf{X}}(\mathbf{x};\theta )$ is the PDF of $\mathbf{X}$

Probability vs Likelihood

• Probability - Describes the chance of an event occurring, assuming a known distribution.
• Likelihood - Measures how well a statistical model fits the observed data. Likelihood function = likelihood as we vary the parameter(s) of the distribution.