25 Cross Entropy

For two discrete distributions $P$ and $Q$, cross entropy is defined as :

$$H(P,Q)=-\sum _{x\in {\mathcal{R}}_X}P(x)\log Q(x)=H(P)+D_{KL}(P\parallel Q)$$