Cross Entropy

The cross entropy of distributions $p$ and $q$ is
$$H(p,q)=E_p[-\log q]$$ It can be viewed as
$$H(p,q)=H(p)+D_{KL}(p\|q)$$ where $H(p)$ is the entropy of $p$ and $D_{KL}(p\|q)$ is the non-negative Kullback–Leibler divergence.

From an source coding perspective, it is the total bits required to encode information if the estimated distributed $q$ diverged from the true distribution $p$, where $H(p)$ is the minimum.

This quantity is very useful in machine learning.  Viewed from a vector quantization point of view, logistic regression is a way of finding an optimal boundary to classifying samples in a (possibly high dimensional) space of interest.   This expression quantifies the loss of estimating distribution $q$ instead of the true distribution $p$.  Since $H(p)$ is fixed because it is a property of the underlying true distribution, minimizing cross-entropy is equivalent to minimizing KL divergence in this setting.