Differential Entropy

This originally appeared on my tech blog. It’s primarily notes on the subtleties of differential entropy, but also contains a review of discrete entropy, various entropy-related information quantities such as mutual information, and a listing of various axiomatic formulations.

Read the Notes (PDF)