Information Theory answers : How to quantify information ?
Self Information
To measure the information content of an event, Shannon proposed self-information based on 3 axioms:
- An event with 100% certainty is perfectly unsurprising and yields no information.
- The less probable an event is, the more surprising it is, so the more information it yields.
- If two independent events are measured separately, the total amount of information is the sum of the self-information’s of the individual events.