Mathematics · Advanced university
Entropy of a uniform discrete distribution
State the logarithm base; base 2 gives bits. The probabilities sum to one.
Method and conditions
State the logarithm base; base 2 gives bits. The probabilities sum to one.
H = −Σp log₂p = log₂m for m equiprobable outcomes
Worked examples
A discrete variable is uniformly distributed over 37 outcomes. Find its Shannon entropy in bits. Round the final result to 4 decimal places if needed. Answer in bits.
Each probability is 1/37. Hence H = −37(1/37)log₂(1/37) = log₂(37).
5.2095 bits
A discrete variable is uniformly distributed over 50 outcomes. Find its Shannon entropy in bits. Round the final result to 4 decimal places if needed. Answer in bits.
Each probability is 1/50. Hence H = −50(1/50)log₂(1/50) = log₂(50).
5.6439 bits
Try it
A discrete variable is uniformly distributed over 29 outcomes. Find its Shannon entropy in bits. Round the final result to 4 decimal places if needed. Answer in bits.
A discrete variable is uniformly distributed over 59 outcomes. Find its Shannon entropy in bits. Round the final result to 4 decimal places if needed. Answer in bits.
Check your work
Answers and marking notes
- 4.858 bits
Each probability is 1/29. Hence H = −29(1/29)log₂(1/29) = log₂(29).
- 5.8826 bits
Each probability is 1/59. Hence H = −59(1/59)log₂(1/59) = log₂(59).
Further reading and review references
These are reference links, not copied exercises, institutional endorsement or proof of independent review.
AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.