Thought experiment

1.

32 events all equally likely, all coded by a sequence of binary digits
How many digits do you need to code a long sequence of events (on average)?

2.

32 events but on 2 can happen( and these are equally likely)
How many digits on average to code long sequence of events.

3.

Messages $m_n \quad n=0,1,2,...$ each with prob $p(m_n)=(1-\alpha)\alpha^n$. How many bits on average here?
$c(m_0)=1\qquad\qquad\rightarrow l(c(m_0))=1$
$c(m_1)=01\qquad\qquad\qquad\qquad =2$
$c(m_2)=001\qquad\qquad\qquad\qquad =3$
etc

$$E(l)=\sum_{n=0}^{\infty}(1-\alpha)\alpha^n(n+1)=\frac{\alpha}{1-\alpha}+1$$

say $\alpha=1/2$

E(l)=2

Infinite set of nonzero prob messages, but average code length=2