Basic Model

Receiver wnats to know what sender sent
2 different processes here.

1.

sender generates "information"

2.

the block in figure transform it

Assume that sender can only send discrete messages m_n, n=1,2,......
Basic idea:less probable messages inherently carry more "information"
$X_i \in {a_j}$ source letters Prob(a_j)=p_j
If X_i indept. then source is memoryless.