Algorithm

1.

order the probabilities

2.

combine the 2 lowest to generate a secondary distribution

3.

prepare a 0 or 1 to the two corresponding codewords

4.

goto 1

HUFFMAN CODES=FUN
How can you construct a code for a given source distribution such that L is minimized?

Huffman algorithm (D-ary)

1.

order the probabilities

2.

combine the D lowest probs to generate a secondary distribution

3.

Prepared 0,1,...,D-1 to the two corresponding codewords as combined

4.

goto 1