Write down 3 questions you are confused/interested about before doing the exercises:

Write down 3 questions you are confused/interested about after doing the exercises:

Give a short answer to the following conceptual questions:

1. Back in 4.3 we saw that compression can happen by losing some information. Now we are doing compression without losing the ability to encode some symbols. Why is it that we have to lengthen some codes to shorten others, in other words have varied-length codes?
2. After transitioning from fixed-length block codes to varied-length symbol codes, what happens to the source coding theorem? Compare theorem 5.19 with theorem on page 81.
3. What is the relationship between Shannon entropy and the minimal expected length of the symbol code? Is satisfying the Kraft inequality sufficient for guaranteeing the existence of that lower bound?

Kraft Inequality:

• carry out the check on Kraft inequality. explain the purpose of why we want to know there is a prefix code having those lengths. then read the solution and compare your answer.

  

Huffman coding:

• Checking that Huffman code is achieving the optimal code length

  

• application of the Huffman method, after reading 5.5-5.6

  

  • write a python code for Huffman coding and compress the following message

    source alphabet: X = {0,1,2} probability mass distribution: p = (p_0, p_1, p_2) = (.2,.4,.4) sequence to encode: x1, …, x3 = 2 1 0