Abstract

In Part I, we started with Shannon’s two-stage communication architecture. The role of the first stage (source coding) is to convert an information source (possibly of a different type) into a common currency of information, bits. The role of the second stage (channel coding) is to convert the bits into a signal so that it can be reliably transmitted over a channel to the receiver. We then studied the source coding theorem which quantifies the maximum compression rate of an information source. We also learned about how to construct optimal codes that achieve the limit. As an example of an optimal code, we investigated the Huffman code and exercised its Python implementation.