The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.

• Claude Shannon , 1948

🍍 Information Theory

🍌 Entropy, Data Compression

• 🍌 “Optimal code”, Huffman coding.

🍉 Channel Capacity, Error Correction

• 🍉 Construct minimal error-correcting codes, cryptic transmissions.

Computers process information, but what is information?

Information is related to questions. Represents answers to those questions.

graph TD;
    ?-->Yes;
    ?-->No;
    Yes-->I;
    No-->O;

Shannon's Information

Reduction of uncertainty obtained by receiving a message.

Syntactic, but not semantic.

Entropy

The uncertainty of a situation of ignorance is measured by entropy.