| Week |
Monday |
Wednesday |
|
| 1 |
9/1 Introduction to class |
9/3 Probability and Random Variables |
|
| 2 |
9/8 Monte Hall
Lecture Note (PDF)
Summary (PDF)
Video (Link) |
9/10 Information/Horse Race |
HW#1 |
| 3 |
9/15 Entropy
(Makeup) |
9/17 Mutual Information
(Zoom Video) |
|
| 4 |
9/22
Video links:
Jensen’s Inequality,
Relative Entropy part 1,
Relative Entropy part 2 and Mutual Information,
Mutual Information,
Conditioning reduces entropy
|
9/24
Video links:
Concavity of log,
Concavity of Entropy : other approach,
Zero MI and Independence,
Markov chain and Data Processing Inequality,
State Markov Chain 2nd Thermodynamics
(No class — watch videos) |
HW#2 (Homework Collection File)
Due 10/1 before Quiz#1
Grading due 10/13
Chapter2 – 3 with HW2 and HW3
Jensen Inequality
Hash Algorithm과 PoW 성공 확률
Independance – Sufficient Statistics
|
| 5 |
9/29
Applications to Transformer
Video links:
Sufficient Statistics: Example and Proof 1,
Sufficient Statistic Proof 2,
One more example on Sufficient Statistics,
Fano’s Inequality
Problem 2.23 Cover on Thomas Part 1,
Problem 2.23 Cover on Thomas Part 2,
Statistics Sufficient part 1,
Statistics Sufficient part 2
|
10/1
(Quiz #1)
Video links:
Fano’s Inequality Proof,
Types of Convergences,
Relationship between different types of Convergences,
Law of Large Numbers & Surface Hardening
Review of Types of convergences,
Asymptotic Equipartition Property (AEP),
The size of the Typical Set,
|
HW#3
Due 10/15
Quiz #1
One more example on Sufficient Statistics
Typical Set.exel |
| 6 |
10/6 Chu-Suk Holiday |
10/8 Chu-Suk Holiday |
|
| 7 |
10/13
Video links:
High Probability Set vs. Typical Set,
Entropy Rate & Shannon’ s English Description,
Cesaro Mean
|
10/15
Video links:
Data Compression,
Non Singular Code & Uniquely Decodable Code,
Prefix Code,
Kraft Inequality
|
HW#4
Due 10/23
5:00PM |
| 8 |
10/20
Video links:
Optimal Codes,
Huffman Code,
Huffman Codes Vs Shannon Codes,
Lempel-Ziv Code
|
10/23 Review for Midterm
|
|
| 9 |
10/27 Midterm
|
Midterm Week |
| 10 |
11/3
No class. Please sutdy the note and videos
Video links:
Communications over Channels
Introduction to Channel Capacity
|
11/5
Video links:
Hamming Codes Encoding and Decoding
Erasure error correction
|
HW#5 Due 11/17
Post-midterm, we will use the homework posted here, rather than the homework collection.
Channel-capacity and HW5
Hamming Codes
|
| 11 |
11/10
Video links:
Channel Capacity Definition,
Noisy Typewritter
|
11/12
Video links:
Cap of BSC,
Cap of BEC plus 1,
Cap of BEC plus 2,
Weakly Symmetric channels,
Symmetric channels
|
HW#6
due 11/24
Information Bit Error Rate for (7,4) Hamming code with HW6
|
| 12 |
11/17
(GIST Founding Day)
No class. Please study the video and note.
Video links:
Channel Coding Theorem Proof,
Properties of Channel Capacity,
Information bit error rate 1,
Information bit error rate 2,
Information bit error rate 3
|
11/19
Video links:
Channel Coding Theorem Set up,
Channel Coding Theorem Key Ideas,
Channel Coding Theorem Forward Proof
|
Proof-of-Channel-Capacity-Theorem-HW7
|
| 13 |
11/24
Quiz #2
Video links:
Converse Proof Zero Error Capacity
Converse Proof using Fano’s Inequality
|
11/26
Video links:
Gaussian Channel Capacity – Differential Entropy1,
Gaussian Channel Capacity – Differential Entropy2,
Gaussian Channel Motivation,
The additive white Gaussian Channel
|
HW#7
due 12/8
Gaussian Channel Capacity |
| 14 |
12/1
Video links:
Gaussian Channel Capacity
Gaussian Coding Theorem Sphere Packing
Gaussian Coding Theorem Forward Backward
|
12/3
Video links:
Parallel Gaussian Channels,
PGC with colored noise
|
HW#8
Do not submit
Parallel Gaussian Channel Capacity HW8
|
| 15 |
12/8
Multiple Access Channels, Slepian-Wolf Coding |
12/10
Final Preparation |
|
| 16 |
12/15 Final Exam
|
Final Week |
| |
|
|
|
