https://www.xiaohongshu.com/explore/69e9e40a0000000020013000?app_platform=android&ignoreEngage=true&app_version=9.27.1&share_from_user_hidden=true&xsec_source=app_share&type=normal&xsec_token=CBONabWLySKwT5N3Z6fa4y4icPr28W6h-pqwTTL5oZmlo%3D&author_share=1&shareRedId=ODlGMTZKO0E2NzUyOTgwNjg5OTg1NUlA&apptime=1777370922&share_id=9c576526fcf543c7bc4c49dd8ac42811&share_channel=copy_link&appuid=63d13e7800000000260110c7&xhsshare=CopyLink

---

Problem 3

Consider a symmetric binary channel $\Gamma$, where the input symbols are $A = \{0, 1\}$, and the output symbols are $B = \{0, 1\}$. We denote the probability distribution of $A$ as $P_A$, and the probability of event $a$, which belongs to $A$, as $P_A(a)$. Suppose that $P_A(0) = p$, $P_A(1) = 1 - p$, and the probability of a symbol swap is $q$ whichever symbol is inputted to channel $\Gamma$. Answer the following questions. In this problem, the base of logarithm is 2.

(1) Draw a channel diagram of channel $\Gamma$.

(2) Describe the probability distribution of $B$, $P_B$, using $p$ and $q$.

(3) Describe the entropy $H(B)$ and the conditional entropy $H(A|B)$ using $p$ and $q$.

(4) Calculate $H(A|B)$ and the mutual information $I(A; B)$ for each of the two cases: ($p = 0.25, q = 0$) and ($p = 0.25, q = 0.25$). Also, explain the meaning of $I(A; B)$ in this channel comparing the $I(A; B)$ values in both cases.

---

Suppose that memoryless channels $\Gamma_1$ and $\Gamma_2$ are cascade-connected. In channel $\Gamma_1$, the input symbols are $X = \{x_1, x_2\}$ and the output symbols are $Y = \{y_1, y_2\}$. In channel $\Gamma_2$, the input symbols are $Y$ and the output symbols are $Z = \{z_1, z_2\}$. We denote the simultaneous probability of $x$ and $y$ as $P(x, y)$, and the conditional probability of $x$ given $y$ as $P(x|y)$. Answer the following questions.

(5) Prove that Eq. (i) is true on channel $\Gamma_1$ and channel $\Gamma_2$.

$H(X|Z) - H(X|Y) = \sum_{i=1}^{2} \sum_{j=1}^{2} \left[ P(y_i, z_j) \sum_{k=1}^{2} P(x_k|y_i) \{ \log P(x_k|y_i) - \log P(x_k|z_j) \} \right] \quad \text{(i)}$

(6) Prove that $H(X|Z) \ge H(X|Y)$ is true. Based on this, explain the meaning of the mutual information $I(X; Z)$.

(7) Suppose that the probability distribution of $X$ as $P_X(x_1) = s$ and $P_X(x_2) = 1 - s$, and the probability of a symbol swap is $r$ on channel $\Gamma_1$ and channel $\Gamma_2$. The capacity in the entire channel is denoted as $C_{12}$. Obtain the values of $r$ and $C_{12}$ when $C_{12}$ is maximized.

答案