Interview Question

RF Systems Engineer Interview

-Toronto, ON

Kepler Communications

what is the physics behind Shannon limit?

Answer

Interview Answers

2 Answers

0

There are no physics "behind" the Shannon Limit (or the Noisy Channel Coding Theorem). This theorem holds as a consequence of Fano's Inequality. It is a mathematical THEOREM that is necessarily true, regardless of the laws of nature. Fano's inequality is a consequence of the Gibbs Inequality, which is a mathematical theorem on the entropy of a discrete probability distribution. Now the Gibbs Inequality is used extensively in statistical physics (statistical mechanics and thermodynamics) to build theories on stochastic physical systems. To say that there are physics "behind" the Shannon Noisy Coding Theorem is wrong and confused, There are mathematics "behind" both the Shannon Limit and results in statistical physics, but the statistical physics results are in no way "behind" the results of Mathematical Information Theory. In fact, this truth is one of the most profound and celebrated aspects of Information Theory, namely that it is true in every possible world, not just in our own physical world. :-) There is an analogue to Fano's Inequality in physics, known as the Gibbs Inequality, which is used extensively in statistical physics. It's

Mohammad Tabari on

0

There are no physics "behind" the Shannon Limit (or the Noisy Channel Coding Theorem). This theorem holds as a consequence of Fano's Inequality. It is a mathematical THEOREM that is necessarily true, regardless of the laws of nature. Fano's inequality is a consequence of the Gibbs Inequality, which is a mathematical theorem on the entropy of a discrete probability distribution. Now the Gibbs Inequality is used extensively in statistical physics (statistical mechanics and thermodynamics) to build theories on stochastic physical systems. To say that there are physics "behind" the Shannon Noisy Coding Theorem is wrong and confused. There are mathematics "behind" both the Shannon Limit and several results in statistical physics, but the statistical physics results are in no way "behind" the Shannon Limit mathematical result. In fact, this truth is one of the most profound and celebrated aspects of Information Theory; namely that it is true in every possible world, not just in our own physical world. :-)

Mohammad Tabari on

Add Answers or Comments

To comment on this, Sign In or Sign Up.