Information Theory Made Simple

Updated: May 12, 2019



This is really a brilliant article that should be read in full. Its critical for undertstanding information theory and why it matters in the Atheist debate and why the type of information in life only comes from intelligence.

This is a portion of the original article to read the rest click on the link below.

Claude Shannon’s 1948 paper “A Mathematical Theory of Communication” is the paper that made the digital world we live in possible. Scientific American called it “The Magna Carta of the Information Age.”

Shannon defined modern digital communication and determined things like how much information can be transmitted over a telephone line, the effects of noise on the signal, and the measures you have to take to get a perfect signal on the other end. It made the Internet possible.

Trouble is, it’s tough reading – college level material for engineers and math geeks. HOWEVER Shannon’s concepts are simple and easy to explain. In just a few minutes you’ll understand Shannon’s concepts and you’ll see that any 7th grader can easily grasp them.

{Snip}

An encoder receives input and encodes a message according to the rules of the code.

The code is transmitted across a communications channel.

The code is decoded by the decoder, also according to a fixed set of rules and the message is understood on the other side.

2. What is a Code?

The dictionary defines code as “a system of symbols for communication.”

I define Information as: “Communication between an encoder and decoder using agreed-upon symbols.”

Here, we are interested in digital codes. The most important thing in the system is the code itself.

{Snip}

The letter A is not an electrical impulse, nor is it a magnetic field. The electrical impulse or magnetic field represents A. This is what is meant when we say “The message is independent of the medium it’s stored in.”

You can write the word “dog” with ink on a piece of paper but a dog is not ink or paper, and ink and paper are not a dog.

A long string of DNA like ACGGGTCTTTAAGATG——- that DNA pattern might build a claw, but that string of letters itself is not a claw and the claw is not a string of letters.

One of the most common questions I get about DNA and codes is:

“Why isn’t sunlight a code? Why isn’t radioactive decay a code? Why isn’t H2O a code?”

Sunlight is not a code because sunlight is just a stream of photons. There is no encoder in the sun. That photon does not symbolically represent some other thing. The sun does not send out digital streams of photons that obey the laws of a code.

The photon IS sunlight, it does not SAY sunlight. It does not give instructions for making sunlight. It doesn’t have any instructions at all. It’s just a photon. It represents nothing other than itself.

{Snip}

Also, there is no such thing anywhere in engineering or computer science as a percentage of the time that noise “accidentally” improves a signal.

Nor is there an “optimum” level of noise that you would want in a signal. The ideal amount of noise to have in a signal is ZERO.

Shannon pointed out that the best way to combat noise was through redundancy: Extra letters or numbers in the signal that help you fill in the blanks if there are missing letters.

For example “The quick brown fox jumps over the lazy dog” is still somewhat readable even if 1/3 of the letters are missing:

“Th q ic br wn fo jum s ove the l zy dog”

That’s because the English language is about 50% redundant. You can usually figure out what the original sentence was as long as at least half the letters are still there.

For the rest of the Article see the link below.

Original Article: https://evo2.org/information-theory-made-simple/

This article was given in compliance with fair use policies if you would like to comment on it on this site do so at our forum:

#ProofofGod #education #information #DNAcode

3 views0 comments

Recent Posts

See All