(trumpet jazz music)
- [Andrew] The flaming trumpet.
Once when I was in junior high school,
preparing to perform in the school talent show,
my father asked me if I thought it would be a good effect
to have fire shooting out of the trumpet as I played.
(trumpet jazz music)
Of course, I had to admit
that that would indeed be very impressive.
He then went into his workshop and began tinkering around.
(trumpet jazz music)
And by the very next day, I found myself holding
in my hands a flaming trumpet. And by the very next day, I found myself holding
in my hands a flaming trumpet.
(cheerful piano music)
- Who's Claude Shannon?
- Claude Shannon?
- You're asking me this question?
- I often go back to this...
Well, you've probably heard the word bit. - I often go back to this...
Well, you've probably heard the word bit.
- We live in the information age Well, you've probably heard the word bit.
- We live in the information age
and he's the father of the information theory.
- Doesn't everybody know who Claude Shannon is?
- If you value the fact that your iPhone can contact - Doesn't everybody know who Claude Shannon is?
- If you value the fact that your iPhone can contact
and call for you an Uber, and then place a phone call,
and then text a friend, and then check your email,
all of those things you owe to Claude Shannon and his work.
- I truly believe he's at the same level
as Newton or Einstein. - I truly believe he's at the same level
as Newton or Einstein.
- Why don't people know about him?
They know about Alexander Graham Bell,
they know about Henry Ford, but so few people have heard They know about Alexander Graham Bell,
they know about Henry Ford, but so few people have heard
of Claude Shannon, and his impact is bigger than all
of the household names.
- I didn't know that that was one person,
and whoever that one person was
is probably the most important person in our time.
- What would the world look like
if Claude Shannon hadn't done what he did?
(piano key boings)
(gentle instrumental music)
- [Jimmy] By the 1980s, the revolution (gentle instrumental music)
- [Jimmy] By the 1980s, the revolution
that he started is in full swing, - [Jimmy] By the 1980s, the revolution
that he started is in full swing,
but he is nowhere to be found.
He's a founding father,
but he's a distant historical figure,
sort of like George Washington.
And nobody really knows what he's doing.
(gentle instrumental music)
(Claude clears throat)
- [Claude] Is this where you want me to sit?
- Yes, please.
Let's see...
- [Claude] Here we go.
- [Michelle] Thank you again for agreeing
to see me, Dr. Shannon. - [Michelle] Thank you again for agreeing
to see me, Dr. Shannon.
This is an amazing room.
- I guess it must seem pretty funny a grown man
has a room full of toys. (chuckles)
My life's work. (laughs) has a room full of toys. (chuckles)
My life's work. (laughs)
- [Michelle] I don't think you need to worry
about just being remembered for your toys.
- Claude doesn't overly worry about how he'll be remembered.
- No. - Claude doesn't overly worry about how he'll be remembered.
- No.
How'd you uncover me again, Michelle?
- [Michelle] As I said on the phone, I'm working
on a project about the founders of the modern age,
and I kept seeing over and over again,
"Claude Shannon, the father of Information Theory."
(Claude chuckles)
But nobody seemed to know what you're doing now.
Your 1948 paper was a blueprint
for what people are calling The Information Age.
- Well, I wrote that nearly 40 years ago. for what people are calling The Information Age.
- Well, I wrote that nearly 40 years ago.
- [Michelle] Yes.
In a sense you saw the future. - [Michelle] Yes.
In a sense you saw the future.
How did you do that?
(Betty chuckles)
(Claude chuckles) (Betty chuckles)
(Claude chuckles)
- How long do you plan on being here?
(Claude and Betty laugh) - How long do you plan on being here?
(Claude and Betty laugh)
- Maybe I should get some coffee.
- Oh, good.
(Claude and Betty laugh)
The fact is, Michelle, that my mind wanders around
and conceives of different things day and night.
Well, like a science fiction writer, I think,
"What if it were like this?"
And I never particularly cared
whether anybody was interested in it or wanted to work on it
or thought I was right.
(dramatic music) or thought I was right.
(dramatic music)
I was always interested in communication things, whether
or not people thought that's what I should be working on. I was always interested in communication things, whether
or not people thought that's what I should be working on.
(dramatic music) or not people thought that's what I should be working on.
(dramatic music)
(man shouts)
- [Andrea] Shannon thought about the problem
of communication in a way that nobody else
had ever thought of.
The basic problem is how do you get a message
from one place to another? The basic problem is how do you get a message
from one place to another?
The challenge was always how to go faster,
how to go farther, and how to be more accurate.
(horse neighs)
The huge advance was the discovery
of how to use electricity to communicate.
(phone ringing)
- [Woman] Hello?
Hello? - [Woman] Hello?
Hello?
- But initially it was very unreliable, Hello?
- But initially it was very unreliable,
and efforts to improve things were just haphazard.
And then, Shannon came along and asked a question
that hadn't even occurred to anyone,
"Could you construct a universal theory of communication
"that could allow perfect communication?"
He drew a very simple diagram.
You have some information you wanna send
and you have to convert it into something like sound waves
or electrical pulses or light that can be sent
over a channel or medium, like air or wires.
There's things that can disrupt the signal on that medium,
like static or smoke or noise There's things that can disrupt the signal on that medium,
like static or smoke or noise
before it gets to the receiver. like static or smoke or noise
before it gets to the receiver.
And then the receiver converts the signal back
into something that's hopefully really close And then the receiver converts the signal back
into something that's hopefully really close
to the original message.
The diagram looked trivial, but it allowed Shannon
to break the problem of communication down
into its essential parts and create a universal theory
that you can apply to anything you want.
- This is a calculator I built to do calculations
with Roman numerals.
I call it THROBAC:
Thrifty Roman Numeral Backward Looking Computer. I call it THROBAC:
Thrifty Roman Numeral Backward Looking Computer.
- [Michelle] Did you make that here at MIT?
- No, I made it when I was at Bell Laboratories - [Michelle] Did you make that here at MIT?
- No, I made it when I was at Bell Laboratories
when I should've been working on communication things.
- It didn't have a big commercial market.
(Claude chuckles)
- There are some important things
that made Claude Shannon the thinker
and the mind that he was. that made Claude Shannon the thinker
and the mind that he was.
I believe it was important that he grew up
in Gaylord, Michigan, I think it was important
that he came of age in the Midwest,
and I think it was equally important that he came of age that he came of age in the Midwest,
and I think it was equally important that he came of age
in a town that built things, where a boy like him could,
with his best friend, go to his friend's backyard
and build an elevator in a barn.
- [Claude] My mother was a principal
at the high school. - [Claude] My mother was a principal
at the high school.
My father had a bunch of jobs.
He had a furniture store.
He was a probate judge.
And he was also an undertaker.
He got me an erector set.
It's still one of my favorite toys.
My grandfather was a very interesting character,
he was always inventing things.
He was intent on determining if the turn
of the century was 1900 or 1901.
I was very impressed he spent so much time
on something most people thought was totally useless.
(cheerful pianola music)
- [Michelle] So, would you say your first experiment
in communication was a failure?
- [Claude] No, but there was a lot of noise
on the line. - [Claude] No, but there was a lot of noise
on the line.
- [Michelle] Motivation to come up with something better?
- [Claude] A little error could turn - [Michelle] Motivation to come up with something better?
- [Claude] A little error could turn
"Do you like me?" into "Do you hear me?"
And which she was answering was very important to me. "Do you like me?" into "Do you hear me?"
And which she was answering was very important to me.
- So, you used Morse code?
- Oh, you bet.
Morse already invented the binary code,
with dots and dashes, and he had the great idea of going Morse already invented the binary code,
with dots and dashes, and he had the great idea of going
to a print shop to find out which letters we use the most. with dots and dashes, and he had the great idea of going
to a print shop to find out which letters we use the most.
E, the most common letter, gets the shortest code, dot. to a print shop to find out which letters we use the most.
E, the most common letter, gets the shortest code, dot.
T, the second most common, gets the dash, and so on.
Exactly the sort of things I was thinking
about 100 years later.
- He had what we would probably call now
a free range childhood,
even though there wasn't really a term for it back a free range childhood,
even though there wasn't really a term for it back
in the 1910s and '20s; but these were things
like rigging up a barbed wire telegraph
to connect his house to his neighbors house like rigging up a barbed wire telegraph
to connect his house to his neighbors house
along the fences, things like learning wigwag signaling,
which is the idea of sending Morse code with flags
so that your body kind of becomes a sort which is the idea of sending Morse code with flags
so that your body kind of becomes a sort
of Morse code telegraph,
things like solving math puzzles.
Shannon's favorite story growing up
was Edgar Allan Poe's "The Goldbug."
(gentle clavichord music)
- Poe loved codes and cryptography, just like me.
He even invited his readers
to send him their own cryptograms that he'd solve. He even invited his readers
to send him their own cryptograms that he'd solve.
In "The Goldbug," a man finds a scrap of paper
with a coded message that tells where the pirate,
Captain Kidd, buried his treasure.
(gentle instrumental music)
The message is written in invisible ink though
and only appears when it's held up to a flame. The message is written in invisible ink though
and only appears when it's held up to a flame.
Poe shows how you can break the code
by connecting the most used symbols
to the most used letters in English.
(gentle instrumental music)
It's just like Morse did, in reverse.
Just like Alan Turing did, too, during the war,
looking for patterns to break the German codes. Just like Alan Turing did, too, during the war,
looking for patterns to break the German codes.
Finding patterns, that's the key.
- The foremost characteristic of my father was playfulness.
Play infused everything he did.
And you can see that in all the machines he made,
like, who makes a flaming trumpet?
Making a chair lift to go down to lake and come back,
that was a playful thing that he did with Marvin Minsky.
Nothing was sacred.
My brother, Andrew, had an idea for a rocket-powered Frisbee
and my father loved that. My brother, Andrew, had an idea for a rocket-powered Frisbee
and my father loved that.
(Frisbee buzzing)
If we had a tray of coffee cake, somebody,
if they were cutting a piece,
they usually wouldn't cut a piece off the end,
they would cut a piece out of the middle.
And they might cut it out in an interesting shape,
it wouldn't just be a square.
It might be an octagon, it might be a pentagon, it wouldn't just be a square.
It might be an octagon, it might be a pentagon,
something like that.
He tried to fool with everything a little bit
and see if he could make it more fun. He tried to fool with everything a little bit
and see if he could make it more fun.
- I was a big fan of "Alice in Wonderland"
and "Through the Looking-Glass."
You know, Lewis Carroll was a mathematician, too.
(rabbit sneezes)
Wonderland is full of word games and codes
and mathematical ideas.
"Through the Looking" is almost like a big chess game
with crazy rules of logic.
- [Woman] Stop, stop, stop, stop, stop, stop, stop.
- [Claude] Carroll liked
to invent funny word puzzles
that you could solve with mathematics.
He used something called Boolean algebra
to convert the words into equations.
The only numbers in Boolean algebra are zero
and one, nothing and everything.
And you could reach a conclusion by adding
and multiplying the equations. And you could reach a conclusion by adding
and multiplying the equations.
- [Michelle] So, what's the answer?
- Babies cannot manage crocodiles.
- He couldn't do mental arithmetic any better
than the average person. - He couldn't do mental arithmetic any better
than the average person.
And if he had two numbers to multiply, 26 times 42,
he'd go on the blackboard, 26, 42,
and do long multiplication, which was...
You'd think he'd just be, you know, a numbers genius.
No, he was much bigger than that.
- I got two Bachelor's degrees at Michigan.
One in electrical engineering, the other in mathematics.
I really wasn't sure what to do at all,
and I saw a little postcard on the wall
that said MIT was looking for a research assistant
to run a machine that could solve equations. that said MIT was looking for a research assistant
to run a machine that could solve equations.
They called it The Thinking Machine.
That sounded interesting.
Vannevar Bush (chuckles)
right over there, he invented it.
He was pretty famous in the world of engineering by then,
so I applied and spent the next four years at MIT.
(intense instrumental music)
- [Newscaster] The electromechanical Differential Analyzer (intense instrumental music)
- [Newscaster] The electromechanical Differential Analyzer
performs scientific and industrial marvels - [Newscaster] The electromechanical Differential Analyzer
performs scientific and industrial marvels
at the Massachusetts Institute of Technology.
Perforated tape introduces the data to the 100-ton marvel,
which solves problems Perforated tape introduces the data to the 100-ton marvel,
which solves problems
beyond the ordinary methods of calculation.
- The Thinking Machine,
which officially was called the Differential Analyzer, - The Thinking Machine,
which officially was called the Differential Analyzer,
was the smartest calculating machine of the time.
All sorts of gears and rotating shafts and wheels and so on,
it must've weighed 100 tons.
My job was to set the different wheels
and things to solve equations.
The heart of the machine though was electric switches,
just like the switches you find The heart of the machine though was electric switches,
just like the switches you find
in a telephone network to route calls.
It occurred to me,
because I'd taken the Boolean algebra course in Michigan, It occurred to me,
because I'd taken the Boolean algebra course in Michigan,
that there was a beautiful parallel
between electric switches, which are just on or off,
and Boolean algebra which reduced everything
to true and false.
I used one for true and zero for false.
I worked out the mathematics,
which had some funny things like,
if a true statement plus a true statement which had some funny things like,
if a true statement plus a true statement
equals true statement, that means one plus one equals one.
I wrote it all up for my Master's thesis
and that was the beginning of my great career. (laughs)
- So, his Master's thesis is considered
the most important Master's thesis of all time.
And he writes this at the age of 21. the most important Master's thesis of all time.
And he writes this at the age of 21.
He even wins a prize.
It's the Noble Prize.
Now, that's not Noble, it's N-O-B-L-E is the prize
that he wins.
It's very prestigious, it's given to the top engineer.
He's the youngest recipient ever, and he gets prize money,
a certificate, he's invited to give the paper He's the youngest recipient ever, and he gets prize money,
a certificate, he's invited to give the paper
to an audience of engineers in Washington D.C.
and, best part, he gets written up in the New York Times.
- Shannon's Master's thesis
is his first fundamental leap of insight. - Shannon's Master's thesis
is his first fundamental leap of insight.
George Boole came up with laws of thought,
and they were nice and beautiful,
but Shannon was looking at electromechanical switches,
and what is the connection between laws
of thought and switches?
(clinking)
Well, Shannon realized they are the same thing.
(clinking)
And that was just mind-boggling,
because if the laws of thought could be implemented
in switches then it's not outlandish because if the laws of thought could be implemented
in switches then it's not outlandish
to surmise that at some point machines could think.
(gentle instrumental music)
- No, no, not that way.
Back up, back up a little bit.
Theseus would've figured this out already. Back up, back up a little bit.
Theseus would've figured this out already.
- I've heard people say that your Master's thesis
was the most important Master's thesis of the century.
Did you have a eureka moment? was the most important Master's thesis of the century.
Did you have a eureka moment?
- (chuckles) I might have, if I'd known how to spell it.
I had a lot of fun figuring that thing out.
I had a great time.
(gentle instrumental music)
- I was going to Radcliffe, the girls' part of Harvard,
and I had two friends who said, "Oh, Norma, come with us.
"We know some nice guys over at MIT."
I said, "Okay," and we went.
And it was a living room with three bedrooms off it.
Now, two of the guys who lived there came out
and talked to my friends, and the door was open Now, two of the guys who lived there came out
and talked to my friends, and the door was open
to the third bedroom.
(cheerful jazz music)
And I could see someone in it not coming forth,
but what came forth from there was Bix Beiderbecke.
(cheerful jazz music)
So, I walked over that way
and said, "Why don't you come out?"
And he came just sort of halfway out.
(cheerful jazz music)
And he said, "Why don't you come in?"
(cheerful jazz music) And he said, "Why don't you come in?"
(cheerful jazz music)
He was so funny and charming, and delicious and cute,
and saw things, I mean, his comments
about things were like an artist.
He didn't sound like a scientist, not at all.
He liked poetry, and he liked E.E. Cummings.
He wrote a poem to my bosom and I tore it up.
Yes, I'm ashamed and unhappy that I did, but there you are.
(gentle guitar music)
He wanted to learn to fly, and he did.
He had just done his one plus one equals one,
it was his Master's thesis, and he had to prepare French
to get his PhD.
(gentle instrumental music)
And I had to teach him enough French to pass the doctorate.
(gentle instrumental music)
I love it that he wasn't that good at that.
(big band music) I love it that he wasn't that good at that.
(big band music)
- [Claude] "Dear Dr. Bush, I did not,
"as you may have anticipated, marry a lady scientist,
"but rather a writer.
"She was helping me with my French
"and apparently it ripened into something more than French."
(big band music)
- [Michelle] So, you turned to biology for your PhD?
(Claude laughs)
- Vann Bush.
Vann liked how I used my queer algebra so much,
that's what he called it, that he suggested I try
to apply it to genetics.
I didn't know anything about genetics.
This was in 1940, before Watson
and Crick had even figured out DNA structure.
But, you know, DNA might be
the ultimate information storage device.
So, I read all the introductory genetic text books
that I could find, and what I discovered was
that you could represent different genetic possibilities
mathematically in matrices.
And then you could just use matrix algebra
to make predictions of how traits were passed on
through generations. to make predictions of how traits were passed on
through generations.
But I don't think the biologists knew what to make of it.
Maybe it was too far out for them.
- He'd got a national research fellowship in mathematics
to the Institute for Advanced Study in June 1940,
and that's when we moved to Princeton.
(gentle instrumental music)
Claude liked my taste.
What he liked particularly was a painting that I loved,
the Seurat, Dimanche a la Grande Jatte, Sunday in the Park.
He liked the timelessness of it.
(intense instrumental music)
I looked at that a million times
and I didn't think it was a painting about time, but he did.
- I met Einstein, Gödel, von Neumann,
but you've got to get the picture right, Michelle,
I said I met them but they probably wouldn't say they met me
because they wouldn't remember.
But I remember being very impressed
when I met Einstein, for example.
He always walked to work, when I met Einstein, for example.
He always walked to work,
and when I drove by I'd wave at him.
I don't think he had any idea who I was,
but he probably thought I was some kind of weirdo.
- [Michelle] Did you try to talk to them about your ideas,
about information and...
Dr. Shannon?
- Oh, I...
I had a couple of good conversations with Hermann Weyl,
who was my supervisor, about the revolutions taking place I had a couple of good conversations with Hermann Weyl,
who was my supervisor, about the revolutions taking place
in science: quantum mechanics,
the Heisenberg uncertainty principle
saying things couldn't be measured precisely,
Kurt Gödel showing the limitations in mathematics.
But the people at the institute
were very pure mathematics-oriented
and not so interested in more practical things.
(gentle clarinet music)
I didn't run across anyone who was much interested
in the things I was trying to do.
- The last six months in Princeton,
something was going wrong.
When he came home from there, he was unhappy
and he didn't even wanna talk about it.
And it wasn't going well.
(gentle clarinet music)
He stayed home, he didn't wanna go out,
he talked less and less,
he played the clarinet more and more. he talked less and less,
he played the clarinet more and more.
- [Michelle] Was it possibly related to what was going on
in the world?
The build up to the war?
- [Norma] Not at all.
He didn't wanna know about anything political.
I thought he was just unhappy
with his work at the institute.
- [Michelle] Is that why he left the institute
after two months?
- [Norma] No. after two months?
- [Norma] No.
We got there in September - [Norma] No.
We got there in September
and I was living in Princeton through July. We got there in September
and I was living in Princeton through July.
- [Michelle] But he was only working
at the institute for two months.
- Are you telling me that when I thought he was
at the Institute for Advanced Study he wasn't there?
When I was with him?
You mean he resigned and he didn't tell me?
Was he going somewhere else?
What was he doing?
- [Michelle] He was working What was he doing?
- [Michelle] He was working
for the National Defense Research Committee
through Princeton University. for the National Defense Research Committee
through Princeton University.
- Well, that's interesting.
Listen, something funny was going on
because he looked different,
he was a completely different guy.
Something peculiar was happening.
It was hard living with him.
I said, "You know, I can't live this way,
"and I've got to get a divorce."
(gentle clarinet music)
- Uh.
About halfway through the year at Princeton,
I could smell the...
I could smell the war coming along.
It occurred to me that I would be more useful
and safer working full-time for the war effort.
Safer against the draft, which I didn't exactly fancy.
So, what I did was I resigned my fellowship halfway through,
and I went down and asked if I could get a job at Bell Labs.
(blues music)
♪ I went to the restaurant monday ♪
- He played the blues day and night.
Teddy Grace was the singer that he adored.
Oh, I loved her, too.
I still love her. Oh, I loved her, too.
I still love her.
Very sad blues that he played, but beautiful ones.
He had a very big record player.
And you could hear it through the ceiling. He had a very big record player.
And you could hear it through the ceiling.
He played it day and night, that's how I met him.
I said, you know, "Even though I'm up all night,
"a good part of the night, but you keep me up.
"I can't even study.
"It's too noisy."
He said, "Well, I'll move it."
And then he asked me would I go out with him.
He was very good-looking, really striking.
He had this black thing, you know, a curl
that came over his forehead always, He had this black thing, you know, a curl
that came over his forehead always,
that he had to shake off.
He loved jazz.
I mean, it was a passion with him, a real passion.
He said the improvisation, the unpredictability
of it is what he loved about it, he said.
When I asked him, "What are you doing?
"How are you making a living?"
He said, "I work with numbers."
And I thought, "Oh, he's an accountant."
You know, I think it was a suffering time.
I had German measles, and Claude was trying to get them.
He said if he got it then he wouldn't have to go to work,
because he hated the work he was doing.
- Initially they had me working on missile firing systems,
but there was another group at Bell working on systems
to scramble the communications
between Churchill and Roosevelt. to scramble the communications
between Churchill and Roosevelt.
That was the sort of thing
I was interested in, communications.
So, I asked if I could go work with them.
Most of the cryptography group were trying
to figure out how to crack specific messages,
very practical stuff, code breaking things,
like Sherlock Holmes and so forth.
But nobody seemed to be very interested like Sherlock Holmes and so forth.
But nobody seemed to be very interested
in theoretical questions. But nobody seemed to be very interested
in theoretical questions.
Like, how many messages you needed to intercept
before you could get a solution?
Or, could you create a completely unbreakable code?
That was the sort of thing I was more interested
in trying to do.
- [Maria] Norma had told him to get in analysis,
because he must've been very depressed already
when he was with her.
- [Claude] I didn't get as much response
to my cryptography work as I expected. - [Claude] I didn't get as much response
to my cryptography work as I expected.
I guess it wasn't very useful,
practically speaking, to cryptographers.
But after the war though, I was called in as a consultant practically speaking, to cryptographers.
But after the war though, I was called in as a consultant
to the National Security Agency in Washington. But after the war though, I was called in as a consultant
to the National Security Agency in Washington.
So, I guess someone...
I probably better...
Even though this was a long time ago,
I guess I shouldn't tell you about that.
- I asked him, "How is it going in analysis?"
He said, "The guy's finally learned something
"about mathematics and how to think properly." He said, "The guy's finally learned something
"about mathematics and how to think properly."
He said that what had hurt him so in life was
that there was certainty until a certain time,
that once he knew a thing to be true it would remain true,
and everything had changed.
Everything became sort of relative, he said.
He had believed in black and white truth, absolute truth.
And those had gone for him.
He had to adjust to that.
(gentle instrumental music)
"With love and kisses and curses and hisses. (gentle instrumental music)
"With love and kisses and curses and hisses.
"Claude, 1944."
That was a long time ago.
(gentle instrumental music)
It doesn't seem to be that long ago, really not.
I didn't know how much I cared for him, really.
I should've known.
That secret life, I think, really got between us.
- I found this 1945 cryptography report you wrote.
And it uses the words "information theory".
And I think that that's the first time that phrase appeared, And it uses the words "information theory".
And I think that that's the first time that phrase appeared,
those two words together in print, anywhere.
- Is that so?
- Yes.
See?
And I'm wondering, your monumental paper
on information theory didn't come out until 1948.
- Hmm.
- [Michelle] So is it accurate to say
that information theory came out of your cryptography work?
- No.
Look, Michelle, you're looking for a simple... - No.
Look, Michelle, you're looking for a simple...
It would be more accurate to say they were going on
at the same time. It would be more accurate to say they were going on
at the same time.
In my mind, they're connected.
How you make something unintelligible
or how you make it intelligible are two sides
of the same coin.
- [Michelle] But officially you were just working
on cryptography, right?
- [Claude] Are you looking to work on cryptography, right?
- [Claude] Are you looking to work
for the district attorney's office? (chuckles)
- [Michelle] No. for the district attorney's office? (chuckles)
- [Michelle] No.
No, I'm just--
- The cryptography work was just a way No, I'm just--
- The cryptography work was just a way
of legitimizing the other work I was doing
at home outside office hours, so to speak,
to make it look like I was working on decent,
worthwhile things, if you know what I mean.
- [Michelle] Did you feel like you had worthwhile things, if you know what I mean.
- [Michelle] Did you feel like you had
to hide the other work?
- Not hide exactly, but I didn't know to hide the other work?
- Not hide exactly, but I didn't know
if it would all work out.
- Shannon works on his universal theory
of communication for 10 years.
He works on it through a failed marriage
and through this war work that he really dislikes.
He works at nights, he works on the weekends,
he collects snatches of time here and there,
and he works alone. he collects snatches of time here and there,
and he works alone.
He doesn't tell anyone what he's working on. and he works alone.
He doesn't tell anyone what he's working on.
What he realizes is that in order
to understand communication, he needs a new understanding
of the thing you wanna communicate.
He needs a radical new definition of information.
- Information, which they called intelligence at the time,
was a slippery concept.
I wanted to try to find a way to treat information was a slippery concept.
I wanted to try to find a way to treat information
like a physical thing, like energy that you could measure.
(gentle instrumental music)
What I realized was, to measure information,
you have to look at it without regard to meaning.
That upsets a lot of people.
That content is irrelevant. That upsets a lot of people.
That content is irrelevant.
(intense instrumental music)
The point of sending a message is to remove uncertainty,
something you didn't know but now you know.
Do you have a coin, Michelle?
- A coin? - Yes, a coin.
I'd get up to get one myself,
but you'll just tell me to sit back down. (chuckles)
- [Michelle] Um...
Yes.
- There.
(Claude clears throat)
Heads or tails?
- [Michelle] Tails.
- Heads.
A coin toss is an example of the simplest form
of information to communicate, a binary choice. A coin toss is an example of the simplest form
of information to communicate, a binary choice.
True or false, heads or tails, a one or a zero.
A bit.
(gentle instrumental music)
- Shannon's big idea number one:
convert all information to bits.
(coin clinking)
(gentle instrumental music)
(glass crashing)
(gentle instrumental music)
All information can be represented as zeros and ones.
Pictures, music, (upbeat music)
video, all information can be treated the same way.
(gentle instrumental music)
Which means the problem of communication reduces
to just two things: how do I convert things
to zeros and ones,
and how do I make sure that those zeros
and ones get through accurately?
- Is your coin legal?
- [Michelle] Sorry? - Is your coin legal?
- [Michelle] Sorry?
- Your coin's not counterfeit, is it?
- [Michelle] Not that I know of.
- So, if I toss it 100 times, - [Michelle] Not that I know of.
- So, if I toss it 100 times,
each time I toss it there's an equal probability
that it's a head or tail.
I'd need 100 bits of information, one for each toss,
to communicate the results.
Now, if it turns out you're some sort of a con woman to communicate the results.
Now, if it turns out you're some sort of a con woman
and you thought it might be fun to pull the wool
over some gullible old man's eyes...
Heads, right?
- [Michelle] It's heads on both sides.
- Yeah, I didn't need any information from you
to know the result.
If something's completely predictable,
there's zero information. If something's completely predictable,
there's zero information.
But, on the other hand, if something's completely random,
you need the information about everything.
It's counterintuitive,
but gibberish contains more information It's counterintuitive,
but gibberish contains more information
than great literature.
- Not if it's "Finnegans Wake."
(Claude laughs) - Not if it's "Finnegans Wake."
(Claude laughs)
- No, James Joyce is unpredictable. (chuckles)
- Or the Jabberwocky poem,
"Twas brillig, and the slithy toves.
"Did gyre and gimble in the wabe."
(Claude laughs)
- You could barely guess the next letter of that.
But normal English though, I calculated,
is over 50% redundant.
You know if certain words follow each other,
there's grammar rules. You know if certain words follow each other,
there's grammar rules.
When you learn a language,
you inherently know the statistics,
that's why you can drop letters and even words
and still understand the message.
- Looking at the redundancy
in English gives Shannon his big thought number two:
compress your information.
(gentle instrumental music)
Eliminate the redundancy.
Just send what you can't predict.
(gentle instrumental music)
And then he asked the question
that nobody had really thought of before,
is there a minimum size, a minimum number of bits
that I can shrink my information to
and not lose anything essential?
(playful music)
He discovers there is a minimum
and he shows how to calculate it.
His formula is based on the probabilities in the message.
It has a very intriguing form, it's almost identical
to a fundamental quantity in physics called entropy,
which is related to randomness and disorder.
(intense instrumental music)
Shannon calls his quantity entropy, too.
And what he proves is that the entropy
is actually the answer to his question, And what he proves is that the entropy
is actually the answer to his question,
"What's the best I can do?"
The entropy is the absolute minimum size
you can compress your message without any loss.
(intense orchestral music)
Shannon's ideas that you can compress a huge amount
of information into something very small
is what makes The Information Age possible. of information into something very small
is what makes The Information Age possible.
- Do you know what the first message was that was sent
on the Washington-Moscow hotline? - Do you know what the first message was that was sent
on the Washington-Moscow hotline?
- [Michelle] Uh... on the Washington-Moscow hotline?
- [Michelle] Uh...
"Do you hear me?"
(Claude chuckles) "Do you hear me?"
(Claude chuckles)
- "The quick brown fox jumped over the lazy dog's back."
One, two, three, four, five, six, seven, eight, nine, zero.
It uses every letter of the alphabet,
but it isn't the most efficient
because some of the letters are repeated. but it isn't the most efficient
because some of the letters are repeated.
Betty and I had a grand time trying to come up
with the shortest sentence that uses every letter only once. Betty and I had a grand time trying to come up
with the shortest sentence that uses every letter only once.
- We spent a lot of time looking in the dictionary
for words heavy on consonants and light on vowels.
- The hard part is that there's only a few vowels
and every word needs to have one.
Squdgy Fez Blank Jimp Crwth Vox.
(intense orchestral music)
- Shannon's ideas that you should first change everything
to zeros and ones and then compress it is a huge step
towards the universal theory of communication,
but it's not the end of the story,
because you still have to get those zeros
and ones across your channel, because you still have to get those zeros
and ones across your channel,
and the channel has physical limitations,
whether it's a wire or a pipe with water flowing through it. and the channel has physical limitations,
whether it's a wire or a pipe with water flowing through it.
And then, there's always noise, (static)
things like static or interference,
that can mix up or flip the zeros and ones and wipe you out.
Shannon wonders, "Can I calculate for any channel
"a speed limit beyond which you know
"things are gonna fall apart?" "a speed limit beyond which you know
"things are gonna fall apart?"
He comes up with another formula,
it's a theoretical maximum speed,
an information speed limit, like the speed of light.
We call it the Shannon limit.
You can't go faster than the Shannon limit.
If you try, you go off a cliff.
- The idea that there was a speed limit
to information was so out of the blue.
I mean, even just conceptually this was a shock.
It was like when Einstein said
that you couldn't make things go faster
than the speed of light.
But what Shannon showed is if you tried
to get past this limit, above the Shannon limit,
you cannot recover any useful information.
(static)
- Shannon's third big idea, huge idea:
use mathematics to eliminate noise.
This seemed crazy, what does that even mean? use mathematics to eliminate noise.
This seemed crazy, what does that even mean?
He proved that mathematics could eliminate errors
from static and interference.
You could use mathematical codes to overcome noise.
And then, as the icing on the cake,
Shannon claimed this could work right up
to the Shannon limit.
If you could find the right codes,
you could have perfect communication with as little errors
as you wanted, and that was the revolution.
(alarm-clock ringing)
(wrist-watch bleeping)
(telephone ringing) (wrist-watch bleeping)
(telephone ringing)
(cell phone ringing)
(intense orchestral music)
(satellite bleeping)
- Suppose you wanted to use Shannon's ideas
to help with one of the most difficult known problems
in communication: communicating with a spouse.
- One communication scenario I can think of
applying Shannon's ideas to is communication in a marriage, - One communication scenario I can think of
applying Shannon's ideas to is communication in a marriage,
which, as we all know, can be really difficult. applying Shannon's ideas to is communication in a marriage,
which, as we all know, can be really difficult.
- This communication is often prone to serious errors, which, as we all know, can be really difficult.
- This communication is often prone to serious errors,
which sometimes lead to disasters.
- There's all this noise, there's interference, which sometimes lead to disasters.
- There's all this noise, there's interference,
maybe they had a bad day or a good day,
you know, maybe the kids are shouting in the background
or the dog's barking.
And so, how do you convey exactly what you wanna say or the dog's barking.
And so, how do you convey exactly what you wanna say
to your spouse in a way that they understand?
- One way of communicating is repeating your message over
and over again. - One way of communicating is repeating your message over
and over again.
- And that probably will eventually get through,
but your partner may get annoyed with you.
- And another way is to shout louder.
- But that could also be misconstrued.
- So, what Shannon really said is - But that could also be misconstrued.
- So, what Shannon really said is
that nagging, you know, doesn't really work,
or speaking louder doesn't work either.
- It's how you pick your words. or speaking louder doesn't work either.
- It's how you pick your words.
- So, we should say things perhaps many times
but in different fashions, in different presentations.
We should code them well, but in different fashions, in different presentations.
We should code them well,
and at the end of our communication We should code them well,
and at the end of our communication
we should always add an error-correcting code,
which in this case could be something like,
"But I love you."
(bomb explodes)
- When Shannon's paper came out, it was like a bombshell. (bomb explodes)
- When Shannon's paper came out, it was like a bombshell.
To be able to communicate with as few errors as you wanted,
people couldn't believe it.
Some people couldn't understand the math.
Engineers thought it was so far removed
from anything they could build, Engineers thought it was so far removed
from anything they could build,
they just basically thought it was science fiction.
- But there were some people who became big champions
and saw the potential in what Shannon had done. - But there were some people who became big champions
and saw the potential in what Shannon had done.
They became totally enamored of this new field, information.
So, he's placed on a list of the leading scientists
of the day, alongside Richard Feynman and Watson and Crick. So, he's placed on a list of the leading scientists
of the day, alongside Richard Feynman and Watson and Crick.
He's given write ups in Time Magazine and Fortune Magazine
and Scientific American.
And he's even given a spread in Vogue Magazine and Scientific American.
And he's even given a spread in Vogue Magazine
with a photo shoot from Henri Cartier-Bresson,
the renowned photographer,
whose other photo shoots included Mahatma Gandhi,
Chairman Mao's funeral, Queen Elizabeth's coronation,
and Claude Shannon at his home in Massachusetts.
- When Shannon's paper came out in 1948,
people embraced it far beyond what Shannon ever envisioned
it would be embraced as,
which was a mathematical theory of information.
So, it started being applied to philosophy
and pretty much almost every field embraced it, So, it started being applied to philosophy
and pretty much almost every field embraced it,
"Oh, we're gonna use this mathematical theory
"of information in what we do."
And it got a little bit out of control,
and it was being applied in ways that didn't make any sense.
- Just a bit more.
- I found this wonderful T-shirt
(Claude laughs)
in the London airport.
(Michelle chuckles)
- [Michelle] Were you famous when your paper came out?
- [Claude] Not too much.
- [Betty] He wrote a paper telling them
to get off the bandwagon.
- I was worried that there was still a lot
of hard slugging left to do in our own professional house.
- [Michelle] I read that a major American mathematician
criticized your paper
for not being mathematically rigorous enough.
- Maybe it needed people a little brighter to understand it.
- Russian mathematicians got it right away.
- [Claude] Yes, they did.
- But the codes that you said had to exist,
you didn't show how to get them.
And now it's 35 or so years later and all your ideas
about converting everything to digital, compressing audio,
those things are all happening,
but as far as your fundamental theorem, nobody's been able
to find codes that get anywhere near the Shannon limit.
Do you think that...
Are you worried that you'll be proven right?
- I knew what I was doing, and it came out exactly right.
Some people question whether the technology
will ever be powerful enough to implement it, but
there's a lot of bright young people trying
to find those codes.
(roulette rattling)
- The first few times I met him,
he was giving a lecture or something.
I was very young then, very shy, very immature,
afraid to go up and talk to him.
I was working on a problem
that I thought was very important.
It was an information theoretic problem. that I thought was very important.
It was an information theoretic problem.
I knew he would be interested in it also.
I started to talk to him about it.
And suddenly, the problem seemed too complicated for him,
which surprised me, but he said,
"Let's simplify it a little bit."
So we took one piece out, then we took another piece out,
and another piece out,
suddenly it was a silly little problem
and the answer became obvious.
And then we slowly put the pieces back in
and the problem was solved.
How do you do that?
How do you take pieces out in the right way
so that when you get done taking them out
you can solve the problem?
He would build these little experiments
and his little experiments would give him some insights
about whether the general problem could be solved.
- Hello.
I'm Claude Shannon, a mathematician here
at the Bell Telephone Laboratories.
This is Theseus.
Theseus is an electrically-controlled mouse. This is Theseus.
Theseus is an electrically-controlled mouse.
He has the ability to solve a certain class of problems
by trial and error means, and then remember the solution.
In other words, he can learn from experience.
Like his classical namesake, Theseus has a problem
with finding his way through a maze.
The mouse moves by this electromagnet.
And its brain are these switches,
just like the switches you'd find
in a telephone switch board.
Betty did most of the wiring, by the way.
I took it over to Bell Labs one day to show them. Betty did most of the wiring, by the way.
I took it over to Bell Labs one day to show them.
They liked it very much, they were very impressed.
They even wanted to use it in a film.
- When did you build this?
It must've been one
of the earliest artificial intelligence devices.
- Um... of the earliest artificial intelligence devices.
- Um...
Uh...
Betty!
When did we make Theseus?
- 1950. - Right. When did we make Theseus?
- 1950. - Right.
- He made a little turtle first that would walk
around the house and bump into things, and back off.
And then the mouse. around the house and bump into things, and back off.
And then the mouse.
- So, you said that you did most And then the mouse.
- So, you said that you did most
of your information theory work between 1943 and 1945,
so why didn't you publish your paper until 1948?
- Laziness, I guess.
I have a number of papers that are unpublished, - Laziness, I guess.
I have a number of papers that are unpublished,
they're even more tardy than that.
- Oh, we've got boxes and boxes
of unfinished papers upstairs.
Who knows what's in them?
Maybe we should punish you and make you go through them.
(Claude laughs)
- Well, I never liked writing very much.
I liked ideas, but I was lazy about putting them down
on paper, which is how you get the acclaim.
- Hmm.
After we were married, he would dictate his papers to me
and I would write them up.
- [Michelle] You were a mathematician
at Bell Labs, too, right?
- I married a computer.
- My mother was much, much better
at arithmetic than my father was.
She went to Douglas College, which is part
of Rutgers University,
which was a wonderful opportunity for her.
She was a math major, did very well, and graduated just
at the beginning of World War II
when everybody was looking for math majors.
And she had several job offers, which was exciting for her.
She accepted the one at Bell Labs to be,
what they called, a computer. She accepted the one at Bell Labs to be,
what they called, a computer.
Her term is, it was a secretarial pool for math majors.
So, it was a room full of women who had been math majors
who would then do the calculating
for the male mathematicians and engineers.
After Bell, my mother took up weaving,
which is very mathematical. After Bell, my mother took up weaving,
which is very mathematical.
She became an accomplished weaver which is very mathematical.
She became an accomplished weaver
and a pioneer in using computers in hand-weaving.
My father, of course, had to build her a loom.
My parents were a great partnership. My father, of course, had to build her a loom.
My parents were a great partnership.
They both appreciated each other's intelligence,
and together they were a great team.
- But I didn't meet Claude until after the 1948 paper,
so I couldn't make him write that one quicker.
(Claude laughs)
- You have to understand, too,
that during the war this wasn't a priority, this work.
And I didn't think in the early stages
it would have such an impact. And I didn't think in the early stages
it would have such an impact.
(intense instrumental music)
- One of the most beautiful things about Shannon's paper is (intense instrumental music)
- One of the most beautiful things about Shannon's paper is
that he reduced the general problem of communication - One of the most beautiful things about Shannon's paper is
that he reduced the general problem of communication
to just two numbers: the minimum size
you could compress a message, it's entropy, to just two numbers: the minimum size
you could compress a message, it's entropy,
and the maximum rate you could send it without errors,
the Shannon limit.
And it just started a race to find ways
to reach those limits,
and Bell Labs was at the center of that race.
It was the top research institute
in the world for communications.
They just invented the transistor right down the hall in the world for communications.
They just invented the transistor right down the hall
from Shannon's office in 1947,
the year before his paper came out. from Shannon's office in 1947,
the year before his paper came out.
They were geniuses across many fields,
but even there Shannon stood out.
(gentle instrumental music)
- I would call it the grand puzzle,
that's what I call it, the Shannon puzzle.
All of this great stuff where he showed
that communicating information is possible,
he proved it mathematically,
but he never told anyone how to actually do it.
So, that is the puzzle.
It's tantalizing.
You know that those codes exist,
but Shannon gave really almost no clue how to go You know that those codes exist,
but Shannon gave really almost no clue how to go
about finding them.
- Of course, that only means
that Theseus can solve any problem that can be solved. - Of course, that only means
that Theseus can solve any problem that can be solved.
Like the rest of us, he occasionally finds himself
in a situation something like this.
(intense instrumental music)
- [Michelle] Why did you decide to leave Bell Labs
and come to MIT?
(Claude sighs)
- They gave me a lot of freedom at Bell,
but I started to feel a little stale. - They gave me a lot of freedom at Bell,
but I started to feel a little stale.
I also always felt that the freedom
I took there was something of a special favor.
I thought at MIT...
Well, the freedom in the work hours that I took
might be a little less unusual.
(intense instrumental music)
- In the mid '50s, MIT is a powerhouse
in technology in general,
and especially in information theory.
They make a huge effort to recruit Shannon and they get him.
And people expect he's gonna be the leader
in delivering the promise of his 1948 paper.
He's not.
He writes some important extensions of his work,
but he starts coming to campus less and less.
And, in fact, at a certain point
he's coming in so infrequently
they actually put two graduate students in his office.
And people are having trouble finding the codes he said had
to exist, so there begins to be some skepticism
about an information theory.
And everybody's wondering, "Where's Claude Shannon?"
(cheerful piano music)
- When I was quite young, he would dress up
and go to work the way other people did.
And then, at some point that diminished
and he was home more often.
At some point my mother explained
that he had taken a leave from MIT.
After he sort of moved on from the more theoretical
and academic work that he did,
and more into things like unicycling and juggling,
many people would consider that he wasn't working anymore,
but I still think of that as his work.
My dad got interested in building all kinds
of different unicycles, very small wheels,
very large wheels, very tall, short, with pedals,
without pedals, with seat, without seat.
And we tried to ride them all.
He had an interest in unicycles
because my mother bought him one,
not so easy to come by back in maybe 1950 or '51.
It was sort of a running joke when I was a child
about how if I wasn't doing too well in school
I could always run off and join the circus, about how if I wasn't doing too well in school
I could always run off and join the circus,
and my parents would love that.
(cheerful piano music)
Before I was born he knew how to juggle,
and he taught me how to juggle when I was maybe eight Before I was born he knew how to juggle,
and he taught me how to juggle when I was maybe eight
or nine years old. and he taught me how to juggle when I was maybe eight
or nine years old.
He also started trying to build machines to juggle.
- Juggling relates to patterns.
There's a topology and a combinatorial aspect.
I wrote a paper for Scientific American. There's a topology and a combinatorial aspect.
I wrote a paper for Scientific American.
(cheerful piano music) (toy rattling) I wrote a paper for Scientific American.
(cheerful piano music) (toy rattling)
Oh, come on, Mr. Fields.
(cheerful piano music) (toy rattling)
Ah.
I came up with a formula for juggling
that relates the number of balls you're juggling
to how long each is in the air versus in the hand. that relates the number of balls you're juggling
to how long each is in the air versus in the hand.
- [Michelle] Did you ever think
about commercializing your inventions?
- Oh, fiddle, we don't believe in commercializing fun.
(Claude chuckles)
- [Michelle] You didn't worry (Claude chuckles)
- [Michelle] You didn't worry
that they might not be practical?
- Well, that'd be the last thing. that they might not be practical?
- Well, that'd be the last thing.
I built a lot of useless gadgets
just because they're fun to make.
I'm more interested in, is this an interesting problem?
Is it an elegant problem?
I get a big bang out of proving a theory Is it an elegant problem?
I get a big bang out of proving a theory
or seeing a clever way to do something.
I make my money in the stock market,
not by proving theorems.
- [Michelle] You play the stock market?
You're interested in making money?
- Oh, sure.
Aren't you?
- [Michelle] Were you ever successful?
- [Betty] Beyond reasonable expectations.
(Claude chuckles)
- I even did some work on the theory of stocks. (Claude chuckles)
- I even did some work on the theory of stocks.
In a certain way it's like channel capacity
in a noisy channel.
- [Michelle] You found that you
could apply information theory to the stock market?
- Inside information.
- We had good friends who started good companies.
- Yeah. - We had good friends who started good companies.
- Yeah.
The bottom line is that theories aren't as important
as products or people.
(gentle instrumental music)
- In the '60s and '70s Shannon's ideas began
to have a lot of impact.
Computers are shrinking from the size of rooms,
data is being compressed and stored on drives
and CDs that could be scratched and still play. data is being compressed and stored on drives
and CDs that could be scratched and still play.
The Voyager satellite is sending images billions
of miles back to Earth.
But when it comes to finding the codes
that could get you anywhere near the Shannon limit,
there's a lot of frustration.
A lot of very talented people, over decades,
tried and failed to find the codes he said existed.
By the 1980s there's a sense
that maybe we've reached a practical limit,
a real world boundary you can't get through.
The Shannon limit is beautiful, an elegant theoretical idea,
but it's unattainable.
So people began to lose faith and wonder,
"Is information theory dead?"
And they begin to abandon information theory and, "Is information theory dead?"
And they begin to abandon information theory and,
in a sense, abandon Shannon too.
- It seems like after your big discoveries
you stopped really working on information theory.
Is that because you just grew tired of it?
- Well, the first part is right, that I didn't do much work
on it later, but, you know, most
of the great mathematicians have done their finest work on it later, but, you know, most
of the great mathematicians have done their finest work
in their '20s and '30s and not so much
when they're older, like athletes.
Although, I've just recently taken up jogging. (chuckles)
- But it seems like you spend a lot more time working
on your gadgets now.
- I have a lot of things I like to spend time on.
The intelligent machine idea is my big interest now.
The transformation of information that a human brain does,
not just the transmission, how the brain works or doesn't.
I think that's the biggest challenge now.
(bleeping)
Simulating the human brain.
Alan Turing and I often had lunch together and we'd talk
about if you could get a computer the equivalent of a brain, Alan Turing and I often had lunch together and we'd talk
about if you could get a computer the equivalent of a brain,
a human brain, or even a lot better.
I wrote a paper about how a computer could play chess. a human brain, or even a lot better.
I wrote a paper about how a computer could play chess.
I have a standing bet
that a computer will beat the grand champion by 2001.
(big band music) that a computer will beat the grand champion by 2001.
(big band music)
- Back in 1949, when computers were first
becoming available, Shannon had the vision
that it would be great to program one of these contraptions becoming available, Shannon had the vision
that it would be great to program one of these contraptions
to play the complicated, difficult game of chess.
He laid out a complete roadmap
of how to build a chess program.
- Of course, being Claude Shannon, it's not enough
to just write a paper, he also builds one
of the world's first successful chess-playing computers.
Now, computers weren't powerful enough at the time
to play a full game of chess,
so his computer can only play the last six moves of a game. to play a full game of chess,
so his computer can only play the last six moves of a game.
He calls it End Game, because that's what it can play.
It took eight years from when End Game was completed
to when someone else came and used Shannon's ideas
to build a computer that could play a full game of chess.
- Shannon observed that although chess
is incredibly complicated, the astronomical number - Shannon observed that although chess
is incredibly complicated, the astronomical number
of possible chess games, and he calculated those,
you don't have to look all the way to the end of the game
to figure out what a good move is, you only have you don't have to look all the way to the end of the game
to figure out what a good move is, you only have
to look ahead a few moves and conduct some sort
of evaluation about the possible moves you could reach
and find the path that leads to the best one.
This principle became the basis for every chess program
that came after it.
(big band music) that came after it.
(big band music)
- When you've done something like Shannon did in '48
and then again, or earlier in '45, I mean, cryptography
and information theory, what do you do for an encore? and then again, or earlier in '45, I mean, cryptography
and information theory, what do you do for an encore?
It's really hard.
And I think people are waiting for either something good
to come out, or some people who, maybe jealous,
are waiting for you to fall down and say,
"See, he's a has been."
- There is that small contusion of people though who say,
"Well, you know, Shannon worked on information theory alone
"in private for 10 years, "Well, you know, Shannon worked on information theory alone
"in private for 10 years,
"and then dropped this great achievement on everyone's lap
"without telling a soul, maybe, just maybe, "and then dropped this great achievement on everyone's lap
"without telling a soul, maybe, just maybe,
"he's up to something again, like that,
"that he's working on something.
"Could it be that he just has a project that he's working on
"that he hasn't told anyone about?"
- Do you wanna see the first wearable computer?
- [Michelle] Okay.
- I built this with Ed Thorp,
who was a professor here at MIT.
Ed had a paper he wanted to get published about blackjack
in the proceedings of the National Academy of Sciences.
Since I was the only member of the academy at the time in the proceedings of the National Academy of Sciences.
Since I was the only member of the academy at the time
at MIT, he asked for my help.
We got to talking, and it turns out
he had some pretty interesting ideas about roulette, too. We got to talking, and it turns out
he had some pretty interesting ideas about roulette, too.
(intense tropical music)
- My father was very interested in gambling. (intense tropical music)
- My father was very interested in gambling.
He loved con men, he loved that sort
of shadier side of people. He loved con men, he loved that sort
of shadier side of people.
Not interested in violence at all,
he was interested in cleverness.
So, movies that captured that cleverness, like "Topkapi"
or "The Music Man", were favorites of his.
So, he ordered a roulette wheel from Las Vegas
and they set it up in the basement with a strobe light
and went to work on a wearable computer
that could try to improve the odds for roulette.
- Our little computer could time the speed of the wheel
and the speed of the ball in the other direction.
It's a simple dynamic system with very little friction.
We discovered that nearly all roulette tables It's a simple dynamic system with very little friction.
We discovered that nearly all roulette tables
are slightly tilted, so we could predict two thirds
of the time which half of the wheel the ball would fall in.
The computer would be in my pocket, of the time which half of the wheel the ball would fall in.
The computer would be in my pocket,
and these little wires would run down my pant leg
into my shoe, and there was a little switch in there and these little wires would run down my pant leg
into my shoe, and there was a little switch in there
that I operated with my big toe.
And then these ran up to a little loud speaker in my ear,
and the computer would send one of the eight musical notes
of the scale to mark the octants in the wheel.
Ed had a little loud speaker in his ear, too,
and I operated the computer, and Ed placed the bets
and pretended he didn't know me. and I operated the computer, and Ed placed the bets
and pretended he didn't know me.
(intense tropical music)
We had some practical problems when we headed to Las Vegas.
The wires going down my pant leg kept shorting
and giving me little shocks.
(all laugh)
- And Ed's speaker fell out of his ear,
and he nearly scared this lady to death,
because she thought it was some alien insect.
(all laugh) because she thought it was some alien insect.
(all laugh)
We were lucky we didn't get arrested.
(all laugh)
(intense tropical music)
- He said this at one point,
he wasn't interested in practicality. - He said this at one point,
he wasn't interested in practicality.
I don't think that was quite what he meant.
He was very interested in what's needed to be practical,
but he didn't wanna spend his life starting a company.
He knew that that wasn't the right thing for him. but he didn't wanna spend his life starting a company.
He knew that that wasn't the right thing for him.
People who start companies have
to feel a very strong competitive urge, they have to want
to beat other people, and he was not a competitive person.
He was a person who, whenever you talked to him,
he was not trying to dominate you intellectually,
he was very much a cooperator, not a competitor.
He really was interested in practical things.
All the things he built were very practical. He really was interested in practical things.
All the things he built were very practical.
His maze, a toy, yes, but it was a toy
that illustrated some very interesting things.
His chess-playing machine, again, a toy,
but artificial intelligence people
just learned so many things from that, but artificial intelligence people
just learned so many things from that,
and it was highly useful.
His cryptography during the second World War,
the fact that it wasn't very practical then
is almost irrelevant.
He still wrote the best paper on cryptography is almost irrelevant.
He still wrote the best paper on cryptography
that's been written yet. He still wrote the best paper on cryptography
that's been written yet.
- He asked questions that no one asked,
and that was an amazing thing.
The questions are very meaningful, and he answered them,
and no one thought those questions The questions are very meaningful, and he answered them,
and no one thought those questions
could be answered that way.
- [Robert] Practical or playful?
It's a silly question in most cases.
He was into understanding.
- [Michelle] What do you think about the idea
that a machine might one day outthink a human?
- [Claude] Well, I'm rooting for the machines.
- But do you think a machine would ever come up
with the idea of a flaming trumpet? - But do you think a machine would ever come up
with the idea of a flaming trumpet?
(Claude chuckles)
- I think that creative stuff is a long way off,
but in the meantime they can be a boon to humanity.
(Claude sighs)
And I'd like to see that happen as soon as possible. (Claude sighs)
And I'd like to see that happen as soon as possible.
- Shannon's Master's thesis lays the foundation
for this idea that machines could be made to think,
that they could capably mimic a human brain. for this idea that machines could be made to think,
that they could capably mimic a human brain.
And he himself invents devices that mimic a brain.
There's Theseus, the mouse that can navigate its way
through a maze, there's End Game, the computer
that can play the last six moves in a chess game,
but it's at precisely the time when artificial intelligence
and computing is taking off but it's at precisely the time when artificial intelligence
and computing is taking off
that Shannon is on the sidelines.
Even though information theory was being applied that Shannon is on the sidelines.
Even though information theory was being applied
in ways beyond what Shannon could've imagined. Even though information theory was being applied
in ways beyond what Shannon could've imagined.
- [Michelle] I heard that one of your last papers proved
that you could make a reliable circuit
from faulty components.
Do you think that this might have applications
for the brain?
(Claude chuckles)
- I hope so.
I mean, the brain can suffer all kinds of damage
and still handle things pretty well,
which means there must be redundancy.
The fact that the brain has 10 billion neurons
probably means that it was cheaper for biology
to just make more components than
to figure out sophisticated circuits. to just make more components than
to figure out sophisticated circuits.
Trying to work out all the different connections to figure out sophisticated circuits.
Trying to work out all the different connections
would be astronomical, impossible,
but if we can find patterns,
well, there may be ways to simplify things.
- If we look at the data in the brain,
all the data is interrelated to each other.
So, there's a huge amount of redundancy.
If I have this type of neuron there, it tells me
that there's another kind of neuron there, If I have this type of neuron there, it tells me
that there's another kind of neuron there,
and this kind of neuron, and this one's not there,
and this one's not there. and this kind of neuron, and this one's not there,
and this one's not there.
So, we develop rules
that decide this is how neurons connect.
We take that brain model,
and we basically attach it to a virtual mouse body.
The virtual mouse body has simple models of the eyes
and the ears and the skin and the whiskers
so they can receive sensory input, and the ears and the skin and the whiskers
so they can receive sensory input,
then you let the virtual mouse loose
to start running through the maze. then you let the virtual mouse loose
to start running through the maze.
So, in that way it's quite similar to start running through the maze.
So, in that way it's quite similar
to what Shannon did nearly 60 years ago.
Einstein's theory of general relativity,
he had an enormous impact on society and the world
and physics, and pretty much how the world is today.
But I think that we haven't even begun to see the kind
of impact that Shannon is going to have on society,
because I think the more we understand how...
Understanding entropy and using information theory because I think the more we understand how...
Understanding entropy and using information theory
to help us understand complex systems,
such as the brain or biology or diseases, to help us understand complex systems,
such as the brain or biology or diseases,
I think that this is gonna open an amazing new door.
- Most physicists didn't really care about information
as a primary focus for many, many years.
Even most people thinking about abstruse,
esoteric questions in fundamental physics,
but that really changed in the 1980s
when a lot of people focused but that really changed in the 1980s
when a lot of people focused
on the famous black hole information paradox.
If you collide particles with enough energy,
(whooshing)
they'll have so much mass that they'll collapse the region
that they collided in into a black hole.
In a classical world, nothing can get out of the black hole,
even something going at the speed of light could not get out
of a black hole. even something going at the speed of light could not get out
of a black hole.
But Stephen Hawking showed
that quantum-mechanically black holes leak,
but what came out of the black hole
is completely independent of what made the black hole, but what came out of the black hole
is completely independent of what made the black hole,
and one was left with a major paradox. is completely independent of what made the black hole,
and one was left with a major paradox.
Where did the information about what made the black hole go? and one was left with a major paradox.
Where did the information about what made the black hole go?
Right?
Because seemingly you could make a black hole Right?
Because seemingly you could make a black hole
in a zillion different ways.
You could throw in an elephant,
you could throw in an encyclopedia,
but what came out always looked exactly the same. you could throw in an encyclopedia,
but what came out always looked exactly the same.
It looked like information was lost.
And that is a no-no.
That cannot happen according to any of the laws
of either classical or quantum mechanics.
And in order to decide whether the information is lost
or not, one thing that became apparent eventually is
that thinking more carefully about what the information was
and very much making use of the ideas that thinking more carefully about what the information was
and very much making use of the ideas
of entropy ended up being very important.
What's clear is that information
in physics is more important
than you would naively have thought.
And many of the most fundamental ideas really go back
to Shannon. And many of the most fundamental ideas really go back
to Shannon.
- In 1993, I was finishing up my PhD.
I was one year away from graduating,
and I went to a communications conference.
It wasn't an information theory conference.
It was in Switzerland and it's a very big conference,
so there's maybe 800 or 900 people that attend.
There's usually 10 to 12 parallel sessions
on everything from what's the next generation There's usually 10 to 12 parallel sessions
on everything from what's the next generation
of cell phone standards gonna look like
to more mathematical types of talks
in more theoretical dimensions.
So, I went to this session 'cause there was nothing else
to see, and maybe there was one person in the session So, I went to this session 'cause there was nothing else
to see, and maybe there was one person in the session
that I thought was a good speaker.
So I thought, "Okay, I'll go listen to that talk."
And there was a presentation. So I thought, "Okay, I'll go listen to that talk."
And there was a presentation.
I didn't know any of the authors, they were really unknown
to information theorists, coding theorists,
nobody knew who they were.
And these guys that got up, they were two French guys
and one guy from Thailand who nobody knew.
I mean, they were completely unknown. and one guy from Thailand who nobody knew.
I mean, they were completely unknown.
And so, most people were probably chatting with each other,
and maybe there were a few people paying attention. And so, most people were probably chatting with each other,
and maybe there were a few people paying attention.
And they get up and they say and maybe there were a few people paying attention.
And they get up and they say
they basically achieved the Shannon limit,
and everybody was incredibly skeptical.
We're like, "Oh, there must be a mistake and everybody was incredibly skeptical.
We're like, "Oh, there must be a mistake
"in their simulations, or, you know,
"they don't know what they're talking about,
"or their math is wrong."
- It was such a shock that people were just,
"It doesn't make any sense, this ugly thing." - It was such a shock that people were just,
"It doesn't make any sense, this ugly thing."
- It was mostly an experimental result. "It doesn't make any sense, this ugly thing."
- It was mostly an experimental result.
And the idea was to take multiple simple things
and combine them in a way
that created this very complex code.
And then they showed their simulation results, that created this very complex code.
And then they showed their simulation results,
which basically hit the Shannon limit, and it was amazing.
(intense instrumental music)
50 years on, the final piece of Shannon's vision
of the future was proved to be true.
You could have perfect communication
with as little errors as you wanted.
- Actually, everything that Shannon has predicted back
in 1948 came true. - Actually, everything that Shannon has predicted back
in 1948 came true.
He's completely vindicated.
- [Andrea] Anytime you store, process,
or transmit information in digital form,
every time you save a file on a hard drive, play a CD
or DVD, talk on a cell phone, send an email, Tweet, text,
Instagram, the wireless revolution,
you're relying on the concepts in Shannon's 1948 paper.
- Millions of people spend huge chunks
of their day consuming the codes
that Claude Shannon said had to be there.
Information is everything to us.
- [Andrea] And today, everyone carries Shannon around
in their pocket.
- Wow!
- My ultimate machine.
(machine buzzing)
(snapping)
- (chuckles) What?
- You know what Fats Waller said about swing music,
don't you, Michelle? - You know what Fats Waller said about swing music,
don't you, Michelle?
- Oh, no.
- "Either you got it or you ain't."
- Uh.
(machine buzzing)
(gentle instrumental music)
- I often wonder if Claude Shannon understood
how much information there would be
after his many discoveries.
I wonder, did he understand after his many discoveries.
I wonder, did he understand
that we would have this information explosion?
That in the last couple that we would have this information explosion?
That in the last couple
of years we've generated more information than the sum That in the last couple
of years we've generated more information than the sum
of all human history.
Literally, he not only won the lottery,
he invented the lottery. Literally, he not only won the lottery,
he invented the lottery.
(gentle instrumental music)
- When I was starting my PhD program at Harvard,
and this would've been 1992,
so my father already had Alzheimer's
and was already fairly compromised in terms
of his cognitive ability, he asked me what I was going
to be working on, and at the time I was working on a project
about how the magnetic field of the earth flips
from time to time, approximately every million years,
I was astonished at how quickly he was able
to ask the very core questions of the problem, I was astonished at how quickly he was able
to ask the very core questions of the problem,
he cut right to it, and I was amazed.
And I thought, "This is fantastic.
"I'm going to have this fabulous mind at my side (chuckles)
"while I go through graduate school. "I'm going to have this fabulous mind at my side (chuckles)
"while I go through graduate school.
"This is gonna be so much fun for both of us."
Alas, he was not able to continue that kind of questioning
or even thinking for very much longer,
so I didn't in fact have him at my side for graduate school.
But I remember very clearly that conversation and thinking,
"Wow!
"That's something."
(gentle instrumental music) "That's something."
(gentle instrumental music)
- There's science fiction books in which the dates
of things are so many years A.S.,
and the A.S. stands for After Shannon. of things are so many years A.S.,
and the A.S. stands for After Shannon.
In some sense, if you look at the human civilization,
the sudden change in human civilization that was created
by the digital revolution, computers,
happened at such a short period of time
that there's nothing else like that in history.
So, it could very well be that historically people
will look back and say, "What happened there?"
And if you were going to tie it to anybody,
Shannon is the one that gave us the tools
that people would still be using in the future.
So, it's not so implausible that people would still be using in the future.
So, it's not so implausible
that people would start measuring years after Shannon.
(gentle instrumental music)
- [Claude] Inventing something
is like sending a message in a bottle into the future.
(gentle jazz music)
- [Peggy] I think it was special
that he had both the ability and interest
in theoretical mathematics and this passion for building.
In a way that a toddler would explore the world,
he didn't lose that childlike curiosity and delight
in learning things, discovering things, he didn't lose that childlike curiosity and delight
in learning things, discovering things,
putting things together, making connections.
(gentle jazz music)
(intense instrumental music)