Shannons Formula & Hartleys Rule: Olivier Rioul Jos Carlos - PowerPoint PPT Presentation

Shannon’s Formula & Hartley’s Rule: Olivier Rioul ∗ José Carlos Magossi † ∗ Télécom ParisTech † Unicamp

c l a u d e s h a n n o n 2/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

c l a u d e s h a n n o n a s o u n d c h a n n e l 2/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

c l a u d e s h a n n o n a s o u n d c h a n n e l Shannon’s formula: 1 + P � � C = 1 2 log 2 bits/symbol N 2/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

c l a u d e s h a n n o n a s o u n d c h a n n e l Shannon’s formula: 1 + P � � or... C = W log 2 bits/second N 2/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Claude Shannon Shannon’s formula: 1 + P � � C = 1 2 log 2 N “A Mathematical Theory of Communication,” The Bell System Technical Journal , Vol. 27, pp. 623–656, October, 1948 . 3/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Claude Shannon Shannon’s formula: 1 + P � � C = 1 2 log 2 N “A Mathematical Theory of Communication,” The Bell System Technical Journal , Vol. 27, pp. 623–656, October, 1948 . 3/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Claude Shannon Shannon’s formula: 1 + P � � C = 1 2 log 2 N “A Mathematical Theory of Communication,” The Bell System Technical Journal , Vol. 27, pp. 623–656, October, 1948 . 3/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Ralph Hartley 20 years before... in the same journal... 4/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Ralph Hartley 20 years before... in the same journal... Hartley’s rule: 1 + A C ′ = log 2 � � bits/symbol ∆ “Transmission of Information,” The Bell System Technical Journal , Vol. 7, pp. 535–563, July 1928 . 4/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Ralph Hartley 20 years before... in the same journal... Hartley’s rule: 1 + A C ′ = 2 W log 2 � � or... bits/second ∆ “Transmission of Information,” The Bell System Technical Journal , Vol. 7, pp. 535–563, July 1928 . 4/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Ralph Hartley Hartley’s rule: 1 + A C ′ = log 2 � � ∆ (Wozencraft-Jacobs textbook, 1965) 5/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Ralph Hartley Hartley’s rule: 1 + A C ′ = log 2 � � ∆ ◮ amplitude “SNR” A / ∆ (factor 1/2 is missing) ◮ no coding involved (except quantization) ◮ zero error (Wozencraft-Jacobs textbook, 1965) 5/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline Hartley’s C ′ = log 2 � � 1 + A came 20 years before Shannon ∆ 6/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline Hartley’s C ′ = log 2 � � 1 + A came 20 years before Shannon ∆ � � Shannon’s C = 1 1 + P 2 log 2 came unexpected in 1948 N 6/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline Hartley’s C ′ = log 2 � � 1 + A came 20 years before Shannon ∆ � � Shannon’s C = 1 1 + P 2 log 2 came unexpected in 1948 N Hartley’s rule is inexact: C ′ � = C 6/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline Hartley’s C ′ = log 2 � � 1 + A came 20 years before Shannon ∆ � � Shannon’s C = 1 1 + P 2 log 2 came unexpected in 1948 N Hartley’s rule is inexact: C ′ � = C Besides, C ′ is not the capacity of a noisy channel 6/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline Hartley’s C ′ = log 2 � � 1 + A came 20 years before Shannon ∆ � � Shannon’s C = 1 1 + P 2 log 2 came unexpected in 1948 N Hartley’s rule is inexact: C ′ � = C Besides, C ′ is not the capacity of a noisy channel (no question) 6/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline Wrong! 7/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline This Hartley’s rule C ′ = log 2 � � 1 + A is not Hartley’s ∆ 8/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline This Hartley’s rule C ′ = log 2 � � 1 + A is not Hartley’s ∆ � � Many authors independently derived C = 1 1 + P 2 log 2 in 1948. N 8/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline This Hartley’s rule C ′ = log 2 � � 1 + A is not Hartley’s ∆ � � Many authors independently derived C = 1 1 + P 2 log 2 in 1948. N In fact, C ′ = C ( a coincidence? ) 8/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline This Hartley’s rule C ′ = log 2 � � 1 + A is not Hartley’s ∆ � � Many authors independently derived C = 1 1 + P 2 log 2 in 1948. N In fact, C ′ = C ( a coincidence? ) Besides, C ′ is the capacity of the “uniform” channel 8/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline This Hartley’s rule C ′ = log 2 � � 1 + A is not Hartley’s ∆ � � Many authors independently derived C = 1 1 + P 2 log 2 in 1948. N In fact, C ′ = C ( a coincidence? ) Besides, C ′ is the capacity of the “uniform” channel (and we can explain) 8/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?

Outline This Hartley’s rule C ′ = log 2 � � 1 + A is not Hartley’s ∆ � � Many authors independently derived C = 1 1 + P 2 log 2 in 1948. N In fact, C ′ = C ( a coincidence? ) Besides, C ′ is the capacity of the “uniform” channel (and we can explain) 9/31 23 Sept 2014 Shannon’s Formula & Hartley’s Rule: A Mathematical Coincidence?