Elements of Information Theory

(0) Donner la première évaluation
CHF 91.00
Download est disponible immédiatement
eBook (pdf)
Informations sur les eBooks
Les eBooks conviennent également aux appareils mobiles (voir les instructions).
Les eBooks d'Ex Libris sont protégés contre la copie par ADOBE DRM: apprenez-en plus.
Pour plus d'informations, cliquez ici.

Description

The latest edition of this classic is updated with new problem sets and material The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory. All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points. The Second Edition features: * Chapters reorganized to improve teaching * 200 new problems * New material on source coding, portfolio theory, and feedback capacity * Updated references Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications.



Auteur

THOMAS M. COVER, PHD, is Professor in the departments of electrical engineering and statistics, Stanford University. A recipient of the 1991 IEEE Claude E. Shannon Award, Dr. Cover is a past president of the IEEE Information Theory Society, a Fellow of the IEEE and the Institute of Mathematical Statistics, and a member of the National Academy of Engineering and the American Academy of Arts and Science. He has authored more than 100 technical papers and is coeditor of Open Problems in Communication and Computation.

JOY A. THOMAS, PHD, is the Chief Scientist at Stratify, Inc., a Silicon Valley start-up specializing in organizing unstructured information. After receiving his PhD at Stanford, Dr. Thomas spent more than nine years at the IBM T. J. Watson Research Center in Yorktown Heights, New York. Dr. Thomas is a recipient of the IEEE Charles LeGeyt Fortescue Fellowship.

Texte du rabat

THE LATEST EDITION OF THIS CLASSIC IS UPDATED WITH NEW PROBLEM SETS AND MATERIAL

The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory.

All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points.

The Second Edition features:

  • Chapters reorganized to improve teaching
  • 200 new problems
  • New material on source coding, portfolio theory, and feedback capacity
  • Updated references

Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications.

Contenu

Contents v

Preface to the Second Edition xv

Preface to the First Edition xvii

Acknowledgments for the Second Edition xxi

Acknowledgments for the First Edition xxiii

1 Introduction and Preview 1

1.1 Preview of the Book 5

2 Entropy, Relative Entropy, and Mutual Information 13

2.1 Entropy 13

2.2 Joint Entropy and Conditional Entropy 16

2.3 Relative Entropy and Mutual Information 19

2.4 Relationship Between Entropy and Mutual Information 20

2.5 Chain Rules for Entropy, Relative Entropy, and Mutual Information 22

2.6 Jensen's Inequality and Its Consequences 25

2.7 Log Sum Inequality and Its Applications 30

2.8 Data-Processing Inequality 34

2.9 Sufficient Statistics 35

2.10 Fano's Inequality 37

Summary 41

Problems 43

Historical Notes 54

3 Asymptotic Equipartition Property 57

3.1 Asymptotic Equipartition Property Theorem 58

3.2 Consequences of the AEP: Data Compression 60

3.3 High-Probability Sets and the Typical Set 62

Summary 64

Problems 64

Historical Notes 69

4 Entropy Rates of a Stochastic Process 71

4.1 Markov Chains 71

4.2 Entropy Rate 74

4.3 Example: Entropy Rate of a Random Walk on a Weighted Graph 78

4.4 Second Law of Thermodynamics 81

4.5 Functions of Markov Chains 84

Summary 87

Problems 88

Historical Notes 100

5 Data Compression 103

5.1 Examples of Codes 103

5.2 Kraft Inequality 107

5.3 Optimal Codes 110

5.4 Bounds on the Optimal Code Length 112

5.5 Kraft Inequality for Uniquely Decodable Codes 115

5.6 Huffman Codes 118

5.7 Some Comments on Huffman Codes 120

5.8 Optimality of Huffman Codes 123

5.9 ShannonFanoElias Coding 127

5.10 Competitive Optimality of the Shannon Code 130

5.11 Generation of Discrete Distributions from Fair Coins 134

Summary 141

Problems 142

Historical Notes 157

6 Gambling and Data Compression 159

6.1 The Horse Race 159

6.2 Gambling and Side Information 164

6.3 Dependent Horse Races and Entropy Rate 166

6.4 The Entropy of English 168

6.5 Data Compression and Gambling 171

6.6 Gambling Estimate of the Entropy of English 173

Summary 175

Problems 176

Historical Notes 182

7 Channel Capacity 183

7.1 Examples of Channel Capacity 184

7.1.1 Noiseless Binary Channel 184

7.1.2 Noisy Channel with Nonoverlapping Outputs 185

7.1.3 Noisy Typewriter 186

7.1.4 Binary Symmetric Channel 187

7.1.5 Binary Erasure Channel 188

7.2 Symmetric Channels 189

7.3 Properties of Channel Capacity 191

7.4 Preview of the Channel Coding Theorem 191

7.5 Definitions 192

7.6 Jointly Typical Sequences 195

7.7 Channel Coding Theorem 199

7.8 Zero-Error Codes 205

7.9 Fano's Inequality and the Converse to the Coding Theorem 206

7.10 Equality in the Converse to the Channel Coding Theorem 208

7.11 Hamming Codes 210

7.12 Feedback Capacity 216

7.13 SourceChannel Separation Theorem 218

Summary 222

Problems 223

Historical Notes 240

8 Differential Entropy 243

8.1 Definitions 243

8.2 AEP for Continuous Random Variables 245

8.3 Relation of Differential Entropy to Discrete Entropy 247

8.4 Joint and Conditional Differential Entropy 249

8.5 Relative Entropy and Mutual Information 250

8.6 Properties of Differential Entropy, Relative Entropy, and Mutual Information 252

Summary 256...

Afficher plus

Détails sur le produit

Titre
Elements of Information Theory
Auteur
EAN
9780471748816
ISBN
978-0-471-74881-6
Format
eBook (pdf)
Producteur
Wiley-Interscience
Editeur
Wiley-Interscience
Genre
Autres
Parution
28.11.2012
Protection contre la copie numérique
Adobe DRM
Taille de fichier
5.64 MB
Nombre de pages
776
Année
2006
Edition
2. Aufl.
Afficher plus
Les clients ayant acheté cet article ont également acheté :