# INFORMATION THEORY AND CODES

If you can't find the course description that you're looking for in the above list, please see the following instructions >>
Versione italiana
2017/2018
Teacher
VELIO TRALLI
Credits
6
Didactic period
Primo Semestre
SSD
ING-INF/03

#### Training objectives

The course presents the fundamentals of the information theory and of the coding techniques used in the digital communication systems.
The student will gain the knowledge of:
- the main concepts of the information theory related to the representation/compression of analog and digital sources, and the transmission of information in the communication channels.
- the main theoretical elements for the construction, decoding and performance evaluation of codes for digital communications.
At the end of the course the student will be able to:
- evaluate and use entropy, mutual information and capacity with reference to simple sources and communication channels
- define and use simple encoding and decoding algorithms, in hard-decision or soft-decision mode, for block and convolutional codes.

#### Prerequisites

Knowledge of the fundamentals of signal theory and communication systems, probability theory and statistics, linear algebra

#### Course programme

- Information, entropy, source coding and related limits, Lempel-Ziv coding, coding for compression, rate-distortion function, scalar quantization and vector quantization, coding for voice signals.

- Mutual information, Fano inequalitiy, DMC channel, channel capacity, channel coding, channel coding theorem, random coding bounds and cut-off rate, differential entropy, noisy channel and capacity, capacity of vector channel and water-filling

- FEC and ARQ schemes, Es-Eb/No plane, linear block codes, error detection and error correction, error probability with hard and soft decision.

- Cyclic codes and their polynomial description, encoding in systematic form, irreducible, primitive and minimal polynomials, BCH codes, Singleton bound, Reed-Solomon Codes, decoding of BCH codes

- Convolutional codes and their description, weight enumerating functions, MLSE decoding, Viterbi algorithm, error probability with hard and soft decision.

- Investigation of advanced coding techniques with iterative decoding and applications

#### Didactic methods

The course is based on classroom lessons that cover all the items in the programme. The presentation of the theory is followed by selected examples or exercises. At the end of the course, each student is required to individually investigate an advanced topic chosen in a list provided by the teacher. This investigation includes reading and understanding a book section and/or a scientific article, and the composition of a report on the topic.

#### Learning assessment procedures

The final examination is suitably organized to check the achievement of the learning objectives as outlined above. It consists of a single oral part that includes the discussion of the chosen advanced topic and some questions addressing the theoretical parts of the programme. If requested, English can be used for the final exam.

Passing the final exam is the proof that knowledge and abilities outlined in the training objectives of the course have been achieved.

#### Reference texts

1. Cover, Thomas, Elements of Information Theory, Wiley Interscience
2. W.E.Ryan, S.Lin, Channel Codes - Classical and Modern, Cambridge
3. S.lin, D.J.Costello, Error Control Coding, Pearson - Prentice Hall
4. Wilson, Digital Modulation and Coding, Wiley
5. J.G.Proakis, Digital Communications, McGraw Hill

The content of the course is almost completely covered by the textbooks 1 and 2. The other textbooks, together with 1 and 2, can be used to enlarge the knowledge on specific topics.