STOCHASTIC PROCESSES

Course objectives

General objectives: The objective of the course is to give students an overview of the main stochastic processes such as Markov chains and Brownian motion, explaining its fundamental dynamic-probabilistic properties and various relevant applications such as the Monte Carlo method and the probabilistic representation of solutions of differential equations. Entropic concepts of information theory for Markov chains will also be introduced. Specific objectives: The course mainly deals with the following topics: discrete and continuous time Markov chains (stationarity, reversibility, convergence to equilibrium, ergodic theorem); Monte Carlo method; relative entropy for Markov chains (entropy of a random variable, conditional entropy, relative entropy for a succession of random variables); Brownian motion and heat equation; approximation of Brownian motion by means of random walks. Knowledge and understanding: The student will acquire knowledge on stochastic processes and some of their applications in information theory and applied mathematics. Application of knowledge and understanding: The student will be able to model random evolutions through stochastic processes, study their qualitative and quantitative behavior. He/she will also become familiar with the probabilistic foundations of important simulation and approximation techniques based on stochastic processes. Autonomy of judgment: The student will be able to understand the problems related to random dynamics. Communication skills: The student must acquire an appropriate scientific language to explain in a clear and linear way the concepts underlying the stochastic processes. Learning ability: The course allows the student to fully understand the use of stochastic processes in various applications, including IT.

Channel 1
MARCO ISOPI Lecturers' profile

Program - Frequency - Exams

Course program
Refresher of basic probability. Bernoulli processes. Stationary processes. Markov chains with finite states (discrete time). Branching processes. Random walks. Poisson processes. Markov chains with finite states (continuous time). Birth and death processes. Queueing theory. If time permits: Hidden Markov Models, renewal processes.
Prerequisites
Basic probability, Differential and integral calculus, Finite dimensional linear algebra.
Books
Grimmert-Stirzaker - Probabaility and Random Processess.
Frequency
Attendance not mandatory, but strongly recommended.
Exam mode
Problem-solving ability and competence on course topics will be assessed through a written test (mandatory) and an oral test (optional).
Bibliography
Ross - Stochastic Processes
Lesson mode
Traditional lectures and discussion of homework problems.
  • Lesson code1031451
  • Academic year2024/2025
  • CourseMathematical Sciences for Artificial Intelligence
  • CurriculumSingle curriculum
  • Year3rd year
  • Semester1st semester
  • SSDMAT/06
  • CFU6
  • Subject areaFormazione Modellistico-Applicativa