Catalogo dei Corsi di studio

GENERAL OBJECTIVES:
Acquire familiarity with advanced deep learning techniques based on differentiable neural network models with supervised, unsupervised and reinforced learning paradigms; acquire skills in modelling complex problems through deep learning techniques, and be able to apply them to different application contexts in the fields of physics and basic and applied scientific research.

Discussed topics include: general machine learning concepts, differentiable neural networks, regularization techniques. Convolutional neural network, neural network for sequence analysis (RNN, LSTM / GRU, Transformers). Advanced learning techniques: transfer learning, domain adaptation, adversarial learning, self-supervised and contrastive learning, model distillation.
Graph Neural Networks (static and dynamic) and application to structured models for physics: dynamic models, simulation of complex fluids, GNN Hamiltonians and Lagrangians. Generative and variational models: variational mean-field theory, expectation maximization, energy based and maximum entropy models (Hopfield networks, Boltzman machines and RBM), AutoEncoders, Variational AutoEncoders, GANs, Autoregressive flow models, invertible networks, generative models based on GNN. Quantum Neural Networks.


SPECIFIC OBJECTIVES:
A - Knowledge and understanding
OF 1) Knowledge of the functioning of neural networks and their mathematical interpretation as universal approximators
OF 2) Understanding of the limits and potential of advanced machine learning models
OF 3) Understanding of the limits and potential of DL in solving physics problems

B - Application skills
OF 4) Design, implementation, commissioning and analysis of deep learning architectures to solve complex problems in physics and scientific research.

C - Autonomy of judgment
OF 5) To be able to evaluate the performance of different architectures, and to evaluate the generalization capacity of the same

D - Communication skills
OF 6) Being able to clearly communicate the formulation of an advanced learning problem and its implementation, its applicability in realistic contexts
OF 7) Being able to motivate and to evaluate the generalization capacity of a DL model

E - Ability to learn
OF 8) Being able to learn alternative and more complex techniques
OF 9) Being able to implement existing techniques in an efficient, robust and reliable manner

Programme

Theory lectures [50% of the course hours]
Recalls of Machine Learning basics, Probability and Information Theory concepts, bias-variance decomposition and metrics for complexity (VC-dimension, Rademacher complexity, etc.).
Neural Networks, Universal approximation theorem, Computational graphs and backpropagation, Regularisation techniques.
Deep learning and deep neural network. Convolutional neural network, NN for sequence modeling (RNN, LSTM/GRU, Transformers).
Advanced learning techniques: transfer learning, domain adaptation, adversarial learning, self-supervised and contrastive learning, model distillation.
Graph Neural Networks (static and dynamic) and application to structured models for physics: dynamic models, simulation of complex fluids, Hamiltonian Ordinary Differential Equation Networks, Lagrangian Graph Neural Networks.
Variational methods and deep learning: variational mean-field theory, expectation maximization, energy based and maximum entropy models (Hopfield networks, Boltzman machines and RBM), AutoEncoders, Variational AutoEncoders, GANs, Autoregressive flow models, invertible networks, methods based on graph-NN.
Quantum Neural Network and implementation on Quantum Computers.

Computational hands-on sessions [50% of the course hours]
Hands-on implementation and applications with Tensorflow and pytorch of the different models to physics problems.

Adopted texts

Given the highly dynamic nature of the area covered by this advanced course, there is no single reference text. During the course the sources will be indicated and provided from time to time in the form of scientific articles and book chapters.

Bibliography

Reference texts: I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, MIT Press C.M. Bishop: Pattern Recognition and Machine Learning, Springer  T.Hastie, R. Tibshirani, J. Friedman: The Elements of Statistical Learning, Springer F.Chollet: Deep Learning with Python 2nd ed., Manning A.Gulli, A.Kapoor, S.Pal: Deep LEarning with Tensorflow 2 and Keras, Packt R.Atienza: Advanced Deep LEarning with Tensorflow 2 and Keras Packt V. Subramian: Deep Learning with PyTorch, Packt

Prerequisites

Useful: basics of machine learning. Important: calculus, linear algebra, basic notions of statistical and quantum mechanics. Essential: Python language programming fundamentals.

Exam modes

To pass the exam it is necessary to pass a computational test and an home project assigned near the end of the course.

A minimal score of 15/30 is mandatory to pass the computational test and a minimal score of 18/30 for the home project.
The final grade will be a weighted mean of the score of the computational test (weight 60%) and the score of the home project (weight 40%), incremented by 5%. Students can decide to accept the proposed vote and register it, or can ask for an additional oral exam (covering full course program) to improve it.
It is necessary to demonstrate to have acquired sufficient knowledge of both concept and applications of the advanced DL methods discussed during the course.

To achieve a score of 30/30 cum laude, the student must demonstrate that he has acquired an excellent knowledge of the topics covered in the course, and to be able to master the software tools needed to develop and implement the computational model treated during the course.

The determination of the final grade takes into account the following elements:
1. Computational test 60%
It is an individual test in which the student implements advanced DL-based programs aimed at solving an assigned physics problem using a dataset (provided or to be produced by simulation as part of the test).
The evaluation will take into account:
- Quality of the procedures and methods applied, and quality of the achieved results
- Correctness of the concepts exposed;
- Clarity of presentation;

2. Home project 40%
The project will be dimensioned in such a way that it requires a maximum of 1 week of work to be completed and documented (a written report of maximum 10 pages + code and dataset is required to reproduce the results reported in the report). It will consist in reproducing and possibly improving the results reported in a scientific article in which DL methods are applied to an interesting and accessible problem for the student.
The evaluation will take into account:
- Correctness of the concepts exposed;
- Clarity of presentation;
- Ability to elaborate the concepts learned in the development of original projects.

Exam reservation date start	Exam reservation date end	Exam date
01/05/2022	28/06/2022	29/06/2022
01/05/2022	12/07/2022	13/07/2022
01/07/2022	31/08/2022	01/09/2022
01/07/2022	23/10/2022	24/10/2022
01/07/2022	01/11/2022	02/11/2022
01/07/2022	10/11/2022	11/11/2022
26/09/2022	10/01/2023	20/01/2023