Physics

STEFANO GIAGU
STEFANO GIAGU
ANDREA CIARDIELLO
ANDREA CIARDIELLO

GENERAL OBJECTIVES:
Acquire familiarity with advanced deep learning techniques based on differentiable neural network models with supervised, unsupervised and reinforced learning paradigms; acquire skills in modelling complex problems through deep learning techniques, and be able to apply them to different application contexts in the fields of physics and basic and applied scientific research.

Discussed topics include: general machine learning concepts, differentiable neural networks, regularization techniques. Convolutional neural network, neural network for sequence analysis (RNN, LSTM / GRU, Transformers). Advanced learning techniques: transfer learning, domain adaptation, adversarial learning, self-supervised and contrastive learning, model distillation.
Graph Neural Networks (static and dynamic) and application to structured models for physics: dynamic models, simulation of complex fluids, GNN Hamiltonians and Lagrangians. Generative and variational models: variational mean-field theory, expectation maximization, energy based and maximum entropy models (Hopfield networks, Boltzman machines and RBM), AutoEncoders, Variational AutoEncoders, GANs, Autoregressive flow models, invertible networks, generative models based on GNN. Quantum Neural Networks.


SPECIFIC OBJECTIVES:
A - Knowledge and understanding
OF 1) Knowledge of the functioning of neural networks and their mathematical interpretation as universal approximators
OF 2) Understanding of the limits and potential of advanced machine learning models
OF 3) Understanding of the limits and potential of DL in solving physics problems

B - Application skills
OF 4) Design, implementation, commissioning and analysis of deep learning architectures to solve complex problems in physics and scientific research.

C - Autonomy of judgment
OF 5) To be able to evaluate the performance of different architectures, and to evaluate the generalization capacity of the same

D - Communication skills
OF 6) Being able to clearly communicate the formulation of an advanced learning problem and its implementation, its applicability in realistic contexts
OF 7) Being able to motivate and to evaluate the generalization capacity of a DL model

E - Ability to learn
OF 8) Being able to learn alternative and more complex techniques
OF 9) Being able to implement existing techniques in an efficient, robust and reliable manner

GENERAL OBJECTIVES:
Acquire familiarity with advanced deep learning techniques based on differentiable neural network models with supervised, unsupervised and reinforced learning paradigms; acquire skills in modelling complex problems through deep learning techniques, and be able to apply them to different application contexts in the fields of physics and basic and applied scientific research. Discussed topics include: general machine learning concepts, differentiable neural networks,
regularization techniques. Convolutional neural network, neural network for sequence analysis (RNN, LSTM / GRU, Transformers). Advanced learning techniques: transfer learning, domain adaptation, adversarial learning, self-supervised and contrastive learning, model distillation. Graph Neural Networks (static and dynamic) and application to structured models for physics: dynamic models, simulation of complex fluids, GNN Hamiltonians and Lagrangians. Generative and variational models: variational mean-field theory, expectation maximization, energy based and maximum entropy models (Hopfield networks, Boltzman machines and RBM), AutoEncoders, Variational AutoEncoders, GANs, Autoregressive flow models, invertible networks, generative models based on GNN. Quantum Neural Networks.

SPECIFIC OBJECTIVES:

A - Knowledge and understanding
OF 1) Knowledge of the functioning of neural networks and their mathematical interpretation asuniversal approximators
OF 2) Understanding of the limits and potential of advanced machine learning models
OF 3) Understanding of the limits and potential of DL in solving physics problems

B - Application skills
OF 4) Design, implementation, commissioning and analysis of deep learning architectures to solve complex problems in physics and scientific research.

C - Autonomy of judgment
OF 5) To be able to evaluate the performance of different architectures, and to evaluate the generalization capacity go these models

D - Communication skills
OF 6) Being able to clearly communicate the formulation of an advanced learning problem and its implementation, its applicability in realistic contexts
OF 7) Being able to motivate and to evaluate the generalization capacity of a DL model

E - Ability to learn
OF 8) Being able to learn alternative and more complex techniques
OF 9) Being able to implement existing techniques in an efficient, robust and reliable manner

important: calculus, linear algebra, basic notions of statistical and quantum mechanics.
Essential: basics of machine learning and deep neural networks, good knowledge of the python language programming and use of the numpy, matplotlib, sklearn pandas, and pytorch libraries

Theory lectures [50% of the course hours]
General recalls on differentiable artificial neural networks and use of the pytorch library for ANN design, training and testing. Basic architectures: MLP, Convolutional neural network, neural network for sequence analysis (RNN, LSTM / GRU). Bayesian-NN. Attention, Self-Attention, Transformers and Visual Transformers. Advanced learning techniques: transfer learning, domain adaptation, adversarial learning, self-supervised and contrastive learning, model distillation. Models for Object detection and semantic segmentation and applications. Graph Neural Network and Geometrical Deep Learning. Unsupervised models and anomaly detection. Generative Deep Learning: autoregressive models, invertible networks: diffusion and normalizing flow, generative GNN. Quantum Machine Learning on near-term quantum devices, design and training of quantum neural networks, Reinforcement Learning, Uncertainty quantification in DNNs. Energy models: Associative memories, Boltzmann Machines and RBMs.

Computational hands-on sessions [50% of the course hours]
Hands-on implementation and applications with Tensorflow and pytorch of the different models to physics problems.

Given the highly dynamic nature of the area covered by this advanced course, there is no single reference text. During the course the sources will be indicated and provided from time to time in the form of scientific articles and book chapters.

Reference texts:
I. Goodfellow, Y. Bengio, A. Courville: Deep Learning, MIT Press (https://www.deeplearningbook.org/)
P. Baldi, Deep Learning in Science, Cambridge University Press
W. L. Hamilton, Graph Representation Learning Book, MCGill Uni press (https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf)
M.Schuld, F.Petruccione, Machine Learning with Quantum Computers, Springer

The course is constituted for about 50% of lectures supported by slides projections and exercises aimed at providing advanced knowledge of Deep Learning techniques.
The remaining 50% is based on hands-on computational experiences that will provide the practical skills needed to develop and implement advanced Deep Learning models able to solve different problems in the field of physics and scientific research in general.

Attendance to the lectures is not mandatory but strongly recommended.
Attendance to the laboratory activities is mandatory for at least 2/3 of the sessions

to pass the course it is necessary to develop and document an individual project assigned at the end of May to be delivered before the exam in which you wish to participate. During the exam session the student will be asked to present and discuss the project through a short presentation with slides (~15-20 min max), answering specific questions posed by the exam commission

final grade is given by a weighted average between the vote on the project (50%) and the presentation and discussion (50%)

To achieve a score of 30/30 cum laude, the student must demonstrate that he has acquired an excellent knowledge of the topics covered in the course, and to be able to master the software tools needed to develop and implement the computational model treated during the course.

The determination of the final grade takes into account the following elements:

1. Home project 50%
The project will be dimensioned in such a way that it requires a maximum of 2 weeks of work to be completed and documented (a written report of maximum 15 pages + code and dataset is required to reproduce the results reported in the report). It will consist in reproducing and possibly improving the results reported in a scientific article in which DL methods are applied to an interesting and accessible problem for the student.
The evaluation will take into account:
- Correctness of the concepts exposed;
- Clarity of presentation;
- Ability to elaborate the concepts learned in the development of original projects.

2. Presentation and discussion 50%
The evaluation will take into account:
- Correctness of the concepts exposed and of the answers to specific questions posed by the exam commission;
- Clarity of presentation;

develop and document an individual project assigned at the end of the course focused on reproducing and/or improving the results of a specific document that demonstrates an advanced application of ML in physics or sciences in general.

present and discuss the project results and respond to some specific technical questions about the developed code.

• general conceptsand context
  • Books: C.M. Bishop: Deep Learning, Springer


• insights
  • Books: C.M. Bishop: Deep Learning, Springer and W. L. Hamilton, Graph Representation Learning Book, MCGill Uni press


• computational lab exercitations
  • Books: notebooks and notes provided by the instructors