Management Engineering

1. Introduction. Definition of learning systems. Goals and applications of machine learning (classification and regression). Basics on statistical learning theory (Vapnik Chervonenkis bound). Underfitting and Overfitting. Use of data: training set, test set, validation set. 2. Review of optimization tools and comparison of learning algorithms from the optimization point of view. (3 lezioni) 3. Artificial Neural Networks. Neurons and biological motivation. Linear threshold units. The Perceptron and its learning algorithm (proof of convergence). Classification of linearly separable patterns. Multi-Layer Feedforward Neural Networks. Gradient method: basics. Back-propagation (BP) algorithm. BP batch version: proof of convergence and choice of the learning rate. BP on-line version: incremental method, theorem of convergence. Momentum updating rule. Radial-Basis function (RBF) networks: regularized and generalized RBF networks. Their use in interpolation and approximation. learning strategies and error functions. Unsupervised selection of center. Supervised selection of weights and centers: decomposition methods into two blocks and decomposition methods into more blocks. Convergence theory of decomposition methods. Early stopping 4. Support Vector Machines (Kernel methods) Soft and hard Maximum Margin Classifiers. Quadratic programming formulation of the soft/hard maximum margin separators. Kernels methods. Dual formulation of the primal QP problem. Wolfe duality theory for QP. KKT conditions. Frank Wolfe method: basics. Decomposition methods: SMO-type algorithms, MVP algorithm, SVMlight, cyclic methods. Convergence theory. Implementation tricks: Caching, shrinking. Choosing parameters: k-fold cross-validation. Multiclass SVM problems: one-against-one and one-against-all. 5. Practical use of learning algorithms. 6. Use of open-source software

Linear algebra, principles of mathematical analisys for multivariate functions (Taylor, partial derivatives). Convexity No propeudicity with other courses is envisaged.

Teaching material is made up of lectures slides and lectures notes. The following books are also suggested Pattern Recognition and Machine Learning - Bishop - 2006 Deep Learning - Goodfellow, Bengio, Courville - 2016

The teaching is held in presence in the classroom with face-to-face teaching. The possibility of conducting online lessons is envisaged in the event of needs related to the current pandemic situation. Attendance is not mandatory, but recommended. The teaching method involves face-to-face teaching and interaction with the students by means of self-assessment questionnaires carried out in the classroom.

in presence

The evaluation involves one or two projects. If carried out during the semester of the course the projects are two and associated with two multiple-choice and/or open written questions. If carried out after the end of the course, only one project is to be awarded in conjunction with one written and one oral test. The aim of the project is to acquire skills in the autonomous development of a machine learning system at various levels, starting from the use of open source software up to the development of own code. The written and/or oral test is aimed at testing methodological skills.

Pattern Recognition and Machine Learning - Bishop - 2006 Deep Learning - Goodfellow, Bengio, Courville - 2016

in presence