Space and astronautical engineering

FEDERICO FUSCO

General objectives:

Acquire the basic principles of the field of Artificial Intelligence, specifically the modeling of intelligent systems through the notion of intelligent agent.
Acquire the basic techniques developed in the field of Artificial Intelligence, concerning symbol manipulation and, more speicifically, discrete models.

Specific objectives:

Knowledge and understanding:

Automated search in the space state: general methods, heuristic driven methods, local Search. Factored representations: constraint satisfaction problems, automated planning.
Knowledge Representation through formal systems: propositional logic, first order logic, description logic (hints), non monotonic reasoning (hints). Usage of logic as a programming language: PROLOG.

Applying knowledge and understanding:

Modeling problems by means of the manifold representation techniques acquired through the course. Analysis of the behavior of the basic algorithms for automated reasoning.

Making judgements:
Being able to evaluate the quality of a representation model for a problem and the results of the application of the reasoning algorithms when run on it.

Communication:
The oral communication skills are stimulated through the interaction during class, while the writing skills will be developed thorugh the analysis of exercises and answers to open questions, that are included in the final test.

Lifelong learning skills:
In addition to the learning capabilities arising from the study of the theoretical models presented in the course, the problem solving capabilities of the student will be improved through the exercises where the acquired knowledge is applied.

The expected outcomes of the course include learning the basics of Machine Learning and a wide spectrum of Machine Learning methods and algorithms, together with their properties, convergence criteria and applicability. Students will learn to apply these approaches to various classes of learning problems and to experimentally validate their solution.
The addressed topics include: Linear Regression, Gradient Descent, Optimization, Regularization, Classification, Decision Trees and Kernel Methods, Statistical Learning, VC dimension, PAC Learning, Neural Networks, Clustering, Dimensionality Reduction.
Students will also learn software libraries, such as PyTorch, to actually solve practical problems

Basic knowledge of probability and multivariate analysis is required, as covered in introductory courses in probability and statistics and in calculus.

Introduction to machine learning
Linear Regression and Online Gradient Descent
Optimization in ML
Model selection and regularization
Classification, Decision Trees and Kernel Methods
Statistical Learning, VC dimension, PAC Learning
Neural Networks
Clustering and Dimensionality Reduction
Introduction to PyTorch

Understanding Machine Learning: From Theory to Algorithms. Shai Shalev-Shwartz and Shai Ben-David. Cambridge University Press. 2014.

The course is taught in person. Videos about labs and additional/focus lectures will be occasionally made available.

Attendance is optional but strongly encouraged.

The exam consists of a written test covering all course topics.

1)
- Provide a formal definition of a maximum likelihood (ML) hypothesis

- Comment the following statement: in a classification problem, the class returned by the ML hypothesis
on a new instance x is always the most probable class.

2)
Consider the learning problem of estimating the function f : ℜ → ℜ with dataset D= {(xi,yi)} plotted in
the figure below: ...

1. Describe how to perform regression based on these data using a method of your choice. Specifically,
provide a mathematical formulation of the model, highlighting the model parameters.
2. Considering the method you have chosen describe a way to reduce overfitting.
3. Draw a plausible plot of the learned model based on your choices.

3)
The following data have been collected and we want to learn the general concept Acceptable, by using
Decision Tree Learning.

House Furniture Nr rooms New kitchen Acceptable
1 No 3 Yes Yes
2 Yes 3 No No
3 No 4 No Yes
4 No 3 No No
5 Yes 4 No Yes

1. Formalize the learning problem: decribe exactly the target function to learn and the dataset.
2. Describe qualitatively how attributes are chosen when building a Decision Tree.
3. Simulate the execution of the ID3 algorithm on the data set above and generate the corresponding output
tree.


4)

1. Provide the main steps of classification based on K-nearest neighbors (K-NN).
2. Draw an example for a 4-classes classification problem in 2D. Use symbols (*,x,+,-) for the four classes.
Graphically show the application of the K-NN algorithm with K = 3 for the classification of 3 different
query points.

• Decision Theory


• Linear Regression


• Logistic Regression


• Perceptron/SVM


• Decision Trees and VC dimension