Electronics Engineering

Introduction to pattern recognition. Classification and clustering problems. Generalization capability. Logic inferences: deduction and induction. Induction principle over normed spaces. Choosing a Metric. Non-metric spaces. Point to point, point to cluster and cluster to cluster proximity measures. Mahalanobis distance. Representation and preprocessing functions. Normalization. Missing Data. Ordinal and nominal discrete data. The basic processing pipeline in machine learning and pattern recognition. Clustering algorithms: k-means, BSAS. The cluster validity problem; sensitivity index; relative validation indexes; Davies-Bouldin index; Silhouette Index; clustering algorithms based on scale parameters; stability indexes; optimized clustering algorithms; constrained and unconstrained unsupervised modelling problems. Hierarchical clustering. Splitting and merging algorithms. Decision rules: K-NN and condensed K-NN. Classification systems: performance and sensitivity measures. Classification model synthesis based on cluster analysis. Robust classification: voting techniques. Ensembles of classifiers. Structured data taxonomy. Dissimilarity measures on structured data. Data fusion. Variable length domains: sequences of events, graphs. Bellman optimality principle; edit distance. Dissimilarity measures on labelled graph spaces (Graph Matching). Automatic Feature selection algorithms. Basic principle on evolutionary optimization. Introduction to Granular Computing. Metric learning. Local metrics. Representations in dissimilarity spaces. Symbolic histograms. Data Mining and Knowledge Discovery. Agent based parallel and distributed algorithms for machine learning. Hardware acceleration on FPGAs and GPUs. Second part: Hardware and software foundations for pattern recognition systems. Setup and configuration of an Anaconda environment for machine learning application prototyping. Visual Studio code and GitHub. Usage of integrated development environments and Jupyter notebooks for interactive experimentation in Python. Core data structures in Python and their application in classical machine learning. Introduction to essential Python libraries for data analysis and scientific computing. Practice with Python notebooks on data preprocessing, distances and dissimilarities, Clustering and K-NN. Basic statistical concepts and probabilistic reasoning. Bayes Theorem and Bayesian learning methods. Probabilistic learning models: maximum a posteriori (MAP) criterion and maximum likelihood estimation. Naïve Bayes classifier. Decision surfaces and discriminant functions for Bayesian classifiers. Practical implementation of Bayesian classification in Python notebooks. Generative and discriminative models. Linear regression and logistic regression. Overfitting and regularization strategies. Validation techniques including N-Fold cross-validation. Introduction to reinforcement learning. Distinction among supervised, unsupervised, and reinforcement learning paradigms. Fundamental strategies and components of reinforcement learning. Deep learning architectures and their comparison with traditional machine learning models. Neural networks, convolutional neural networks, autoencoders. Practical implementation of deep models in Python notebooks with TensorFolow/PyTorch. Natural language processing techniques: traditional approaches and neural language models. Text mining, word embedding, Large Language Models and sequence processing. The transformer and the multi-head attention mechanism. Classical Transformer and BERT. Navigating the GPT architecture. Tokenization and embedding. Positional encoding. Context window and hierarchical processing. In-context learning. Abstract representations. Explainability, reasoning and agentic generative AI. Implementation of NLP models in TensorFlow/PyTorch. Production ready AI agent frameworks: the Cheshire Cat framework. Retrieval-Augmented Generation systems versus model fine-tuning. Case studies on large language model applications.

Elementary notions of Geometry, Algebra, Differential Calculus, Signal Theory, Information Theory, Informatics.

Sergios Theodoridis, Konstantinos Koutroumbas, Pattern Recognition, Fourth Edition, Academic Press, ISBN: 978-1597492720, September 2008. Lecture notes and slides available at teacher's site

The course is organized as a series of lectures and case study illustrations.

It is strongly recommended to attend classroom lessons.

The final exam consists in the evaluation of a homework. The topic of the homework is usually agreed with the teacher.