GASPARETTI FABIO
Obiettivi
The course aims to provide advanced and specific skills in the area of the latest Deep neural network architectures. Particular attention will be given to multimodal models, and networks capable of analyzing complex data structures, such as graphs and multivariate time series; and deep reinforcement learning. At the end of the course, the student will be able to: adequately design and optimize Deep neural networks, be able to distinguish and evaluate different solutions, and be able to select and customize the most effective network architectures to be used in real application domains, supervised, unsupervised, or following a reinforcement learning approach. The course consists of a theoretical and methodological part on advanced and innovative concepts, and a laboratory activity in which these concepts are applied in problem solving using the latest development frameworks.Programma
Introduction to DL; Training of Deep Architecture: hyperparameter tuning, batch normalization, faster optimizers, regularization ; Convolutional Neural Networks (CNN/ConvNets); Recurrent Neural Networks (GRU, LSTM, Bidirectional); Encoder-Decoder, Autoencoders, Variational Autoencoders; Attention layers ; Generative Adversarial Networks (GAN); Deep Reinforcement Learning; Embeddings; Use cases: AlexNet, VGG, NiN, GoogLeNet/Inception, ResNet, DenseNet; Use cases: Computer Vision and NLPPrerequisiti
Basic knowledge of perceptron and backpropagation, and the main training techniques of machine learning models.Testi Adottati
Simon J.D. Prince. "Understanding Deep Learning". MIT Press Dec 5th 2023 A. Geron,A. Geron, “Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems”, O'Reilly Media, Inc, USA, 2019.
Modalità Frequenza
Not mandatory but strongly recommended.Modalità Valutazione
The assessment is based on tests taken during the course and a final exam. The ongoing tests allow the student to take a shorter final exam. All the tests are written examinations, and consist of open-ended and closed-ended questions.Esempi Domande
Convolutional neural networks. Consider a 32×32×3 input image processed by a convolutional layer with 16 filters of size 3×3, stride 1, and padding 1. Determine the output tensor dimensions and the number of trainable parameters, including biases. Show your calculations.Deep neural architectures. Compare ResNet and DenseNet architectures, explaining the differences between residual connections and feature map concatenation. Discuss the implications for gradient propagation, feature reuse, and memory consumption.
Training and generalization. During neural network training, the training loss continues to decrease while the validation loss starts increasing after a certain number of epochs. Identify the phenomenon, explain its possible causes, and propose at least two strategies to improve model generalization.
Neural architecture design. Design a Deep Learning model for classifying variable-length time series. Describe an architecture based on LSTM or Transformer, justify your design choices, and discuss the main advantages and limitations of the selected approach.
