Generative AI: Foundations and Concepts

Overview

Explore the mathematical foundations of generative AI, from neural networks to advanced deep learning models, with practical implementation techniques and applications in engineering.

Syllabus

- Foundations of Neural Networks and Optimization -- In this module, you will explore the foundations of neural networks, including perceptrons, architectures, and learning algorithms. You will dive deeply into optimization methods critical for efficient training, focusing on advanced techniques like Newton’s and quasi-Newton methods, momentum, RMSProp, and Adam optimization algorithms. - Regularization and Advanced Techniques -- This module guides you through the mathematical approaches to regularization techniques that enhance neural network generalization and prevent overfitting. You will analyze concepts including Stein’s unbiased risk estimator, eigen decomposition, ensemble methods, dropout mechanisms, and advanced normalization techniques such as batch normalization. - Convolutional Neural Networks (CNNs) -- In this module, you will examine convolutional neural networks (CNNs), including convolution operations, parameter sharing, kernel methods, and multi-dimensional data structures. You'll explore advanced CNN architectures, regularization, normalization techniques, and the implications of random kernels on network learning behavior. - Generative Models and Maximum Likelihood Learning -- In this module, you will analyze the maths underpinning generative models and maximum likelihood estimation (MLE). You will explore divergence metrics such as Kullback-Leibler divergence, Bayesian network structures, and autoregressive modeling methods, focusing on their theoretical foundations and practical implications.

Taught by

Ramin Mohammadi

Tags