Introduction and artificial neural networks: their history, the perceptron and its limitations, boolean and real multi-layered perceptron, topologies, universal function approximation, decision boundaries, activations, sufficiency of architecture, Training I: recall on Gradient, Jacobian, Hessian; what is learning, empirical risk minimization, gradient descent, calculus of backpropagation, Training II: convergence issues, loss surfaces, momentum, optimization, second order methods, regularization strategies, initialization, Convnets I: definitions, types of convolutions, pooling, prominent architectures, Convnets II: vision models with convnets, feature pyramid, transposed convolution, object detection and segmentation, Sequence modeling I: Time series, RNNs, Sequence modeling II: Memory, LSTMs, sequence prediction, Attention: transformers, sequence to sequence predictions, LLMs and their downstream applications, Representation learning: autoencoders, self-supervision, unsupervised approaches, contrastive learning, Generative DL: Variational autoencoders, GANs and diffusion, Deep reinforcement learning: deep q-learning, Graph Neural networks, AI ethics

This course covers the theory and foundations of Artificial Neural Networks and various shallow neural network architectures, including the single & multi-layer perceptrons and deep learning architectures (e.g. convolutional neural networks, recurrent networks, autoencoders, generative adversarial networks, transformers). Students will be expected to develop deep learning systems for machine learning problems for a variety of data types (visual, text, graphs).

S. J. D. Prince, Understanding Deep Learning – https://udlbook.github.io/udlbook/
M. Nielsen, Neural Networks and Deep Learning – http://neuralnetworksanddeeplearning.com/
Goodfellow, Bengio and Courville, Deep Learning – https://www.deeplearningbook.org/
Zhang, A and others, Dive into Deep learning – https://d2l.ai/