Deep Neural Network

Topics covered

• Learning & generalization: Supervised learning, empirical risk minimization, regularization, early stopping, and weight decay.
• Training: Gradient descent, stochastic optimization, Adam, initialization, and normalization.
• Neural architectures: Convolutional, graph, and recurrent neural networks; data augmentation.
• Language models: Transformers and autoregressive modeling.