From Theory to Practice: Deep Learning for NLP

Hassan Sajjad and I got another chance to teach our deep learning course, but this time it was only for five days, delivered over roughly 18 hours! This course is a bit more general than our DGfS course, and has a lot more practical material.

The official spiel

In this lecture series, we cover the basics of machine learning, neural networks and deep neural networks. We look at several deep neural network architectures from the perspective of applying them to various classification tasks, such as sequence prediction and generation. Every lecture is accompanied with practice problems implemented in Keras, a popular Python framework for deep learning.

Pre-requisites

• Python
  • Python/numpy tutorial
  • iPython/Jupyter notebook tutorial
• Basics of linear algebra
  • Linear Algebra for Machine Learning

Small portions of the lectures and exercises are dedicated to refreshing these concepts as well!

Lecture materials

• Lecture 0 - Introduction & Roadmap [slides]
• Lecture 1 - Introduction to Machine Learning [slides]
  • Practical session [slides]
  • Learning Rate and Optimization Demo [JupyterNotebook]
  • Introduction to Jupyter Notebooks [JupyterNotebook]
  • Introduction to Python and Numpy [JupyterNotebook]
  • Linear Classification by Regression [JupyterNotebook]
  • Binary Linear Classification [JupyterNotebook]
  • Linear Classification on Spiral Data [JupyterNotebook]
  • Supplementary Material [slides]
• Lecture 2 - Neural Networks [slides]
  • Practical session [slides]
  • Neural Networks on Spiral Data [JupyterNotebook]
  • Data [.zip]
  • Sentiment Classification using Neural Networks [JupyterNotebook]
  • Neural Network Language Model [JupyterNotebook]
• Lecture 3 - Recurrent Neural Networks [slides]
  • Recurrent Neural Networks [JupyterNotebook]
  • Hybrid Model [JupyterNotebook]
• Lecture 4 - Sequence to Sequence Models and Practical Considerations
  • Sequence to Sequence Models [slides]
  • Practical Considerations [slides]
  • Per Timestep Prediction [JupyterNotebook]
  • Pretrained Embeddings [JupyterNotebook]
  • Imbalanced Classes [JupyterNotebook]
• Lecture 5 - Advanced Topics, CNN, Multitask, GAN, RL, etc. [slides]