Deep Learning Basics
The List of deep learning tutorials we have read for Learning deep learning
Last Edit of this page: Jan 2018.
- Topic I: Deep Learning Basics
- Topic II: Deep Learning Scale-Up and Benchmarking
- Topic III: Basics of Adversarial / Deep Generative Models
- Topic IV: Basic Variations (Structures/Training/...) of Deep Models
- Topic V: Basic Optimization for Deep Learning
- Topic VI: Basic Deep Reinforcement Learning
Topic I: Deep Learning Basics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
|---|---|---|---|
| ✓DeepBasic | [book] Deep Learning — Ian Goodfellow, Yoshua Bengio and Aaron Courville | (bookLink) | 2016 |
| ✓DeepBasic | [Coursera] Deep Learning — Andrew Ng | (CourseURL) | 2016 |
| ✓DeepBasic | [Coursera] Neural Networks for Machine Learning — Geoffrey Hinton | (ListVideo) | 2016 |
| ✓DeepBasic | Nando de Freitas: Course: Deep learning at Oxford 2015 | (ListVideo) | 2015 |
| ✓Deep | DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part1 - deepNN supervised) | (Video) + (PDFslide) | 2012 |
| ✓Deep | DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part2 - deepNN unsupervised) | (Video) + (PDFslide) | 2012 |
| ✓DeepStructured | DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part3 - deepNN graph transformer network) | (Video) + (PDFslide) + (PDF2) | 2012 |
| ✓Deep | DLSS15: Leon Bottou: Multilayer Neural Networks | (Video) + (Slide) | 2015 |
| ✓Deep | Yoshua Bengio (University of Montreal): Representation Learning with auto-encoder / decoder variants | (Video) | 2012 |
| ✓DeepGM | DeepLearningSummerSchool12: Geoffrey Hinton: Introduction to Deep Learning , Deep Belief Nets (Parts 1 / Relevant Paper: A fast learning algorithm for deep belief nets ) | (Video) + (PDFslide) | 2012 |
| ✓Deep | MLSS2005: Yann Lecun, Tutorial of Energy-based models | (Video) +(Slide) | 2005 |
| ✓Deep | Geoffry Hinton: Learning Energy-Based Models of High-Dimensional Data | (Video) + (PDFslide) | 2012 |
| ✓Deep | EML07: Yoshua Bengio (University of Montreal): Speeding Up Stochastic Gradient Descent | (Video) + (PDFslide) | 2007 |
| ✓Deep | Matt Zeiler ( Founder and CEO of Clarifai Inc, ) : Visualizing and Understanding Deep Neural Networks | (Video) | 2015 |
| ✓DeepTheory | DeepLearningSummerSchool12: Nando de Freitas (University of British Columbia) An Informal Mathematical Tour of Feature Learning | (Video) + (PDFslide) | 2012 |
| ✓Deep | KDD14: Ruslan Salakhutdinov, Deep Learning | (Video) + (PDFslide) | 2014 |
| ✓RNN | Nando de Freitas: Deep Learning Lecture 12: Recurrent Neural Nets and LSTMs | (Video) + (Slide) | 2015 |
| ✓DeepGenerative | Hugo Larochelle: DLSS15: Deep Learning for Distribution Estimation | (Video) + (Slide) | 2015 |
| ✓DeepTheory | Yoshua Bengio: DLSS15: Deep Learning: Theoretical Motivations | (Video) + (Slide) | 2015 |
| ✓App | DLSS15: Christopher Manning: NLP and Deep Learning 2: Compositional Deep Learning; Graham Taylor, Deep Learning to compared |
(Video1) (Video2) (Slide) + (VideoLearn2Compare) | 2015 |
| DeepRBM | DLSS15: multiple tutorials related to RBM | (Video-DeepGM) + (Video-RBM) + (Video-deepRBM) | 2015 |
Topic II: Deep Learning Scaling and Benchmarking | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓DeepScaling | KDD14: Yoshua Bengio, Scaling Up Deep Learning | (Video) + (PDFslide) | 2014 |
| ✓Tools | DeepLearningSummerSchool16: Jeffrey Dean, Google, Inc. : Large Scale Deep Learning with TensorFlow | (Video) + (PDFslide) | 2016 |
| ✓bench | paper: Comparative Study of Deep Learning Software Frameworks | (PDF) | 2015 |
| ✓bench | paper: Benchmarking State-of-the-Art Deep Learning Software Tools | (PDF) | 2016 |
| ✓Tools | DeepLearningSummerSchool16: Alex Wiltschko, Twitter, Inc : Introduction to Torch | (Video) + (PDFslide) | 2016 |
| ✓Hardware | DeepLearningSummerSchool12: Marc'Aurelio Ranzato (Google Inc.), Large Scale Deep Learning | (Video) + (PDFslide) | 2012 |
| ✓Hardware | DeepLearningSummerSchool16:Ryan Olson, NVIDIA Corporation, GPU programming for Deep Learning | (Video) + (PDFslide) | 2016 |
| ✓BasicLarge | Sanjiv Kumar (Columbia EECS 6898), Lecture: Introduction to large-scale machine learning | (PDFSlide) | 2010 |
| ✓BasicLarge | Alex Smola - Berkeley SML: Scalable Machine Learning: Syllabus | (SyllabusURL) | 2012 |
| ✓BasicLarge | William Cohen - CMU Machine Learning with Large Datasets 10-605: Syllabus | (SyllabusURL) | 2014 |
Topic III: Adversarial and Deep Generative Adversarial Basic Papers we read | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓Deep | DLSS15: Ian Goodfellow: Deep Adversarial Examples | (Video) + (Slide) | 2015 |
| ✓GAN | Paper: Generative Adversarial Networks / Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio | (PDF) | 2014 |
| ✓GAN | Paper: Energy Based Generative Adversarial Networks / Junbo Zhao, Michael Mathieu and Yann LeCun | (PDF) | 2017 |
| ✓GAN | Paper: Towards Principled Methods for Training Generative Adversarial Networks /Martin Arjovsky, Soumith Chintala, Léon Bottou | (PDF) | 2017 |
| ✓GAN | Paper: Wasserstein GAN /Martin Arjovsky, Léon Bottou | (PDF) | 2017 |
| ✓DeepGenerative | Yoshua Bengio: DLSS15: Deep Generative Models | (Video) + (Slide) | 2015 |
| ✓Deep | DSLL16: Shakir Mohamed, Google, Inc, Building Machines that Imagine and Reason: Principles and Applications of Deep Generative Models | (Video) + (PDFslide) | 2016 |
| ✓Deep | Paper: A Neural Algorithm of Artistic Style | (Arxiv) + (Github) | 2016 |
| ✓Robust | Paper: Intriguing properties of neural networks / Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus | (PDF) | 2013 |
| ✓Robust | Paper: Explaining and Harnessing Adversarial Examples / Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy | (PDF) | 2014 |
Topic IV: Basic Deep learning Models with varying Structures | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓RNN | Paper: Long Short-Term Memory | (PDF) (BlogExplain) | 1997 |
| ✓NTM | Paper: Neural Turing Machines / Alex Graves, Greg Wayne, Ivo Danihelka | (PDF) | 2015 |
| ✓NTM | Paper: Hybrid computing using a neural network with dynamic external memory/ Alex Graves, etal, Koray Kavukcuoglu, Demis Hassabis | (PDF) | 2015 |
| ✓Memory | Paper: End-To-End Memory Networks / Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus | (PDF) | 2015 |
| ✓s2sLSTM | Paper: Sequence to Sequence Learning with Neural Networks / Ilya Sutskever, Oriol Vinyals, Quoc V. Le | (PDF) | 2015 |
| ✓s2sLSTM | Paper: Neural Machine Translation by Jointly Learning to Align and Translate / Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio | (PDF) | 2016 |
| ✓pointerLSTM | Paper: Pointer Networks /Oriol Vinyals, Meire Fortunato, Navdeep Jaitly | (PDF) | 2015 |
| ✓s2sLSTM | Paper: Order Matters: Sequence to Sequence for Sets / Oriol Vinyals, Samy Bengio, Manjunath Kudlur | (PDF) | 2016 |
| ✓Deep | DSLL16: Edward Grefenstette, Google, Inc. : Beyond Seq2Seq with Augmented RNNs | (Video) + (PDFslide) | 2016 |
| ✓Attention | DSLL16: Sumit Chopra, Facebook Reasoning, Attention and Memory | (Video) + (PDFslide) | 2016 |
| ✓Recurrent | DSLL16: Yoshua Bengio, Department of Computer Science and Operations Research, University of Montreal; A Brief Review of Recurrent Neural Networks | (Video) + (PDFslide) | 2016 |
| ✓OneShot | Paper: Matching Networks for One Shot Learning / Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra | (PDF) | 2016 |
| ✓OneShot | Paper: One-shot Learning with Memory-Augmented Neural Networks / Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap | (PDF)(ICML16) | 2016 |
| ✓HyperNetworks | Paper: HyperNetworks / David Ha, Andrew Dai, Quoc V. Le / ICLR 2017 | (PDF) | 2016 |
| DeepRBM | DLSS15: Aaron Courville, Variational Autoencoder and Extensions | (Video-autoencoder) + (Video-variational) | 2015 |
Topic V: Optimization of Deep learning Models | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓SGD | Tutorial: Optimization Methods for Large-Scale Machine Learning / Léon Bottou, Frank E. Curtis, Jorge Nocedal | (PDF) | 2016 |
| ✓SGD | Tutorial: Efficient BackProp/ Yann Lecun, Leon Bottou, Genevieve Orr, Klaus-Robert Muler | (PDF) | 1998 |
| ✓DeepOptim | DLSS15: Ian Goodfellow: Tutorial on Neural Network Optimization Problems | (Video) + (Slide) | 2015 |
| ✓DeepOptim | DLSS15: Adam Coates: Deep Learning (hopefully faster) | (Video) + (Slide) | 2015 |
Topic VI: Reinforcement Learning Related Basics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓ BasicRL | Basics of Reinforcement Learning: Michael Littman, on Conference on Reinforcement Learning and Decision Making (RLDM)15 | (Video) + (Slide) | 2015 |
| ✓BasicRL | David Silver Course on Reinforcement Learning (10 lectures) | (ListVideo)+ (Slide) | 2015 |
| ✓ RLFunc | NIPS15 Tutorial: Introduction to Reinforcement Learning with Function Approximation | (Video) + (Slide) | 2015 |
| ✓ DeepRL | David Silver: Deep Reinforcement Learning | (Video) + (Slide) | 2015 |
| ✓DeepRL | Nando de Freitas: Deep Learning Lecture 15: Deep Reinforcement Learning - Policy search | (Video) + (Slide) | 2015 |
| DeepRL | Nando de Freitas: Deep Learning Lecture 16: Reinforcement learning and neuro-dynamic programming | (Video) + (Slide) | 2015 |
| many more exciting video tutorials @ http://videolectures.net | |||