Our Reviews of Deep Learning Readings by Date-Read


Foundations I -Andrew Ng - Nuts and Bolts of Applying Deep Learning

NIPS16 Andrew Ng - Nuts and Bolts of Applying Deep Learning: video
DLSS17 Doina Precup - Machine Learning - Bayesian Views (56:50m to 1:04:45 slides) video + slide

Foundations II - Ganguli - Theoretical Neuroscience and Deep Learning DLSS16

Ganguli - Theoretical Neuroscience and Deep Learning

DLSS16 video
DLSS17 video + slide

Reinforcement I - Pineau - RL Basic Concepts

Pineau - RL Basic Concepts

DLSS16 video
RLSS17 slideRaw + video+ slide

Generative I - GAN tutorial by Ian Goodfellow

GAN tutorial (NIPS 2016) paper + video + code
Generative Models I - DLSS 2017 slideraw + video + slide

Foundations III - Investigating Behaviors of DNN

Presenter Papers Information OurPresentation
Rita On the Expressive Power of Deep Neural Networks PDF PDF
Arshdeep Understanding deep learning requires rethinking generalization, ICLR17 PDF PDF
Tianlu On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima, ICLR17 PDF PDF

Foundations IV - Investigating Behaviors of DNN

Presenter Papers Information OurPresentation
Rita Learning Kernels with Random Features, Aman Sinha*; John Duchi, PDF PDF
Beilun Learning Deep Parsimonious Representations, NIPS16 PDF PDF
Jack Dense Associative Memory for Pattern Recognition, NIPS16 PDF + video PDF
  On the Expressive Efficiency of Overlapping Architectures of Deep Learning DLSSpdf + video  

Foundations V - More about Behaviors of DNN

Presenter Papers Information OurPresentation
Tianlu Large-Scale Evolution of Image Classifiers, ICML17 PDF PDF
Ceyer A Closer Look at Memorization in Deep Networks, ICML17 PDF PDF
Bargav Learning Structured Sparsity in Deep Neural Networks, NIPS16 PDF PDF
Arshdeep Learning the Number of Neurons in Deep Networks, NIPS16 PDF PDF

Foundations VI - More about Behaviors of DNN

Presenter Papers Information OurPresentation
SE Equivariance Through Parameter-Sharing, ICML17 PDF  
SE Why Deep Neural Networks for Function Approximation?, ICLR17 PDF  
SE Geometry of Neural Network Loss Surfaces via Random Matrix Theory, ICML17 PDF  
SE Deep learning in the brain DLSS17 + Video  

Structure I - Varying DNN structures for an application

Presenter Papers Information OurPresentation
Jack Learning End-to-End Goal-Oriented Dialog, ICLR17 PDF PDF
Arshdeep Making Neural Programming Architectures Generalize via Recursion, ICLR17 PDF PDF
Bargav Nonparametric Neural Networks, ICLR17 PDF PDF

Structure II - DNN with Varying Structures

Presenter Papers Information OurPresentation
Shijia Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, (Dean), ICLR17 PDF PDF
Xueying Towards Deep Interpretability (MUS-ROVER II): Learning Hierarchical Representations of Tonal Music, ICLR17 PDF PDF
Ceyer Sequence Modeling via Segmentations, ICML17 PDF PDF
Arshdeep Input Switched Affine Networks: An RNN Architecture Designed for Interpretability, ICML17 PDF PDF

Structure III - DNN with Attention

Presenter Papers Information OurPresentation
Rita Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer, ICLR17 PDF PDF
Tianlu Dynamic Coattention Networks For Question Answering, ICLR17 PDF + code PDF
ChaoJiang Structured Attention Networks, ICLR17 PDF + code PDF

Structure IV - DNN with Attention 2

Presenter Papers Information OurPresentation
Jack Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain, ICLR17 PDF PDF
Arshdeep Bidirectional Attention Flow for Machine Comprehension, ICLR17 PDF + code PDF
Ceyer Image-to-Markup Generation with Coarse-to-Fine Attention, ICML17 PDF + code PDF
ChaoJiang Can Active Memory Replace Attention? ; Samy Bengio, NIPS16 PDF PDF
  An Information-Theoretic Framework for Fast and Robust Unsupervised Learning via Neural Population Infomax, ICLR17 PDF  

Structure V - DNN with Memory

Presenter Papers Information OurPresentation
Tianlu Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, ICML17 PDF + code PDF
Jack Reasoning with Memory Augmented Neural Networks for Language Comprehension, ICLR17 PDF PDF
Xueying State-Frequency Memory Recurrent Neural Networks, ICML17 PDF PDF

Structure VI - DNN with Adaptive Structures

Presenter Papers Information OurPresentation
Anant AdaNet: Adaptive Structural Learning of Artificial Neural Networks, ICML17 PDF PDF
Shijia SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization, ICML17 PDF PDF
  Optimal Architectures in a Solvable Model of Deep Networks, NIPS16 PDF  

Reliable Applications I - Understanding

Presenter Papers Information OurPresentation
Rita Learning Important Features Through Propagating Activation Differences, ICML17 PDF PDF
Ji Examples are not Enough, Learn to Criticize! Model Criticism for Interpretable Machine Learning, NIPS16 PDF PDF
Jack Proximal Deep Structured Models, NIPS16 PDF  

Reliable Applications II - Data

Presenter Papers Information OurPresentation
Xueying Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data, ICLR17 PDF PDF
Bargav Deep Learning with Differential Privacy, CCS16 PDF + video PDF
Bargav Privacy-Preserving Deep Learning, CCS15 PDF PDF

Reliable Applications III - Data

Presenter Papers Information OurPresentation
Jack Learning to Query, Reason, and Answer Questions On Ambiguous Texts, ICLR17 PDF PDF
Beilun Conditional Image Generation with Pixel CNN Decoders, NIPS16 PDF PDF

Reliable Applications IV - Robustness to Data

Presenter Papers Information OurPresentation
Ji Delving into Transferable Adversarial Examples and Black-box Attacks,ICLR17 pdf PDF
Shijia On Detecting Adversarial Perturbations, ICLR17 pdf PDF
Anant Parseval Networks: Improving Robustness to Adversarial Examples, ICML17 pdf PDF
Bargav Being Robust (in High Dimensions) Can Be Practical, ICML17 pdf PDF
  Data Noising as Smoothing in Neural Network Language Models (Ng), ICLR17 pdf  

Reliable Applications V - Understanding

Presenter Papers Information OurPresentation
ChaoJiang Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity, NIPS16 PDF PDF
Rita Visualizing Deep Neural Network Decisions: Prediction Difference Analysis, ICLR17 PDF PDF
Xueying Domain Separation Networks, NIPS16 PDF PDF
  The Robustness of Estimator Composition, NIPS16 PDF  

Reliable Applications VI - Robustness

Presenter Papers Information OurPresentation
Tianlu Robustness of classifiers: from adversarial to random noise, NIPS16 PDF PDF
Anant Blind Attacks on Machine Learners, NIPS16 PDF PDF
Arshdeep Axiomatic Attribution for Deep Networks, ICML17 PDF PDF

Optimization I - Understanding DNN Optimization

Presenter Papers Information OurPresentation
Ceyer An overview of gradient optimization algorithms, PDF PDF
Shijia Osborne - Probabilistic numerics for deep learning DLSS 2017 + Video PDF
Jack Automated Curriculum Learning for Neural Networks, ICML17 PDF PDF
  Johnson - Automatic Differentiation slide + video  

Optimization II - DNN for Optimization

Presenter Papers Information OurPresentation
Ji Neural Architecture Search with Reinforcement Learning, ICLR17 PDF PDF
Ceyer Learning to learn DLSS17video+ PDF  
Beilun Optimization as a Model for Few-Shot Learning, ICLR17 PDF + More PDF
  Batched High-dimensional Bayesian Optimization via Structural Kernel Learning PDF  

Optimization III - Optimization for DNN

Presenter Papers Information OurPresentation
Ji Forward and Reverse Gradient-Based Hyperparameter Optimization, ICML17 PDF  
Chaojiang Adaptive Neural Networks for Efficient Inference, ICML17 PDF  
Bargav Practical Gauss-Newton Optimisation for Deep Learning, ICML17 PDF  
Rita How to Escape Saddle Points Efficiently, ICML17 PDF  
Beilun+Arshdeep Mollifying Networks, Bengio, ICLR17 PDF  

Optimization IV - DNN for Optimization 2

Presenter Papers Information OurPresentation
Anant Neural Optimizer Search with Reinforcement Learning, ICML17 PDF  
Shijia Professor Forcing: A New Algorithm for Training Recurrent Networks, NIPS16 PDF + Video  
  Sharp Minima Can Generalize For Deep Nets, ICML17 PDF  

Generative II - Deep Generative Models

Presenter Papers Information OurPresentation
ChaoJiang Courville - Generative Models II DLSS17Slide + video  
Ji Attend, Infer, Repeat: Fast Scene Understanding with Generative Models, NIPS16 PDF + talk  
Anant Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy, ICLR17 PDF + code  
Arshdeep Composing graphical models with neural networks for structured representations and fast inference, NIPS16 PDF PDF
Shijia Marrying Graphical Models & Deep Learning DLSS17 + Video  
  Johnson - Graphical Models and Deep Learning DLSSSlide + video  
  Parallel Multiscale Autoregressive Density Estimation, ICML17 PDF  

Generative III - GAN and More

Presenter Papers Information OurPresentation
Shijia Marrying Graphical Models & Deep Learning DLSS17 + Video  
Arshdeep Generalization and Equilibrium in Generative Adversarial Nets (ICML17) PDF + video  
Arshdeep Mode Regularized Generative Adversarial Networks (ICLR17) PDF  
Bargav Improving Generative Adversarial Networks with Denoising Feature Matching, ICLR17 PDF  
Anant Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy, ICLR17 PDF + code  
  McGan: Mean and Covariance Feature Matching GAN, PMLR 70:2527-2535 PDF  
  Wasserstein GAN, ICML17 PDF  
  Stochastic Generative Hashing, ICML17 PDF  
  Robust Structured Estimation with Single-Index Models, ICML17 PDF  
  Measuring Sample Quality with Kernels, NIPS16 PDF  

RL II - Basic tutorial RLSS17

Presenter Papers Information OurPresentation
Jack Hasselt - Deep Reinforcement Learning RLSS17.pdf + video  
Tianlu Roux - RL in the Industry RLSS17.pdf + video  
Xueying Singh - Steps Towards Continual Learning pdf + video  
Ji Distral: Robust Multitask Reinforcement Learning PDF  

RL III - Basic tutorial RLSS17 (2)

Presenter Papers Information OurPresentation
Anant The Predictron: End-to-End Learning and Planning, ICLR17 PDF  
ChaoJiang Szepesvari - Theory of RL RLSS.pdf + Video  
Ji Mastering the game of Go without human knowledge / Nature 2017 PDF  
  Thomas - Safe Reinforcement Learning RLSS17.pdf + video  
  Sutton - Temporal-Difference Learning RLSS17.pdf + Video  

RL IV - RL with varying structures

Presenter Papers Information OurPresentation
Ceyer Reinforcement Learning with Unsupervised Auxiliary Tasks, ICLR17 PDF  
Beilun Why is Posterior Sampling Better than Optimism for Reinforcement Learning? Ian Osband, Benjamin Van Roy PDF  
Ji Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction, ICML17 PDF  
Xueying End-to-End Differentiable Adversarial Imitation Learning, ICML17 PDF  
  Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs, ICML17 PDF  
  Cooperative Visual Dialogue with Deep RL RLSS17pdf + video  
  FeUdal Networks for Hierarchical Reinforcement Learning, ICML17 PDF  
BackTop