# My Survey Talk at UVA HMI Seminar - A quick and rough overview of DNN

- 20 Feb 2018Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Dr. Qi | A quick and rough survey of Deep-Neural-Networks | ||

Click on a tag to see relevant list of readings.

No. | Date | Title and Information |
---|---|---|

1 | 2018, Feb, 20 | My Survey Talk at UVA HMI Seminar - A quick and rough overview of DNN |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Dr. Qi | A quick and rough survey of Deep-Neural-Networks | ||

No. | Date | Title and Information |
---|---|---|

1 | 2017, Aug, 22 | Foundations I -Andrew Ng - Nuts and Bolts of Applying Deep Learning |

2 | 2017, Aug, 24 | Foundations II - Ganguli - Theoretical Neuroscience and Deep Learning DLSS16 |

3 | 2017, Sep, 5 | Foundations III - Investigating Behaviors of DNN |

4 | 2017, Sep, 7 | Foundations IV - Investigating Behaviors of DNN |

5 | 2017, Sep, 12 | Foundations V - More about Behaviors of DNN |

6 | 2017, Sep, 14 | Foundations VI - More about Behaviors of DNN |

NIPS16 | Andrew Ng - Nuts and Bolts of Applying Deep Learning: video |

DLSS17 | Doina Precup - Machine Learning - Bayesian Views (56:50m to 1:04:45 slides) video + slide |

DLSS16 | video |

DLSS17 | video + slide |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Rita | On the Expressive Power of Deep Neural Networks | ||

Arshdeep | Understanding deep learning requires rethinking generalization, ICLR17 | ||

Tianlu | On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima, ICLR17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Rita | Learning Kernels with Random Features, Aman Sinha*; John Duchi, | ||

Beilun | Learning Deep Parsimonious Representations, NIPS16 | ||

Jack | Dense Associative Memory for Pattern Recognition, NIPS16 | PDF + video | |

On the Expressive Efficiency of Overlapping Architectures of Deep Learning | DLSSpdf + video |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Tianlu | Large-Scale Evolution of Image Classifiers, ICML17 | ||

Ceyer | A Closer Look at Memorization in Deep Networks, ICML17 | ||

Bargav | Learning Structured Sparsity in Deep Neural Networks, NIPS16 | ||

Arshdeep | Learning the Number of Neurons in Deep Networks, NIPS16 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

SE | Equivariance Through Parameter-Sharing, ICML17 | ||

SE | Why Deep Neural Networks for Function Approximation?, ICLR17 | ||

SE | Geometry of Neural Network Loss Surfaces via Random Matrix Theory, ICML17 | ||

SE | Deep learning in the brain | DLSS17 + Video |

No. | Date | Title and Information |
---|---|---|

1 | 2017, Sep, 19 | Structure I - Varying DNN structures for an application |

2 | 2017, Sep, 21 | Structure II - DNN with Varying Structures |

3 | 2017, Sep, 26 | Structure III - DNN with Attention |

4 | 2017, Sep, 28 | Structure IV - DNN with Attention 2 |

5 | 2017, Oct, 3 | Structure V - DNN with Memory |

6 | 2017, Oct, 5 | Structure VI - DNN with Adaptive Structures |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Jack | Learning End-to-End Goal-Oriented Dialog, ICLR17 | ||

Arshdeep | Making Neural Programming Architectures Generalize via Recursion, ICLR17 | ||

Bargav | Nonparametric Neural Networks, ICLR17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Shijia | Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, (Dean), ICLR17 | ||

Xueying | Towards Deep Interpretability (MUS-ROVER II): Learning Hierarchical Representations of Tonal Music, ICLR17 | ||

Ceyer | Sequence Modeling via Segmentations, ICML17 | ||

Arshdeep | Input Switched Affine Networks: An RNN Architecture Designed for Interpretability, ICML17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Rita | Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer, ICLR17 | ||

Tianlu | Dynamic Coattention Networks For Question Answering, ICLR17 | PDF + code | |

ChaoJiang | Structured Attention Networks, ICLR17 | PDF + code |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Jack | Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain, ICLR17 | ||

Arshdeep | Bidirectional Attention Flow for Machine Comprehension, ICLR17 | PDF + code | |

Ceyer | Image-to-Markup Generation with Coarse-to-Fine Attention, ICML17 | PDF + code | |

ChaoJiang | Can Active Memory Replace Attention? ; Samy Bengio, NIPS16 | ||

An Information-Theoretic Framework for Fast and Robust Unsupervised Learning via Neural Population Infomax, ICLR17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Tianlu | Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, ICML17 | PDF + code | |

Jack | Reasoning with Memory Augmented Neural Networks for Language Comprehension, ICLR17 | ||

Xueying | State-Frequency Memory Recurrent Neural Networks, ICML17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Anant | AdaNet: Adaptive Structural Learning of Artificial Neural Networks, ICML17 | ||

Shijia | SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization, ICML17 | ||

Jack | Proximal Deep Structured Models, NIPS16 | ||

Optimal Architectures in a Solvable Model of Deep Networks, NIPS16 |

No. | Date | Title and Information |
---|---|---|

1 | 2017, Oct, 10 | Reliable Applications I - Understanding |

2 | 2017, Oct, 12 | Reliable Applications II - Data |

3 | 2017, Oct, 17 | Reliable Applications III - Data |

4 | 2017, Oct, 19 | Reliable Applications IV - Robustness to Data |

5 | 2017, Oct, 24 | Reliable Applications V - Understanding |

6 | 2017, Oct, 26 | Reliable Applications VI - Robustness |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Rita | Learning Important Features Through Propagating Activation Differences, ICML17 | ||

Ji | Examples are not Enough, Learn to Criticize! Model Criticism for Interpretable Machine Learning, NIPS16 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Xueying | Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data, ICLR17 | ||

Bargav | Deep Learning with Differential Privacy, CCS16 | PDF + video | |

Bargav | Privacy-Preserving Deep Learning, CCS15 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Jack | Learning to Query, Reason, and Answer Questions On Ambiguous Texts, ICLR17 | ||

Beilun | Conditional Image Generation with Pixel CNN Decoders, NIPS16 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Ji | Delving into Transferable Adversarial Examples and Black-box Attacks,ICLR17 | ||

Shijia | On Detecting Adversarial Perturbations, ICLR17 | ||

Anant | Parseval Networks: Improving Robustness to Adversarial Examples, ICML17 | ||

Bargav | Being Robust (in High Dimensions) Can Be Practical, ICML17 | ||

Data Noising as Smoothing in Neural Network Language Models (Ng), ICLR17 | |||

Measuring Sample Quality with Kernels, NIPS16 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

ChaoJiang | Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity, NIPS16 | ||

Rita | Visualizing Deep Neural Network Decisions: Prediction Difference Analysis, ICLR17 | ||

Xueying | Domain Separation Networks, NIPS16 | ||

The Robustness of Estimator Composition, NIPS16 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Tianlu | Robustness of classifiers: from adversarial to random noise, NIPS16 | ||

Anant | Blind Attacks on Machine Learners, NIPS16 | ||

Arshdeep | Axiomatic Attribution for Deep Networks, ICML17 |

No. | Date | Title and Information |
---|---|---|

1 | 2017, Oct, 31 | Optimization I - Understanding DNN Optimization |

2 | 2017, Nov, 2 | Optimization II - DNN for Optimization |

3 | 2017, Nov, 7 | Optimization III - Optimization for DNN |

4 | 2017, Nov, 9 | Optimization IV - DNN for Optimization 2 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Ceyer | An overview of gradient optimization algorithms, | ||

Shijia | Osborne - Probabilistic numerics for deep learning | DLSS 2017 + Video | PDF / PDF2 |

Jack | Automated Curriculum Learning for Neural Networks, ICML17 | ||

Johnson - Automatic Differentiation | slide + video |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Ji | Neural Architecture Search with Reinforcement Learning, ICLR17 | ||

Ceyer | Learning to learn | DLSS17video | |

Beilun | Optimization as a Model for Few-Shot Learning, ICLR17 | PDF + More | |

Batched High-dimensional Bayesian Optimization via Structural Kernel Learning |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Ji | Forward and Reverse Gradient-Based Hyperparameter Optimization, ICML17 | ||

Chaojiang | Adaptive Neural Networks for Efficient Inference, ICML17 | ||

Bargav | Practical Gauss-Newton Optimisation for Deep Learning, ICML17 | ||

Rita | How to Escape Saddle Points Efficiently, ICML17 | ||

Beilun+Arshdeep | Mollifying Networks, Bengio, ICLR17 | PDF / PDF2 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Anant | Neural Optimizer Search with Reinforcement Learning, ICML17 | ||

Shijia | Professor Forcing: A New Algorithm for Training Recurrent Networks, NIPS16 | PDF + Video | |

Sharp Minima Can Generalize For Deep Nets, ICML17 |

No. | Date | Title and Information |
---|---|---|

1 | 2017, Aug, 31 | Generative I - GAN tutorial by Ian Goodfellow |

2 | 2017, Nov, 14 | Generative II - Deep Generative Models |

3 | 2017, Nov, 16 | Generative III - GAN and More |

4 | 2018, Jan, 10 | Generative - Deep Generative Models |

GAN tutorial (NIPS 2016) | paper + video + code |

Generative Models I - DLSS 2017 | slideraw + video + slide |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

ChaoJiang | Courville - Generative Models II | DLSS17Slide + video | |

Ji | Attend, Infer, Repeat: Fast Scene Understanding with Generative Models, NIPS16 | PDF + talk | |

Arshdeep | Composing graphical models with neural networks for structured representations and fast inference, NIPS16 | ||

Johnson - Graphical Models and Deep Learning | DLSSSlide + video | ||

Parallel Multiscale Autoregressive Density Estimation, ICML17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Shijia | Marrying Graphical Models & Deep Learning | DLSS17 + Video | |

Arshdeep | Generalization and Equilibrium in Generative Adversarial Nets (ICML17) | PDF + video | |

Arshdeep | Mode Regularized Generative Adversarial Networks (ICLR17) | ||

Bargav | Improving Generative Adversarial Networks with Denoising Feature Matching, ICLR17 | ||

Anant | Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy, ICLR17 | PDF + code | |

McGan: Mean and Covariance Feature Matching GAN, PMLR 70:2527-2535 | |||

Wasserstein GAN, ICML17 |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Geometrical Insights for Implicit Generative Modeling, L Bottou, M Arjovsky, D Lopez-Paz, M Oquab |

No. | Date | Title and Information |
---|---|---|

1 | 2017, Aug, 29 | Reinforcement I - Pineau - RL Basic Concepts |

2 | 2017, Nov, 21 | RL II - Basic tutorial RLSS17 |

3 | 2017, Nov, 28 | RL III - Basic tutorial RLSS17 (2) |

4 | 2017, Nov, 30 | RL IV - RL with varying structures |

DLSS16 | video |

RLSS17 | slideRaw + video+ slide |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Jack | Hasselt - Deep Reinforcement Learning | RLSS17.pdf + video | |

Tianlu | Roux - RL in the Industry | RLSS17.pdf + video | PDF / PDF-Bandit |

Xueying | Singh - Steps Towards Continual Learning | pdf + video | |

Ji | Distral: Robust Multitask Reinforcement Learning |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Anant | The Predictron: End-to-End Learning and Planning, ICLR17 | ||

ChaoJiang | Szepesvari - Theory of RL | RLSS.pdf + Video | |

Ji | Mastering the game of Go without human knowledge / Nature 2017 | ||

Thomas - Safe Reinforcement Learning | RLSS17.pdf + video | ||

Sutton - Temporal-Difference Learning | RLSS17.pdf + Video |

Presenter | Papers | Information | OurPresentation |
---|---|---|---|

Ceyer | Reinforcement Learning with Unsupervised Auxiliary Tasks, ICLR17 | ||

Beilun | Why is Posterior Sampling Better than Optimism for Reinforcement Learning? Ian Osband, Benjamin Van Roy | ||

Ji | Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction, ICML17 | ||

Xueying | End-to-End Differentiable Adversarial Imitation Learning, ICML17 | ||

Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs, ICML17 | |||

Cooperative Visual Dialogue with Deep RL | RLSS17pdf + video | ||

FeUdal Networks for Hierarchical Reinforcement Learning, ICML17 |