Recent Readings for Basic Topics of Deep Neural Networks (since 2017)


0Basics (Index of Posts):

No. Read Date Title and Information We Read @
1 2019, Nov, 3 A general survey 2019-fall Course
2 2019, Jan, 25 GNN Basics I - Deep Learning Advances on Graphs 2019-W1
3 2018, Feb, 20 Survey18- My Survey Talk at UVA HMI Seminar - A quick and rough overview of DNN 2018-me
4 2017, Aug, 31 Generative I - GAN tutorial by Ian Goodfellow 2017-W2
5 2017, Aug, 29 Reinforcement I - Pineau - RL Basic Concepts 2017-W2
6 2017, Aug, 22 Basic17 -Andrew Ng - Nuts and Bolts of Applying Deep Learning 2017-W1
7 2017, Jan, 20 Basic16- DNN to be Scalable 2017-team
8 2017, Jan, 19 Basic16- Basic Deep NN and Robustness 2017-team
9 2017, Jan, 18 Basic16- Basic Deep NN with Memory 2017-team
10 2017, Jan, 12 Basic16- Basic DNN Embedding we read for Ranking/QA 2017-team
11 2017, Jan, 12 Basic16- Basic DNN Reads we finished for NLP/Text 2017-team

[1]: Basic16- Basic DNN Reads we finished for NLP/Text

0Basics 2Architecture 9DiscreteApp embedding text BERT seq2seq attention NLP curriculum BackProp relational
Presenter Papers Paper URL Our Slides
NLP A Neural Probabilistic Language Model PDF  
Text Bag of Tricks for Efficient Text Classification PDF  
Text Character-level Convolutional Networks for Text Classification PDF  
NLP BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding PDF  
seq2seq Neural Machine Translation by Jointly Learning to Align and Translate PDF  
NLP Natural Language Processing (almost) from Scratch PDF  
Train Curriculum learning PDF  
Muthu NeuroIPS Embedding Papers survey 2012 to 2015 NIPS PDF
Basics Efficient BackProp PDF  

[2]: Basic16- Basic DNN Embedding we read for Ranking/QA

0Basics 2Architecture 9DiscreteApp embedding recommendation QA graph relational
Presenter Papers Paper URL Our Slides
QA Learning to rank with (a lot of) word features PDF  
Relation A semantic matching energy function for learning with multi-relational data PDF  
Relation Translating embeddings for modeling multi-relational data PDF  
QA Reading wikipedia to answer open-domain questions PDF  
QA Question answering with subgraph embeddings PDF  

[3]: Basic16- Basic Deep NN with Memory

0Basics 2Architecture 7MetaDomain memory NTM seq2seq pointer set attention meta-learning Few-Shot matching net metric-learning
Presenter Papers Paper URL Our Slides
seq2seq Sequence to Sequence Learning with Neural Networks PDF  
Set Pointer Networks PDF  
Set Order Matters: Sequence to Sequence for Sets PDF  
Point Attention Multiple Object Recognition with Visual Attention PDF  
Memory End-To-End Memory Networks PDF Jack Survey
Memory Neural Turing Machines PDF  
Memory Hybrid computing using a neural network with dynamic external memory PDF  
Muthu Matching Networks for One Shot Learning (NIPS16) 1 PDF PDF
Jack Meta-Learning with Memory-Augmented Neural Networks (ICML16) 2 PDF PDF
Metric ICML07 Best Paper - Information-Theoretic Metric Learning PDF  

[4]: Basic16- Basic Deep NN and Robustness

0Basics 3Reliable Adversarial-Examples robustness visualizing Interpretable Certified-Defense
Presenter Papers Paper URL Our Slides
AE Intriguing properties of neural networks / PDF  
AE Explaining and Harnessing Adversarial Examples PDF  
AE Towards Deep Learning Models Resistant to Adversarial Attacks PDF  
AE DeepFool: a simple and accurate method to fool deep neural networks PDF  
AE Towards Evaluating the Robustness of Neural Networks by Carlini and Wagner PDF PDF
Data Basic Survey of ImageNet - LSVRC competition URL PDF
Understand Understanding Black-box Predictions via Influence Functions PDF  
Understand Deep inside convolutional networks: Visualising image classification models and saliency maps PDF  
Understand BeenKim, Interpretable Machine Learning, ICML17 Tutorial [^1] PDF  
provable Provable defenses against adversarial examples via the convex outer adversarial polytope, Eric Wong, J. Zico Kolter, URL  

[5]: Basic16- DNN to be Scalable

0Basics 2Architecture 8Scalable scalable random sparsity binary hash compression low-rank distributed dimension reduction pruning sketch Parallel
Presenter Papers Paper URL Our Slides
scalable Sanjiv Kumar (Columbia EECS 6898), Lecture: Introduction to large-scale machine learning 2010 [^1] PDF  
data scalable Alex Smola - Berkeley SML: Scalable Machine Learning: Syllabus 2012 [^2] PDF 2014 + PDF  
Binary Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1    
Model Binary embeddings with structured hashed projections 1 PDF PDF
Model Deep Compression: Compressing Deep Neural Networks (ICLR 2016) 2 PDF PDF

[6]: Basic17 -Andrew Ng - Nuts and Bolts of Applying Deep Learning

0Basics bias-variance
Presenter Papers Paper URL Our Slides
NIPS16 Andrew Ng - Nuts and Bolts of Applying Deep Learning: 1 video    
DLSS17 Doina Precup - Machine Learning - Bayesian Views (56:50m to 1:04:45 slides) video + slide    

[7]: Reinforcement I - Pineau - RL Basic Concepts

6Reinforcement 0Basics RL

Pineau - RL Basic Concepts

Presenter Papers Paper URL Our Slides
DLSS16 video    
RLSS17 slideRaw + video+ slide    

[8]: Generative I - GAN tutorial by Ian Goodfellow

5Generative 0Basics generative GAN
Presenter Papers Paper URL Our Slides
NIPS 2016 ganerative adversarial network tutorial (NIPS 2016) paper + video + code  
DLSS 2017 Generative Models I - DLSS 2017 slideraw + video + slide  

[9]: Survey18- My Survey Talk at UVA HMI Seminar - A quick and rough overview of DNN

0Basics
Presenter Papers Paper URL Our Slides
Dr. Qi A quick and rough survey of Deep-Neural-Networks   PDF

[10]: GNN Basics I - Deep Learning Advances on Graphs

2Graphs 0Basics 8Scalable invariant scalable embedding
Presenter Papers Paper URL Our Notes
Basics GraphSAGE: Large-scale Graph Representation Learning by Jure Leskovec Stanford University URL + PDF  
Basics Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering by Xavier Bresson URL + PDF Ryan Pdf
Basics Gated Graph Sequence Neural Networks by Microsoft Research URL + PDF Faizan Pdf
Basics DeepWalk - Turning Graphs into Features via Network Embeddings URL + PDF  
Basics Spectral Networks and Locally Connected Networks on Graphs 1 Pdf GaoJi slides + Bill Pdf
Basics A Comprehensive Survey on Graph Neural Networks/ Graph Neural Networks: A Review of Methods and Applications Pdf Jack Pdf
GCN Semi-Supervised Classification with Graph Convolutional Networks Pdf Jack Pdf

[11]: A general survey

0Basics tutorial
Presenter Papers Paper URL Our Slides
Dr Qi Survey of Recent DeepLearning to 12 Groups / PDF    
BackTop