Here are Papers We Reviewed. We are science curious.

8Scalable

Recent Readings about Making Learning / DNN Scalable (since 2017) (Index of Posts):

No. Read Date Title and Information We Read @
1 2020, Jan, 5 GNN and Transformer 2020-W1
2 2019, Dec, 6 deep2reproduce 2019 Fall - 8Scalable papers 2019-fall Students deep2reproduce
3 2019, Mar, 25 Edge and Dynamic computing 2019-W10
4 2019, Mar, 22 GNN and scalable 2019-W9
5 2019, Jan, 25 GNN Basics I - Deep Learning Advances on Graphs 2019-W1
6 2018, Oct, 25 Structure18- DNNs Varying Structures 2018-team
7 2018, Aug, 27 Application18- A few DNN for Question Answering 2018-team
8 2017, Oct, 5 Structure VI - DNN with Adaptive Structures 2017-W7
9 2017, Sep, 21 Structure II - DNN with Varying Structures 2017-W5
10 2017, Sep, 19 Structure I - Varying DNN structures 2017-W5
11 2017, Jun, 22 Structures17 - Adaptive Deep Networks II 2017-team
12 2017, Jun, 2 Structures17 -Adaptive Deep Networks I 2017-team
13 2017, Jan, 20 Basic16- DNN to be Scalable 2017-team


Here is a detailed list of posts!


[1]: GNN and Transformer

read on: - 05 Jan 2020
GCN graph attention
Index Papers Our Slides
1 Graph Convolutions: More than You Wanted to Know Derrick Survey
2 Spectral Graph Sparsification Derrick Survey
3 Complexity Analysis of Graph Convolutional Networks and in Attention based GNN Derrick Survey
4 PyTorch-BigGraph: A Large-Scale Graph Embedding System Derrick Survey
5 Scalable GNN Updates: More About PyTorch Geometric (PyG) Derrick Survey
6 Time and Space Complexity of Graph Convolutional Networks Derrick Survey
7 Large Scale GNN and Transformer Models and for Genomics Jack Survey
8 Long Range Attention and Visualizing BERT Jak Survey
9 Benchmarking Graph Neural Networks Sanchit Survey

[2]: deep2reproduce 2019 Fall - 8Scalable papers

read on: - 06 Dec 2019
binarization small-data Quantization
Team INDEX Title & Link Tags Our Slide
T33 The High-Dimensional Geometry of Binary Neural Networks Quantization, binarization, scalable OurSlide
T34 Modern Neural Networks Generalize on Small Data Sets small-data, analysis, ensemble OurSlide
T4 Cognitive Scheduler for Heterogeneous High Performance Computing System system-application OurSlide

[3]: Edge and Dynamic computing

read on: - 25 Mar 2019
mobile binary dynamic
Presenter Papers Paper URL Our Slides
Edge MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications PDF  
Edge XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks URL Ryan PDF
Edge DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices Pdf Eamon PDF
Edge Loss-aware Binarization of Deep Networks, ICLR17 PDF Ryan PDF
Edge Espresso: Efficient Forward Propagation for Binary Deep Neural Networks Pdf Eamon PDF
Dynamic Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution PDF Weilin PDF
Dynamic Dynamic Scheduling For Dynamic Control Flow in Deep Learning Systems PDF  
Dynamic Cavs: An Efficient Runtime System for Dynamic Neural Networks Pdf  

[4]: GNN and scalable

read on: - 22 Mar 2019
graph discrete NLP embedding Hierarchical Parallel Distributed dynamic
Presenter Papers Paper URL Our Slides
Scalable FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling Pdf Ryan PDF + Arshdeep Pdf
Scalable MILE: A Multi-Level Framework for Scalable Graph Embedding Pdf Ryan PDF
Scalable LanczosNet: Multi-Scale Deep Graph Convolutional Networks Pdf Ryan PDF
Scalable Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Pdf Derrick PDF
Scalable Towards Federated learning at Scale: System Design URL Derrick PDF
Scalable DNN Dataflow Choice Is Overrated PDF Derrick PDF
Scalable Towards Efficient Large-Scale Graph Neural Network Computing Pdf Derrick PDF
Scalable PyTorch Geometric URL  
Scalable PyTorch BigGraph URL  
Scalable Simplifying Graph Convolutional Networks Pdf  
Scalable Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks Pdf  

[5]: GNN Basics I - Deep Learning Advances on Graphs

read on: - 25 Jan 2019
invariant scalable embedding
Presenter Papers Paper URL Our Notes
Basics GraphSAGE: Large-scale Graph Representation Learning by Jure Leskovec Stanford University URL + PDF  
Basics Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering by Xavier Bresson URL + PDF Ryan Pdf
Basics Gated Graph Sequence Neural Networks by Microsoft Research URL + PDF Faizan Pdf
Basics DeepWalk - Turning Graphs into Features via Network Embeddings URL + PDF  
Basics Spectral Networks and Locally Connected Networks on Graphs 1 Pdf GaoJi slides + Bill Pdf
Basics A Comprehensive Survey on Graph Neural Networks/ Graph Neural Networks: A Review of Methods and Applications Pdf Jack Pdf
GCN Semi-Supervised Classification with Graph Convolutional Networks Pdf Jack Pdf

[6]: Structure18- DNNs Varying Structures

read on: - 25 Oct 2018
Architecture-Search Hyperparameter dynamic
Presenter Papers Paper URL Our Slides
Arshdeep Learning Transferable Architectures for Scalable Image Recognition PDF PDF
Arshdeep FractalNet: Ultra-Deep Neural Networks without Residuals PDF PDF

[7]: Application18- A few DNN for Question Answering

read on: - 27 Aug 2018
trees metric-learning embedding QA
Presenter Papers Paper URL Our Slides
Derrick GloVe: Global Vectors for Word Representation PDF PDF
Derrick PARL.AI: A unified platform for sharing, training and evaluating dialog models across many tasks. URL PDF
Derrick scalable nearest neighbor algorithms for high dimensional data (PAMI14) 1 PDF PDF
Derrick StarSpace: Embed All The Things! PDF PDF
Derrick Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading, Martin Raison, Pierre-Emmanuel Mazaré, Rajarshi Das, Antoine Bordes PDF PDF

[8]: Structure VI - DNN with Adaptive Structures

read on: - 05 Oct 2017
dynamic Architecture Search structured
Presenter Papers Paper URL Our Slides
Anant AdaNet: Adaptive Structural Learning of Artificial Neural Networks, ICML17 1 PDF PDF
Shijia SplitNet: Learning to Semantically Split Deep Networks for Parameter Reduction and Model Parallelization, ICML17 2 PDF PDF
Jack Proximal Deep Structured Models, NIPS16 3 PDF PDF
  Optimal Architectures in a Solvable Model of Deep Networks, NIPS16 4 PDF  
Tianlu Large-Scale Evolution of Image Classifiers, ICML17 5 PDF PDF

[9]: Structure II - DNN with Varying Structures

read on: - 21 Sep 2017
sparsity blocking nonparametric structured QA Interpretable
Presenter Papers Paper URL Our Slides
Shijia Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, (Dean), ICLR17 1 PDF PDF
Ceyer Sequence Modeling via Segmentations, ICML17 2 PDF PDF
Arshdeep Input Switched Affine Networks: An RNN Architecture Designed for Interpretability, ICML17 3 PDF PDF

[10]: Structure I - Varying DNN structures

read on: - 19 Sep 2017
dialog QA nonparametric structured sparsity
Presenter Papers Paper URL Our Slides
Jack Learning End-to-End Goal-Oriented Dialog, ICLR17 1 PDF PDF
Bargav Nonparametric Neural Networks, ICLR17 2 PDF PDF
Bargav Learning Structured Sparsity in Deep Neural Networks, NIPS16 3 PDF PDF
Arshdeep Learning the Number of Neurons in Deep Networks, NIPS16 4 PDF PDF

[11]: Structures17 - Adaptive Deep Networks II

read on: - 22 Jun 2017
low-rank binary dynamic learn2learn optimization
Presenter Papers Paper URL Our Slides
Arshdeep Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction 1 PDF PDF
Arshdeep Decoupled Neural Interfaces Using Synthetic Gradients 2 PDF PDF
Arshdeep Diet Networks: Thin Parameters for Fat Genomics 3 PDF PDF
Arshdeep Metric Learning with Adaptive Density Discrimination 4 PDF PDF

[12]: Structures17 -Adaptive Deep Networks I

read on: - 02 Jun 2017
low-rank binary dynamic learn2learn optimization
Presenter Papers Paper URL Our Slides
Arshdeep HyperNetworks, David Ha, Andrew Dai, Quoc V. Le ICLR 2017 1 PDF PDF
Arshdeep Learning feed-forward one-shot learners 2 PDF PDF
Arshdeep Learning to Learn by gradient descent by gradient descent 3 PDF PDF
Arshdeep Dynamic Filter Networks 4 https://arxiv.org/abs/1605.09673 PDF PDF

[13]: Basic16- DNN to be Scalable

read on: - 20 Jan 2017
scalable random sparsity binary hash compression low-rank distributed dimension reduction pruning sketch Parallel
Presenter Papers Paper URL Our Slides
scalable Sanjiv Kumar (Columbia EECS 6898), Lecture: Introduction to large-scale machine learning 2010 [^1] PDF  
data scalable Alex Smola - Berkeley SML: Scalable Machine Learning: Syllabus 2012 [^2] PDF 2014 + PDF  
Binary Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1    
Model Binary embeddings with structured hashed projections 1 PDF PDF
Model Deep Compression: Compressing Deep Neural Networks (ICLR 2016) 2 PDF PDF

Here is a name list of posts!


GNN and Transformer

less than 1 minute read

Index Papers Our Slides 1 Graph Convolutions: More than You Wanted to Know Derrick Survey ...

Edge and Dynamic computing

less than 1 minute read

Presenter Papers Paper URL Our Slides Edge MobileNets: Efficient Convolutional Neural Networks f...

GNN and scalable

less than 1 minute read

Presenter Papers Paper URL Our Slides Scalable FastGCN: Fast Learning with Graph Convolutional N...

Basic16- DNN to be Scalable

4 minute read

Presenter Papers Paper URL Our Slides scalable Sanjiv Kumar (Columbia EECS 6898), Lecture: Intro...