Machine Learning Basics
Last Edit of this page: Jan 2018.
Three old seminar courses I taught at UVA on machine learning advanced topics before 2017
| 2013 Fall |
Special Seminar: Elements of Statistical Learning (PhD Student level) |
(This course covered the book: The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman). |
| 2014 Spring |
Special seminar: Machine Learning and Data Mining in Practice for Biomedicine </b> |
(This course covers 6 different topics about connecting machine learning and "big data" in biomedicine".) |
| 2015 Spring |
Special Topic: Large-Scale Machine Learning </b> |
(This course covers a list of tutorials to help students learn advanced topics in machine learning.) |
The List of tutorials we have read for Learning Machine Learning
- topic I: Intro. Large Scale Machine Learning Topics
- topic II: Kernel Methods and SVM Topics
- topic III: Optimization for ML and High-Dim Topics
- topic IV: Graphical Model + Bayesian+Variational Topics
- topic V: Deep Learning Topics
- topic VI: Assorted: structured, low-rank, Metric, and more
- topic VII: Assorted: scalable, random, parallel, and more
- topic VIII: Reinforcement Learning related topics
- topic VIIII: Advanced Tutorials helpful to Research
Topic I: Basic Machine Learning Topics |
|||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
|---|---|---|---|
| ✓Basic | Yanjun Qi (UVA CS Under or Master-level), Course: Introduction to machine learning | (ListSlide) | 2018 |
| ✓Basic | Andrew Ng, Course: machine learning | (ListVideo) | 2008 |
| ✓Basic | Nando de Freitas, Course: machine learning (under-level) | (ListVideo) (ListLecture) |
2013 |
| ✓Basic | Nando de Freitas, Course: machine learning (graduate-level) | (ListVideo) (ListLecture) |
2013 |
| ✓Basic | Yaser Abu-Mostafa : Caltech course: Learning from data | (ListVideo) (ListLecture) |
2013 |
| ✓Basic | Roger D. Peng : Johns Hopkins Biostatistics Advanced Statistical Computing course: Advanced Statistical Computing | (BookContent) | 2018 |
Topic I: Basics of Large-Scale Machine Learning Topics |
|||
| ✓BasicLarge | Sanjiv Kumar (Columbia EECS 6898), Lecture: Introduction to large-scale machine learning | (PDFSlide) | 2010 |
| ✓BasicLarge | Alex Smola - Berkeley SML: Scalable Machine Learning: Syllabus | (SyllabusURL) | 2012 |
| ✓BasicLarge | William Cohen - CMU Machine Learning with Large Datasets 10-605: Syllabus | (SyllabusURL) | 2014 |
Topic II: Kernel Methods Topics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓Kernel | Alexander J. Smola, Kernel methods and Support Vector Machines (Part3 ) | (Video) + (PDFslide) | 2008 |
| ✓ParamReduct | Sanjiv Kumar (Columbia EECS 6898), Lecture: Kernel Methods (I: Scaling up kernel methods) | (PDFslide) | 2010 |
| ✓Hardware Parallel | ICML08: Fast Support Vector Machine Training and Classification on Graphics Processors | (Video) + (PDFslide) | 2008 |
| ✓DataStructure | ECML2007: Large Scale Learning with String Kernels, | (Video) + (PDFslide) | 2007 |
| ✓Advanced | Francis R. Bach, INRIA: Multiple kernel learning for multiple sources | (Video) + (PDFslide) | 2008 |
| ✓Random | Sanjiv Kumar (Columbia EECS 6898), Lecture: Randomized Algorithms | (PDFSlide) | 2010 |
| ✓Random | ECML2007: Efficient Machine Learning using Random Projections | (Video) | 2007 |
| ✓Random | NIPS2007: Random features for large-scale kernel machines (original paper PDF) + ECCV2012: Fourier Kernel Learning | (Video) + (PDFslide) | 2007 |
| ✓Random | Fast Random Feature Expansions for Nonlinear Regression | (Video) + (PDFslide) | 2010 |
| ✓FastOptim | NIPS2010: Multiple Kernel Learning and the SMO Algorithm | (Video) + (PDFPaper) | 2010 |
| ✓FastOptim | Fast training of support vector machines using sequential minimal optimization. In Book: Advances in Kernel Methods - Support Vector Learning, MIT Press | (PaperPDF) | 1999 |
| ✓FastOptim | ICML2006: Collbort & Bottou: Trading Convexity for Scalability. | (PaperPDF)+ (Video) | 2006 |
| ✓FastOptim | ICML2008: Bordes & Bottou: LaRank, SGD-QN - Fast Optimizers for Linear SVM | (SlidePDF)+ (Video) | 2006 |
| ✓FastOptim | JMLR 2005: Working Set Selection Using Second Order Information for Training Support Vector Machines | (PaperPDF) | 2005 |
| ✓FastOptim | PEGASOS: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM | (Slide) + (PaperPDF) | 2005 |
| ✓Kernel | NIPS2009: Fast Subtree Kernels on Graphs | (Video) | 2009 |
| ✓Kernel | NIPS09: Locality-Sensitive Binary Codes from Shift-Invariant Kernels | (Video) + (PDF) | 2009 |
| ✓Kernel | PASCAL07: Graph kernels and applications in chemoinformatics | (Video) | 2007 |
| ✓Kernel | S.V.N. Vishwanathan, Random walk graph kernels and rational kernels | (Video) | 2007 |
Topic III: Optimization for ML or High-Dim/Sparsity Topics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓Basic Optim | Professor Stephen Boyd, Stanford University, Stanford EE364a: Convex Optimization I | (CourseList+Video+Lecture) | 2014 |
| ✓Basic Optim | Professor Stephen Boyd, Stanford University, Stanford EE364b: Convex Optimization II | (CourseList+Video+Lecture) | 2008 |
| ✓Basic Optim | Convex Optimization and Applications - Stephen Boyd | (Video) | 2015 |
| ✓Basic Optim | Mark Schmidt's MLSS (machine learning summer school) 2015 tutorial: | (Video) + (Slide) | 2015 |
| ✓Basic Optim | Mark Schmidt's Note: Least Squares Optimization with L1-Norm Regularization | (NotePDF) | 2005 |
| ✓SparsityOptim | KDD08: Trevor Hastie: Regularization Paths and Coordinate Descent | (Video)+ (PDFslide) | 2008 |
| ✓Sparsity | ICML09: Group Lasso with Overlaps and Graph Lasso. (Original Paper PDF) |
(Video) | 2009 |
| ✓ Optim Sparse | Mark Schmidt: Fast Non-Smooth and Big-Data Optimization | (Video) | 2014 |
| ✓FastOptim | Sanjiv Kumar (Columbia EECS 6898), Lecture: Kernel Methods (II: fast optimization of kernel methods) | (PDFslide) | 2010 |
| ✓FastOptim | Sanjiv Kumar (Columbia EECS 6898), Lecture: Large-Scale Optimization Techniques | (PDFslide) | 2010 |
| ✓Optim | MLSS2013: Stephen Wright (University of Wisconsin-Madison) Optimization 1-3 | (video) + (slide) | 2013 |
| ✓Sparsity | Sanjiv Kumar (Columbia EECS 6898), Lecture: Sparse Methods |
(PDFslide) | 2010 |
| ✓NonConvex | Mark Schmidt's DLSS (deep learning summer school) 2015 tutorial: Non Smooth, Non Finite, and Non Convex Optimization | (Video) + (Slide) | 2015 |
| ✓NonConvex | NIPS 2015 Workshop (LeCun) 15599 Non-convex Optimization for Machine Learning: Theory and Practice | (Video) | 2015 |
| ✓NonConvex | NIPS 2015 Workshop (Anandkumar) 15598 Non-convex Optimization for Machine Learning: Theory and and Practice | (Video) | 2015 |
| OptimDiscrete | MLSS2014: Submodularity and Optimization -- Jeff Bilmes | (VideoI-III)+ (PDFslide) | 2014 |
| Optim | DeepSummer12: Jorge Nocedal (Northwestern University) Tutorial on Optimization methods for machine learning | (video) + (PDFslide) | 2012 |
| Optim | DeepSummerSchool12: Stephen Wright (University of Wisconsin-Madison) Some Relevant Topics in Optimization (PartI+II) | (video) + (PDFslide) | 2010 |
| SparsityOptim | DeepSummer12: Stephen Wright (University of Wisconsin-Madison) Sparse and Regularized Optimization | (video) + (PDFslide) | 2012 |
| Sparse | NIPS2009 tutorial: Francis R. Bach: Sparse Methods for Machine Learning: Theory and Algorithms | (video) + (PDFslide) | 2009 |
| OptimAdvance | NIPS10 tutorial: Stephen J. Wright: Optimization Algorithms in Machine Learning Tutorial | (video) + (PDFslide) | 2010 |
| OptimDiscrete | NIPS12: Satoru Fujishige, Submodularity and Discrete Convexity | (Video) | 2012 |
| OptimDiscrete | ICML13: Tutorial, Submodularity In Machine Learning New-Directions | (Video) | 2013 |
| OptimDiscrete | NIPS11: Francis R. Bach, Learning with Submodular Functions: A Convex Optimization Perspective | (Video) | 2011 |
| ✓HighDim | Martin J. Wainwright, High-Dimensional Statistics: Intro @ SimonInstitute Bootcamp | (Video1)(Video2)+ (slide1) (slide2) | 2013 |
| HighDim | NIPS2010: Peter Buhlmann, High-dimensional Statistics: Prediction, Association and Causal Inference | (Video)+ (PDFslide) | 2011 |
| HighDim | Martin J. Wainwright, High-Dimensional Statistics: Some progress and challenges ahead | (PDFslide) | 2010 |
| HighDim | AISTAT11: Martin J. Wainwright, Convex Relaxation and Estimation of High-Dimensional Matrices | (Video)+ (PDFslide) | 2011 |
| MiniMax | ICM2014 VideoSeries IL12.13 : Martin Wainwright on constrained form of statistical MinMax, Privacy, Communication and Computation | (Video) | 2014 |
Topic IV: Graphical Model and Bayesian and Variational Topics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓GM | MLSS2006: Sam Roweis, Machine Learning, Probability and Graphical Models (Part 1-4) | (Video)+ (PDFslide) | 2007 |
| ✓Basic | David MacKay: course: Lecture 10: An Introduction To Bayesian Inference (II): Inference Of Parameters And Models | (Video) + (PDFslide) + (CourseVideoList) | 2012 |
| ✓BasicVariational | David MacKay: course: Lecture 14: Approximating Probability Distributions (IV): Variational Methods | (Video) + (PDFslide) | 2012 |
| ✓BasicMCMC | David MacKay: course: Lecture 12+13: Approximating Probability Distributions (II+III): Monte Carlo Methods (I): Importance Sampling, Rejection Sampling, Gibbs Sampling, Metropolis Method, Slice Sampling, Hybrid Monte Carlo, Over-relaxation, Exact Sampling | (Video12)(Video13) + (slide12)(slide13) | 2012 |
| ✓MCMC Basic | MLSS2009: Iain Murray : Markov Chain Monte Carlo | (Video)+ (Slide) | 2009 |
| ✓GMTopic | MLSS2009: David Blei, Topic Models (Part I+II) | (Video)+ (PDFslide) | 2009 |
| ✓GMBasic | MLSS2012: Martin J. Wainwright: Tutorial Materials on Graphical Models, Variational Methods and Message-Passing (PDFNote) | (Video-07)+ (Part1)+ (Part2)+ (Part3) | 2012 |
| ✓MCMC | MLSS2008: Nando de Freitas, Monte Carlo Simulation for Statistical Inference, Model Selection and Decision Making (Part 1-6) | (Video) + (PDFslide) | 2008 |
| GMExp | Wainwright and Jordan monograph: More advanced material on exponential families, duality, and variational methods |
(PDFpaper) | 2008 |
| GM | MLSS2007: Zoubin Ghahramani, Graphical models (Part 1-6) | (Video)+ (PDFslide) | 2007 |
| GM | MLSS2004: Christopher Bishop: Graphical Models and Variational Methods | (Video)+ (PDFslide) | 2007 |
| Density | DeepSummer12: Iain Murray (University of Edinburgh) Density estimation | (Video)+ (PDFpaper) | 2008 |
| ✓GProcess | Gaussian Processes in Practice Workshop 2006, David MacKay, : Gaussian Process Basics | (Video)+ (PDFslide) + (More) + (AdvBasic) | 2006 |
| DProcess | MLSS12: Dilan Gorur, Yahoo! Research, Dirichlet Process: Practical Course | (Video)+ (PDFpaper) | 2012 |
| Copulas | ICML13 Tutorial: Gal Elidan, Copulas in Machine Learning (Part I+II) | (Video) | 2013 |
| ✓BayesianScaling | (LSOLDM)2013: Nando de Freitas, Bayesian Optimization in a Billion Dimensions via Random Embeddings | (Video)+ (PDFslide) | 2013 |
| ✓GMScaling | KDD2011: Ron Bekkerman, Misha Bilenko and John Langford, Scaling Up Graphical Model Inference | (PDFSlide) | 2011 |
| ✓GMScaling | MLSS2009: Tom Minka, Microsoft Research, Approximate Inference | (Video) + (PDFslide) | 2009 |
| GMScaling | NIPS09: Pedro Domingos, Large-Scale Learning and Inference: What We Have Learned with Markov Logic Networks | (Video)+ (PDFslide) | 2009 |
| ✓GMScaling | KDD14: Pedro Domingos, Principles of Very Large Scale Modeling | (Video) + (PDFslide) | 2014 |
| ✓GMScaling | Ralf Herbrich, Distributed, Real-Time Bayesian Learning in Online Service | (Video) + (PDFpaper) | 2013 |
Topic V: Deep Learning Tutorials before 2016 | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| </a> See Our Site: https://qdata.github.io/deep2Read/ | |||
Topic VI: Assorted: structured, low-rank, Metric, and more | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓Metric | ICML07 Best Paper - Information-Theoretic Metric Learning | (Video) + (PDF) | 2007 |
| ✓Structured | CIKM08: Charles Elkan, Log-linear Models and Conditional Random Fields | (Video) + (PDF) | 2008 |
| ✓Structured | ECML2012: Thomas Gartner, Fraunhofer IAIS , Algorithms for Predicting Structured Data (Part 1-3) | (Video) + (PDF) | 2012 |
| ✓Structured | MLG08: Thorsten Joachims, Structured Output Prediction with Structural SVMs | (Video) + (PDF) | 2008 |
| ✓Matrix | MLSS2009 : Emmanuel Candes, Department of Statistics, Stanford University : Tutorial, Matrix Completion via Convex Optimization: Theory and Algorithms | (Video) | 2009 |
| ✓LowRank | MLSS2011: Emmanuel Candes, Department of Statistics, Stanford University, Title: Low-rank modeling | (Video) + (PDF) | 2011 |
| Matrix | Matrix completion paper list, e.g. singular value thresholding | (PaperList) | 2008-11 |
| ✓LowRank | Sanjiv Kumar (Columbia EECS 6898), Lecture: Matrix Approximations (Part I + Part II) | (PDF-1)+ (PDF-2) | 2010 |
| ✓LowRank | ICML13 Tutorial: Tensor Decomposition Algorithms for Latent Variable Model Estimation | (Video) | 2013 |
| ✓Spectral | Sham Kakade, Scalable Spectral Approaches for Learning Topics, Clusters, and Communities (JMLR paper: Tensor Decompositions for Learning Latent Variable Models) | (Video) + (PDFpaper) | 2014 |
| ✓Spectral | Arik Azran, Department of Engineering, University of Cambridge: Tutorial, Spectral Clustering | (Video) + (PDF) | 2008 |
| ✓DimReduct | Sanjiv Kumar (Columbia EECS 6898), Lecture: Dimensionality Reduction: | (PDF) | 2010 |
| ✓comSensing | MLSS09: Emmanuel Candes, An Overview of Compressed Sensing and Sparse Signal Recovery via L1 Minimization | (Video) | 2009 |
| ✓comSensing | Richard Baraniuk, "Compressive Sensing," ECE Lecturer Series, U.Delware | (Video) | 2012 |
Topic VII: Scalable / Parallel / Random / Streaming Related Topics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓ApproxNN | Sanjiv Kumar (Columbia EECS 6898), Lecture: Approximate Nearest Neighbor Search (Part I + Part II) | (PDF-1)+ (PDF-2) | 2010 |
| ✓scalable | Alex Smola: MLSS 2014: Scalable machine learning | (Video) | 2014 |
| Basic | Alex Smola: Scalable ML Course: statistics | (Video) | 2012 |
| System | Alex Smola: Scalable ML Course: System | (Video) | 2012 |
| StochasticG | Leon Bottou, ICML2016 Tutorial, Stochastic Gradient | (Video) | 2016 |
| Hashing | John Langford, NYU Course on Big Data, Large Scale Machine Learning - Feature Hashing | (Video) | 2012 |
| random | Michael Mahoney on Recent Results in Randomized Numerical Linear Algebra (NIPS 2013 Workshop on Randomized Algorithms) | (Video) | 2013 |
| random | Francis Bach on Beyond stochastic gradient descent for large-scale machine learning (NIPS 2013 Workshop on Randomized Algorithms) | (Video) | 2013 |
| random | Gautam Dasarathy: Sketching Sparse Covariance Matrices (NIPS 2013 Workshop on Randomized Algorithms) | (Video) | 2013 |
Topic VIII: Reinforcement Learning Related Topics | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| ✓ BasicRL | Basics of Reinforcement Learning: Michael Littman, on Conference on Reinforcement Learning and Decision Making (RLDM)15 | (Video) + (Slide) | 2015 |
| ✓BasicRL | David Silver Course on Reinforcement Learning (10 lectures) | (ListVideo)+ (Slide) | 2015 |
| ✓ RLFunc | NIPS15 Tutorial: Introduction to Reinforcement Learning with Function Approximation | (Video) + (Slide) | 2015 |
| ✓ DeepRL | David Silver: Deep Reinforcement Learning | (Video) + (Slide) | 2015 |
| DeepRL | Nando de Freitas: Deep Learning Lecture 15: Deep Reinforcement Learning - Policy search | (Video) + (Slide) | 2015 |
| DeepRL | Nando de Freitas: Deep Learning Lecture 16: Reinforcement learning and neuro-dynamic programming | (Video) + (Slide) | 2015 |
Topic VIIII: Advanced / Recent Tutorials helpful for Research | |||
| Tag | Title and Information | URLs (Paper/Video/Slide) | Year |
| MLAssorted | Simon Institute Worshop: Workshop on Information Theory, Learning and Big Data talk Videos | (VideoList) | 2016 |
| MLAssorted | Simon Institute Big Data Boot Camp talk Videos | (VideoList) | 2015 |
| MLAssorted | NIPS 2016 conference talk Videos | (VideoList) | 2016 |
| MLAssorted | ICML16 conference talk Videos | (VideoList) | 2016 |
| MLAssorted | ICML15 conference talk Videos | (VideoList) | 2015 |
| MLAssorted | NIPS 2015 Workshop talk Videos | (VideoList) | 2015 |
| MLAssorted | AISTAT14 conference talk Videos | (VideoList) | 2014 |
| MLAssorted | KDD 2014 conference talk Videos | (VideoList) | 2014 |
| CBAssorted | Simon Institute Worshop: Network Biology Conference 16 talk Videos | (VideoList) | 2016 |
| CBAssorted | Simon Institute Worshop: Regulatory Genomics and Epigenomics talk Videos | (VideoList) | 2016 |
| CBAssorted | Simon Institute Worshop: Computational Cancer Biology talk Videos | (VideoList) | 2016 |
| ✓Other | Simon Institute Worshop Talk: Obfuscation: Past, Present, and Possible Futures | (Video) | 2016 |
|
( many more exciting video tutorials @ http://videolectures.net ) ( many more exciting tutorials and papers we read about deep learning @ Notes2LearnDeep |
|||