Reading List - randalburns/atidic-fall18 GitHub Wiki

Definitely Read

NN performance and GPUs

• Hinton et al., A Fast Learning Algorithm for Deep Belief Nets, Neural Computation 18, 1527–1554 (2006)

• Hinton et al., ImageNet Classification with Deep Convolutional Neural Networks, NIPS 2012.

• R Spring and A. Shrivastava. Scalable and Sustainable Deep Learning via Randomized Hashing. KDD, 2016

Stochastic Gradient Descent

• F. Niu et al. Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent. NIPS, 2011.

• X. Lian et al. Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent. NIPS 2017.

Cloud Frameworks

• Chen et al. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. arXiv:1512.01274v1, 3 Dec 2015

• M. Li et al. Scaling Distributed Machine Learning with the Parameter Server. OSDI, 2014.

Architecture

• Jouppi et al. In-Datacenter Performance Analysis of a Tensor Processing Unit. ISCA, 2017.

Performance

• Williams et al. Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures, CACM

• H. Pirk et al. Voodoo - A Vector Algebra for Portable Database Performance on Modern Hardware. VLDB, 9(14), 2016.

• Dongarra et al. Accelerating Numerical Dense Linear Algebra Calculations with GPUs. Chapter of Numerical Computations with GPUs. Springer, Cham. 2014

• N. Bell and M. Garland. Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors. Supercomputing, 2009

• Sparse-matrix representations (http://journals.sagepub.com/doi/pdf/10.1177/1094342011403516)

Algorithms/Methods

Gradient Boosted Trees

• T. Chen and C. Guestrin. XGBoost: A Scalable Tree Boosting System. KDD 2016.

• Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. "LightGBM: A Highly Efficient Gradient Boosting Decision Tree". In Advances in Neural Information Processing Systems (NIPS), pp. 3149-3157. 2017.

Approximate Nearest Neighbors

• Sections 3.3. and 3.4. Leskovec, Rajaraman, and Ullman, Mining of Massive Datasets, Cambridge Unviersity Press, 2014.

• M. Muja and D. Lowe. Scalable Nearest Neighbor Algorithms for High Dimensional Data. IEEE Trans. PAMI. 36(11), 2014.

SVM

• H. P. Graf et al. Parallel Support Vector Machines: The Cascade SVM. NIPS, 2005.

• B. Catanzaro, N. Sundaram, and K. Keutzer. Fast support vector machine training and classification on graphics processors. ICML, 2008

Spectral Clustering

• W. Y. Chen et al. Parallel Spectral Clustering in Distributed Systems. IEEE Trans PAMI, 33(3), 2011.
• L. Zelnik-Manor and P. Perona. Self-Tuning Spectral Clustering. NIPS 2005.

ISOMAP/LLE

• http://www-clmc.usc.edu/publications/T/tenenbaum-Science2000.pdf

Papers we might read

• Z Li et al. Learning to Search Efficiently in High Dimensions. NIPS, 2011.

• Kaushik Datta, Mark Murphy, Vasily Volkov, Samuel Williams, Jonathan Carter, Leonid Oliker, David Patterson, John Shalf, and Katherine Yelick. 2008. Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing (SC '08)

MLLib

• Meng et al. MLlib: Machine Learning in Apache Spark. JMLR 17, 2016.
• Zaharia et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. NSDI 2012.

Read in the Past. Not planning on this year

• Hellerstein, J. The Declarative Imperative Experiences and Conjectures in Distributed Logic. SIGMOD 2010