Foundations and Trends® in Theoretical Computer Science > Vol 10 > Issue 1–2

Sketching as a Tool for Numerical Linear Algebra

By David P. Woodruff, IBM Research Almaden, USA, dpwoodru@us.ibm.com

 
Suggested Citation
David P. Woodruff (2014), "Sketching as a Tool for Numerical Linear Algebra", Foundations and Trends® in Theoretical Computer Science: Vol. 10: No. 1–2, pp 1-157. http://dx.doi.org/10.1561/0400000060

Publication Date: 29 Oct 2014
© 2014 D. P. Woodruff
 
Subjects
Computational Number Theory,  Design and analysis of algorithms,  Dimensionality reduction,  Spectral methods,  Data mining
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction 
2. Subspace Embeddings and Least Squares Regression 
3. Least Absolute Deviation Regression 
4. Low Rank Approximation 
5. Graph Sparsification 
6. Sketching Lower Bounds for Linear Algebra 
7. Open Problems 
Acknowledgments 
References 

Abstract

This survey highlights the recent advances in algorithms for numerical linear algebra that have come from the technique of linear sketching, whereby given a matrix, one first compresses it to a much smaller matrix by multiplying it by a (usually) random matrix with certain properties. Much of the expensive computation can then be performed on the smaller matrix, thereby accelerating the solution for the original problem. In this survey we consider least squares as well as robust regression problems, low rank approximation, and graph sparsification. We also discuss a number of variants of these problems. Finally, we discuss the limitations of sketching methods.

DOI:10.1561/0400000060
ISBN: 978-1-68083-004-0
174 pp. $99.00
Buy book (pb)
 
ISBN: 978-1-68083-005-7
174 pp. $240.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. Subspace Embeddings and Least Squares Regression
3. Least Absolute Deviation Regression
4. Low Rank Approximation
5. Graph Sparsification
6. Sketching Lower Bounds for Linear Algebra
7. Open Problems
Acknowledgments
References

Sketching as a Tool for Numerical Linear Algebra

Sketching as a Tool for Numerical Linear Algebra highlights the recent advances in algorithms for numerical linear algebra that have come from the technique of linear sketching, whereby given a matrix, one first compressed it to a much smaller matrix by multiplying it by a (usually) random matrix with certain properties. Much of the expensive computation can then be performed on the smaller matrix, thereby accelerating the solution for the original problem.

Sketching as a Tool for Numerical Linear Algebra considers least squares as well as robust regression problems, low rank approximation, and graph sparsification. It also discusses a number of variants of these problems. It concludes by discussing the limitations of sketching methods and briefly looking at some open questions.

Sketching as a Tool for Numerical Linear Algebra is an ideal primer for researchers and students of theoretical computer science interested in how sketching techniques can be used to speed up numerical linear algebra applications.

 
TCS-060