The Elements of Statistical Learning: Data Mining, Inference, and Prediction

During the past decade there has been an explosion in computation and information technology. With it have come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics. Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It is a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book. This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression and path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide data (p bigger than n), including multiple testing and false discovery rates. Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie co-developed much of the statistical modeling software and environment in R/S-PLUS and invented principal curves and surfaces. Tibshirani proposed the lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, projection pursuit and gradient boosting.

This major new edition features many topics not covered in the original, including graphical models, random forests, and ensemble methods. As before, it covers the conceptual framework for statistical data in our rapidly expanding computerized world.

Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie co-developed much of the statistical modeling software and environment in R/S-PLUS and invented principal curves and surfaces. Tibshirani proposed the lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, projection pursuit and gradient boosting.

JOURNAL OF CLASSIFICATION, JUNE 2004

"This is a great book. All three authors have track records for clear exposition and are famously gifted for finding intuitive explanations that illuminate technical results'In particular, we admire the book for its:

-outstanding use of real data examples to motivate problems and methods;

-unified treatment of flexible inferential procedures in terms of maximization of an objective function subject to a complexity penalty;

-lucid explanation of the amazing performance of the AdaBoost algorithm in improving classification accuracy for almost any rule;

-clear account of support vector machines in terms of traditional statistical paradigms;

-regular introduction of some new insight, such as describing self-organizing maps as constrained k-means clustering.

'No modern statistician or computer scientist should be without this book."

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, JUNE 2004

"In the words of the authors, the goal of this book was to 'bring together many of the important new ideas in learning, and explain them in a statistical framework." The authors have been quite successful in achieving this objective, and their work is a welcome addition to the statistics and learning literatures'A strength of the book is the attempt to organize a plethora of methods into a coherent whole. The relationships among the methods are emphasized. I know of no other book that covers so much ground."

Introduction.- Overview of supervised learning.- Linear methods for regression.- Linear methods for classification.- Basis expansions and regularization.- Kernel smoothing methods.- Model assessment and selection.- Model inference and averaging.- Additive models, trees, and related methods.- Boosting and additive trees.- Neural networks.- Support vector machines and flexible discriminants.- Prototype methods and nearest-neighbors.- Unsupervised learning.