Abstract

This review presents a unified, efficient model of random decision forests which can be applied to a number of machine learning, computer vision, and medical image analysis tasks.

Our model extends existing forest-based techniques as it unifies classification, regression, density estimation, manifold learning, semi-supervised learning, and active learning under the same decision forest framework. This gives us the opportunity to write and optimize the core implementation only once, with application to many diverse tasks.

The proposed model may be used both in a discriminative or generative way and may be applied to discrete or continuous, labeled or unlabeled data.

The main contributions of this review are: (1) Proposing a unified, probabilistic and efficient model for a variety of learning tasks; (2) Demonstrating margin-maximizing properties of classification forests; (3) Discussing probabilistic regression forests in comparison with other nonlinear regression algorithms; (4) Introducing density forests for estimating probability density functions; (5) Proposing an efficient algorithm for sampling from a density forest; (6) Introducing manifold forests for nonlinear dimensionality reduction; (7) Proposing new algorithms for transductive learning and active learning. Finally, we discuss how alternatives such as random ferns and extremely randomized trees stem from our more general forest model.

This document is directed at both students who wish to learn the basics of decision forests, as well as researchers interested in the new contributions. It presents both fundamental and novel concepts in a structured way, with many illustrative examples and real-world applications. Thorough comparisons with state-of-the-art algorithms such as support vector machines, boosting and Gaussian processes are presented and relative advantages and disadvantages discussed. The many synthetic examples and existing commercial applications demonstrate the validity of the proposed model and its flexibility.

Decision Forests

In recent years, decision forests have established themselves as one of the most promising techniques in machine learning, computer vision and medical image analysis. This book is directed at engineers and PhD students who wish to learn the basics of decision forests as well as more senior researchers who wish to push the state of the art in automated image understanding.

The authors presents a unified, efficient model of random decision forests which can be used in a number of applications such as scene recognition from photographs, object recognition in images, automatic diagnosis from radiological scans and document analysis. Such applications have traditionally been addressed by different, supervised or unsupervised machine learning techniques. In contrast, here we cast diverse tasks such as regression, classification and semi-supervised learning as instances of the same general decision forest model. The flexibility of the forest framework further extends to tasks such as density estimation, manifold learning and semi-supervised learning. The unified forest framework gives us the opportunity to implement and optimize the underlying algorithm only once, and then easily adapt it to individual applications with relatively small changes.

The theoretical basis and numerous explanatory examples presented in this book serve as a solid platform upon which to build exciting future research.