Tree Based Models

The idea behind this family of methods is to segment the region into subspaces and the prediction for any subspace is made by taking the mean or mode of the response in that region.

A simple decision tree may not give performance at par with linear regression or generalized additive models, but when combined with techniques like bagging and boosting, multiple trees can yield significant reduction in bias.

A simple decision tree finds its utility in being relatively simpler in interpretation than it’s derived non-linear family.