ICPSR Summer Program
CLASSIFICATION AND REGRESSION TREES
More examples of building trees, this time with
larger data sets with richer structure
Interplay between trees and regression. Trees are a very
useful way to find higher-order interactions.
Connection between trees and clustering
Supervised/unsupervised learning. Trees, however, can
use (in JMP anyway) any sort of inputs, whereas the
clustering tools only allow continuous variables.
Of course, you could convert everything to dummy variables, but that’s not such
an interesting task and we have more fun things to explore.
Neural networks combine several logistic regression
models together as one.
Visual tools let us see how the neural network compares
to a regression model.
Example (see JMP documentation for discussion of data)
Use JMP’s partitioning function to build a tree to answer
the question “Who buys American-brand cars?”
The partition1 function begins with a “mosaic” plot of
the data, with points scattered in the bins.
Horizontal lines in the mosaic plot imply no effect, like
a regression fit with zero slope.
Goodness of fit
The G2 statistic is twice the negative log-likelihood
based on the current partition. When starting it’s the
log-likelihood assuming no predictor effect.2
Think of G2 as the residual SS in regression. The
smaller G2 gets, the better the fit.
1 JMP’s partition function lurks under the modeling command. “CART” is trademarked.
2 For more info on likelihoods, see the on-line handout posted with these lecture notes.
The algorithm considers all possible ways to split the