Decision trees model the relationship between features and a target variable using the structure of a tree. In classification decision trees, the target variable is a categorical (usually binary) variable. The goal is to use the features to predict the membership of a class. By recursive partitioning (dividing the data set) using the features, homogeneous groups are created, which in turn are divided into further homogeneous groups until a stop criterion sets in or the defined homogeneity is reached. The partitioning may be based on various measures such as information gain or the Gini index.
Decision trees start with a root node which contains all observations (top node in the tree shown below). In further steps, the observations are further partitioned by binary splits, e.g. using the Gini index. If a split takes place, all (remaining) features are considered.The one is selected that optimally divides the data into homogeneous groups. Homogeneous means that the group contains a maximum number of observations of one characteristic of the target variable (the more it contains, the purer it is).
The decision tree above does not represent a categorical target variable, but a continuous one. This means that continuous variables, such as sales, can also be estimated.
from sklearn.tree import DecisionTreeClassifier
from sklearn.externals.six import StringIO
from IPython.display import Image
from sklearn.tree import export_graphviz
import pydotplus
clf = DecisionTreeClassifier(max_depth=4, criterion = “gini”)
clf = clf.fit(X_train, y_train)
export_graphviz(clf, out_file=dot_data, filled=True, rounded=True, special_characters=True)
graph = pydotplus.graph_from_dot_data(dot_data.getvalue())
Image(graph.create_png())
For more information on coding for Python, see the documentation for scikit-learn.
Explanation of the visualized regression tree
The visualized decision tree represents a regression tree that estimates a continuous target variable. The first split occurs at the root node. If the “recency ” is less than 104.1, the left branch is chosen, otherwise the right branch.
The lowest nodes represent end nodes. The size of the decision tree can be controlled by setting various parameters. In this case the parameters have been set so that no large decision tree is grown (variance bias trade-off). If a new observation (such as the orange path) is to be estimated, the decision tree is run from top to bottom. This shows one strength of decision trees: the ease of interpretation. Furthermore, the variable importance can be inferred. The further top a variable is, the more important it is for the prediction.
