Then, we pick the variable/threshold combination that gives us the highest/lowest value for the specific metric that we are using for the resulting children nodes (the highest reduction or increase in the metric). The result of this process is, for a certain node, a list of variables, each with different thresholds, and a calculated metric (Gini or MSE) for each variable/threshold tandem. For continuous features the mean of each two consecutive values (ordered from lowest to highest) of the training data are used as possible thresholds.

For discrete features all of its possible values are evaluated, resulting in N calculated metrics for each of the variables, being N the number of possible value for each categorical value. The process is also different if the feature that we are evaluating at the node is discrete or continuous. This is done by evaluating certain metrics, like the Gini index or the Entropy for categorical decision trees, or the Residual or Mean Squared Error for regression trees. Like we mentioned previously, decision trees are built by recursively splitting our training samples using the features from the data that work best for the specific task.

Leaf nodes: These are the final nodes of the tree, where the predictions of a category or a numerical value are made.Īlright, now that we have a general idea of what Decision trees are, let's see how they are built.

Intermediate nodes: These are nodes where variables are evaluated but which are not the final nodes where predictions are made.

In a normal decision tree it evaluates the variable that best splits the data.

The Root Node: Is the node that starts the graph.

In this figure we can observe three kinds of nodes:

Examdiff pro in decision tree