Tutorial
Jun 26, 20267 min read
0
Decision Trees: Entropy and Gini Impurity
A decision tree is a classifier that asks a sequence of yes/no questions about input features, following a path from root to leaf, and returns the prediction at…
~/blog/tutorials/machine-learning
Decision Tree
Tutorial
Jun 26, 20267 min read
0
Information Gain and Full Tree Construction
Post 01 showed that splitting on Employed reduces entropy more than splitting on Income. This post builds the full decision tree level by level — computing Info…
Tutorial
Jun 26, 20267 min read
0
Splitting Numerical Features in Decision Trees
The previous posts used categorical features with a finite set of values. For numerical features like sq_ft or age, the split can occur at infinitely many point…
Tutorial
Jun 26, 20268 min read
0
Decision Tree Pruning: Pre-Pruning and Post-Pruning
An unconstrained decision tree will grow until every leaf is pure — one sample per leaf if necessary. On training data this achieves 100% accuracy. On test data…
Tutorial
Jun 26, 20268 min read
0
Decision Tree Regression
A classification tree uses entropy or Gini to measure node impurity. A regression tree uses variance — the same algorithmic skeleton, a different objective. At…