I wrote an article titled “Decision Tree Regression from Scratch Without Pointers or Recursion Using C#” in the February 2026 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2026/02/17/decision-tree-regression-from-scratch.aspx.

Decision tree regression is a machine learning technique that incorporates a set of if-then rules in a tree data structure to predict a single numeric value. For example, a decision tree regression model prediction might be, “If employee age is greater than 39.0 and age is less than or equal to 42.5 and years-experience is less than or equal to 8.0 and height is greater than 64.0 then bank account balance is $754.14.”

The article presents decision tree regression, implemented from scratch with C#, using list storage for tree nodes (no pointers/references), a FIFO stack to build the tree (no recursion), weighted variance minimization for the node split function, and saves associated row indices in each node. This design has maximum flexibility.

The output of the demo program is:

Loading synthetic train (200) and test (40) data
Done

Setting maxDepth = 3
Setting minSamples = 2
Setting minLeaf = 18
Using default numSplitCols = -1
Creating and training tree
Done

Tree:
ID 0   | idx   0 | v  -0.2102 | L   1 | R   2 | y 0.3493 | leaf F
ID 1   | idx   4 | v   0.1431 | L   3 | R   4 | y 0.5345 | leaf F
ID 2   | idx   0 | v   0.3915 | L   5 | R   6 | y 0.2382 | leaf F
ID 3   | idx   0 | v  -0.6553 | L   7 | R   8 | y 0.6358 | leaf F
ID 4   | idx  -1 | v   0.0000 | L   9 | R  10 | y 0.4123 | leaf T
ID 5   | idx   4 | v  -0.2987 | L  11 | R  12 | y 0.3032 | leaf F
ID 6   | idx   2 | v   0.3777 | L  13 | R  14 | y 0.1701 | leaf F
ID 7   | idx  -1 | v   0.0000 | L  15 | R  16 | y 0.6952 | leaf T
ID 8   | idx  -1 | v   0.0000 | L  17 | R  18 | y 0.5598 | leaf T
ID 11  | idx  -1 | v   0.0000 | L  23 | R  24 | y 0.4101 | leaf T
ID 12  | idx  -1 | v   0.0000 | L  25 | R  26 | y 0.2613 | leaf T
ID 13  | idx  -1 | v   0.0000 | L  27 | R  28 | y 0.1882 | leaf T
ID 14  | idx  -1 | v   0.0000 | L  29 | R  30 | y 0.1381 | leaf T

Evaluating model

Accuracy train (within 0.10) = 0.3750
Accuracy test (within 0.10) = 0.4750

MSE train = 0.0048
MSE test = 0.0054

Predicting for trainX[0] =
  -0.1660   0.4406  -0.9998  -0.3953  -0.7065
Predicted y = 0.4101

IF
column 0  "gt "   -0.2102 AND
column 0  "lte"   0.3915 AND
column 4  "lte"  -0.2987 AND
THEN
predicted = 0.4101

One advantage of decision tree regression compared to other techniques, such as kernel ridge regression and neural network regression, is that decision trees are significantly more interpretable. This diagram shows how the tree is structured and how the prediction is made:

A single decision tree can be used by itself for regression problems. But a more common approach is to use a collection of decision trees, called an ensemble. Examples include bagging tree regression, random forest regression, adaptive boosting regression, and gradient boosting regression.

Left: “Maleficent” (2014)
Center: “A Monster Calls” (2016)
Right: “The Lord of the Rings: The Two Towers” (2002)