Bottom line: Theory says the sophisticated, complex encoding technique should work better than a super simple encoding technique. In one experiment, the simple encoding technique worked slightly better.

I took a set of synthetic source training data that looks like this:

M,29,Austin,expert,68.5,timid,54600.00
F,51,Juneau,novice,60.0,brave,25700.00
M,32,Boston,master,73.5,alert,74900.00
M,19,Irvine,novice,61.0,brave,30300.00
. . .

Each row represents a person. The fields are sex, age, city (10 values), skill (3 values), height (inches), psychology (3 values), income (dollars). The goal is to predict income from the other six variables, using gradient boosting regression with decision tree weak learners.

There are 200 training items and 40 test items.

I normalized the numeric predictors, age and height, and the target variable income by dividing by 100, 100, 100,000 respectively. I did this just for convenience. The categorical predictors had to be converted to numbers because I wasn’t using a CART style tree.

There are two main schemes to encode categorical data for use with tree-based systems. The first encoding scheme is super simple: use ordinal encoding regardless of categorical type. So for sex, M = 0, F = 1. For city, Austin = 0, Boston = 1, Casper = 2, Denver = 3, Eugene = 4, Fresno = 5, Gallup = 6, Helena = 7, Irvine = 8, Juneau = 9. For skill, novice = 0, expert = 1, master = 2. For psychology, timid = 0, alert = 1, brave = 2.

The super simple normalized and encoded data looks like:

0, 0.29, 0, 1, 0.685, 0, 0.54600
1, 0.51, 9, 0, 0.600, 2, 0.25700
0, 0.32, 1, 2, 0.735, 1, 0.74900
0, 0.19, 8, 0, 0.610, 2, 0.30300
. . . 

With the super simple encoding, you can encode and then perform a train-test split. Easy.

The second encoding scheme is quite complicated. The complex normalized and encoded data looks like:

0, 0.29, 0.383784, 1, 0.685, 1, 0, 0, 0.54600
1, 0.51, 0.580719, 0, 0.600, 0, 0, 1, 0.25700
0, 0.32, 0.363510, 2, 0.735, 0, 1, 0, 0.74900
0, 0.19, 0.515110, 0, 0.610, 0, 0, 1, 0.30300
. . .

The city variable is smoothed-target encoded with smoothing parameter m=10. The psychology variable is one-hot encoded. When using smoothed target encoding, there is a risk of leakage. So you must first split, then compute the smoothed encodings on the training data, then apply to both train and test data.

I ran both versions of the encoded data through a from-scratch C# implementation of gradient boosting regression. The key output of the super simple encoding scheme:

Setting numTrees = 200
Setting maxDepth = 4
Setting minSamples = 2
Setting minLeaf = 1
Setting lrnRate = 0.0500

Creating and training GradientBoostRegression model
Done

Evaluating model

Accuracy train (within 0.10) = 1.0000
Accuracy test (within 0.10) = 0.8000

MSE train = 0.0000
MSE test = 0.0017

R2 train = 0.9994
R2 test = 0.9484

The key output of the complicated encoding scheme:

Setting numTrees = 200
Setting maxDepth = 4
Setting minSamples = 2
Setting minLeaf = 1
Setting lrnRate = 0.0500

Creating and training GradientBoostRegression model
Done

Evaluating model

Accuracy train (within 0.10) = 1.0000
Accuracy test (within 0.10) = 0.7750

MSE train = 0.0000
MSE test = 0.0022

R2 train = 0.9993
R2 test = 0.9360

The super simple encoding scheme is inferior to the complex encoding scheme, but the simple scheme worked slightly (but not statistically significant) better. The reasons why super simple encoding works well with gradient boosting regression are very complicated and subtle — quite literally pages of ideas and details.

I have seen this phenomenon many times in machine learning: complicated doesn’t always beat simple.

I have advanced college degrees in math and computer science, but my first degree was in psychology from UC Irvine. One of my requirements was a class in sociology. I remember learning a few things about strip club science.

Strip clubs and their equivalents have been around for literally thousands of years, dating back to ancient Greece and Rome.