I wrote an article titled “Simple Numerical Optimization Using an Evolutionary Algorithm with C#” in the December 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/12/12/simple-numerical-optimization.aspx.

The goal of a numerical optimization problem is to find a vector of values that minimizes some cost function. The most fundamental example is minimizing the Sphere Function f(x0, x1, .. xn) = x0^2 + x1^2 + .. + xn^2. The optimal solution is X = (0, 0, .. 0) when f(X) = 0.0.

An evolutionary algorithm loosely mimics biological crossover (combining two existing parent solutions to create a new child solution) and mutation (slightly modifying a solution in a random way). In high-level pseudo-code:

create population of random solutions
loop many times
  pick two parent solutions
  create a child solution
  mutate child
  evaluate child
  replace weak solution in pop with child
end-loop
return best solution found

The crossover operation combines two existing solutions to create a new child solution. The mutation operation adds some randomness to the new child.

Evolutionary algorithms are a general purpose approach to solve a specific problem. A classic example is finding the weights and biases for a neural network. Currently, techniques based on backpropagation (which as based on calculus) mathematics are preferable for training. But in the long run, when computing power increases dramatically (presumably, through quantum computing), evolutionary algorithms will become more practical.

I enjoy looking at small dioramas like this one of a mad scientist’s laboratory. But I don’t have the patience or motivation to create such models. But I can spend a lot of time and effort creating machine learning models.