Example of K-Means Clustering Using the scikit Code Library

When I need to do k-means data clustering, I almost always use my own custom implementation instead of a tool or code library because I can have full control over what’s going on. A colleague of mine asked me what I thought about the k-means function in the Python scikit-learn library so I investigated.

Bottom line: the scikit k-means function works fine but the scikit terminology is rather weird. Specifically, scikit calls the objective within-class sum of squares “inertia” (a term that’s misleading and not used by anyone else that I know of), and calls their default initialization algorithm “random” which is just plain incorrect — the algorithm used by scikit is called Forgy; random initialization is completely different.


Example of k-means using the Python scikit library (top image) and custom, raw C# code (bottom). The results are identical.


Also, because k-means is iterative and can fail easily, it’s usual practice to run k-means several times (perhaps 10 times) and then use the best result. So there are two loops involved. There’s an iterative loop used internally by k-means which is usually given a max value called max_iter, and then there’s an outer loop that runs k-means several times. I call the number of times to run k-means looking for the best result “trials”. The scikit k-means calls the internal loop iteration max value “max_iter” which is sensible, but calls the outer loop iteration value “n_init” which is very misleading because it’s not an initialization loop at all. The name n_init is an especially weak name because some forms of k-means (such as k-means++) do in fact have an initializtion loop that requires a value that you’d want to call n_init or something similar.

But, other than some very wacky terminology, the scikit k-means function seems OK to me.

I coded two clustering demos using the same data. I used the scikit k-means functionality and then used my raw C# implementation of k-means. I got identical results. If you have 7 data items and place them into k = 3 clusters, the results of a k-means clustering look like (2, 0, 1, 1, 1, 0, 2) which means the first data item is in cluster 2, the second data item is in cluster 0, and so on. The cluster IDs of 0, 1, 2 are arbitrary so an identical clustering would be (1, 0, 2, 2, 2, 0, 1) where cluster IDs 2 and 1 are switched. So, I fiddled with the random seed for my C# demo to get absolutely identical result displays.


There are dozens of movies that feature identical twins. In most cases an actor or actress plays their own twin. Here are three of my favorite twins in movies. Left: Fred and George Weasley from the Harry Potter series, played by actors James and Oliver Phelps. Center: The fairy twins from “Mothra” (1961) played by Emi and Yumi Ito. Right: Conjoined twins Walt (Greg Kinnear) and Bob Tenor (Matt Damon) in the comedy “Stuck on You” (2003), one of the funniest movies I’ve ever seen.


# cluster_sk_demo.py 

from sklearn.cluster import KMeans
import numpy as np

np.set_printoptions(precision=4)

print("\nBegin k-means using Python scikit demo ")

data = np.array([
[ 0.65, 0.220 ], [ 0.73, 0.160 ],
[ 0.59, 0.110 ], [ 0.61, 0.120 ],
[ 0.75, 0.150 ], [ 0.67, 0.240 ],
[ 0.68, 0.230 ], [ 0.70, 0.220 ],
[ 0.62, 0.130 ], [ 0.66, 0.210 ],
[ 0.77, 0.190 ], [ 0.75, 0.180 ],
[ 0.74, 0.170 ], [ 0.70, 0.210 ],
[ 0.61, 0.110 ], [ 0.58, 0.100 ],
[ 0.66, 0.230 ], [ 0.59, 0.120 ],
[ 0.68, 0.210 ], [ 0.61, 0.130 ]])

print("\nData: ")
print(data)

print("\nSetting n_clusters_ = 3")
print("Using default random init (actually Forgy)")
print("Setting n_init = 10")
print("Setting max_iter = 100 ")
print("Setting random_state = 0")

print("\nStarting clustering")
km = KMeans(n_clusters=3, n_init=10, max_iter=100, \
  random_state=0).fit(data)
print("Done")

print("\nFinal labels: ")
print(km.labels_)

print("\nThe centers:")
print(km.cluster_centers_)

print("\nThe inertia:  ", end="")
print("%0.4f" % km.inertia_)

print("\nEnd demo")
This entry was posted in Machine Learning. Bookmark the permalink.

2 Responses to Example of K-Means Clustering Using the scikit Code Library

  1. Thorsten Kleppe's avatar Thorsten Kleppe says:

    Hey Dr. McCaffrey, hope you’re fine.
    You are just the best! Really, maybe no one told you that, but it’s true. There are so smart people out there, but you top them all. Before I miss this chance after 2 years of wonders, I want to say you that.

    One of the smartest guys from my country told us: if you cannot explain a science to ordinary people, the science is not true.

    And you explain the complicated things so easy I never heard about.

    With every topic you move more and more to a higher and higher level, and you take your reader to that journey.

    You share all this wonderful experience, the passion and ML, thank you so much for all that work, for 134 MSN Articles in so many years, despite to on of the best of them, “Gaussian Mixture Models” is not listed. Your books, your other articles on Visual Studio Magazin. And almost every day a topic that’s so deep. What is a Survival Function? You never explained that, but you showed me on a sophisticated way how I need to implement that, BAM!

    The hottest YouTuber in ML explain topics right now, you had treated a long time ago. And you did it much better if I’am honestly. And they are the best on YouTube, because they was rated.

    Dr. McCaffrey my Thank You to you goes to infinity. Thank you so much.

    You are what means real for me, and you’re the King of Machine Learning. Hooray !!!

Comments are closed.