kmeans-js/src/kMeans.litcoffee

Simple Javascript implementation of the k-means algorithm, for node.js and the browser

Definitions
-----------

n: Features  
m: Datapoints  
K: Clusters  
X: m×n dataset  


    class kMeans

####initialize
Initialize a new k-means clustering object.
Pass the following options, and proceed to clustering.
Options:

* `K` (Integer) *Default: 5*  
  The number of clusters.

* `maxIterations` (Integer) *Default: 100*  
  The number of iterations before the algorithm stops.

* `enableConvergenceTest` (Boolean) *Default: true*  
  Enable convergence test. This test can be computationally intensive, but 
  might also save many iterations.

* `tolerance` (Float) *Default: 1e-9*  
  Floating point value for the convergence tolerance.

* `initialize` (Function)  
  The function used to initialize the centroids.  
  *Default:* The Forgy method, which chooses K random datapoints, as the 
  initial centroids. You can also write your own, with the following signature 
  fn(X, K, m, n), where:

  * `X` (2D Array): The set of datapoints. Remember this is passed by reference 
    and is therefore **mutable**.
  * `K` (Integer): The number of centroids to initialize.
  * `m` (Integer): The number of datapoints in `X`.
  * `n` (Integer): The number of features of each datapoint in `X`.

* `distanceMetric` (Function)
  The function used to measure the distance between centroids and points in its 
  cluster.  
  **Default** is the sum of the squared error. Other metrics might be 
  manhattan distance or minkowski distance.



        constructor: (options = {}) ->
            @K = options.K ? 5
            @maxIterations = options.maxIterations ? 100
            @enableConvergenceTest = options.enableConvergenceTest ? true
            @tolerance = options.tolerance ? 1e-9
            @initialize = options.initialize ? kMeans.initializeForgy
            @distanceMetric = options.distanceMetric ? @sumSquared

            if not (1 <= @K < Infinity)
                throw "K must be in the interval [1, Infinity)"
            if not (1 <= @maxIterations < Infinity)
                throw "maxIterations must be in the interval [1, Infinity)"

####cluster
Initialize clustering over dataset `X`.
`X` should be a m×n matrix of m data rows and n feature columns.

        cluster: (@X) ->
            @prevCentroids = []
            @clusters = []
            @currentIteration = 0

            [@m, @n] = [@X.length, @X[0].length]

            if not @m? or not @n? or @n < 1
                throw "Data must be of the format [[x_11, ... , x_1n], ... , [x_m1, ... , x_mn]]"

            @centroids = @initialize(@X, @K, @m, @n)

            if @centroids.length != @K or @centroids[0].length != @n
                throw "`initialize` must return a K×n matrix"

####step
Used when custom logic is interleaved within the clustering process.
See `autoCluster(X)` method below for an example

        step: ->
            @currentIteration++ < @maxIterations


####autoCluster
Cluster dataset `X` by means of the standard algorithm:
1. Find closest centroid for each datapoint and assign it to the centroids cluster
2. Move the centroid to the mean of its cluster
3. *Optional* Check for convergence, by measuring the distance moved
   by the centroid since the last iteration

        autoCluster: (X) ->
            @cluster X

            while @step()
                @findClosestCentroids()
                @moveCentroids()

                break if @hasConverged()

####initializeForgy
The Forgy method uses K random data points as initial centroids. Accessed as 
`kMeans.initializeForgy`  
*O(K)*

        @initializeForgy: (X, K, m, n) ->
            X[Math.floor Math.random() * m] for k in [0...K]

####initializeInRange
This initialization places K centroids at random, within the range of the data points.
Accessed as `kMeans.initializeInRange`  
*O(n·m+K·n)*

        @initializeInRange: (X, K, m, n) ->
            
            min = Infinity for i in [0...n]
            max = -Infinity for i in [0...n]

            for x in X
                for d, i in x
                    min[i] = Math.min min[i], d
                    max[i] = Math.max max[i], d

            (for k in [0...K]
                (for d in [0...n]
                    Math.random() * (max[d] - min[d]) + min[d]
                )
            )

####findClosestCentroids
Assign each datapoint to the cluster of its closest centroid.
This is done by adding the datapoint's index to an array of clusters,
where the index of each cluster corrosponds to the index of the centroid for
that cluster.

        findClosestCentroids: ->
            if @enableConvergenceTest
                @prevCentroids = (r.slice(0) for r in @centroids) #Clone optimization


Datapoints will be assigned to clusters to optimize the move step
however, this means that it will be hard to find out which cluster a specific
datapoint belongs to, without probing every element of every cluster

            @clusters = ([] for i in [0...@K])

            for x, i in @X
                cMin = 0
                xMin = Infinity
                for c, j in @centroids
                    min = @distanceMetric c, x

                    if min < xMin
                        cMin = j
                        xMin = min

                @clusters[cMin].push i

####moveCentroids
Iterate through each cluster and move it's centroid to the mean of all
the datapoints in that cluster.

        moveCentroids: ->
            for cl, i in @clusters
                continue if cl.length < 1 #Avoid division by 0

                for j in [0...@n]
                    sum = 0
                    for d in cl
                        sum += @X[d][j]
                    @centroids[i][j] = sum/(cl.length)

####hasConverged
Check whether any of the elements of the absolute difference between the
previous centroid positions and the current are greater than a set tolerance.
In case they're none are greater, then the algorithm has converged.

        hasConverged: ->
            return false if not @enableConvergenceTest

            for i in [0...@n]
                for j in [0...@m]
                    absDelta = Math.abs @prevCentroids[i][j] - @centroids[i][j]
                    return true if @tolerance > absDelta

            return false

####sumSquared
The default distance metric used by the `findClosestCentroids` step. Takes the
square of the euclidian distances between points. A custom metric can be used by
changing the `options.distanceMetric` when initializing kMeans.

        sumSqured: (X, Y) ->
            sum = 0
            n   = X.length

            while n--
                sum += (Y[n] - X[n]) * (Y[n] - X[n])

            sum



Check whether module.exports or exports are present, otherwise attach the class
to the window object.

    if module?.exports? or exports?
        module.exports = exports = kMeans
    else
        window.kMeans = kMeans