Differences between Fuzzy C-Means and EM

Question

When clustering a set of data points, what exactly are the differences between Fuzzy C-Means (aka Soft K-Means) and Expectation Maximization?

In slide 30 and 32 of this lecture I found, it says that Soft K-Means is a special case of EM in Soft K-Means only the means are re-estimated and not the covariance matrix, why's that and what are the advantages / disadvantages? How does covariance matrix affect the outcomes of EM?

Another question about these two algorithms: When they converge, all the data points will be hard-assigned to a particular cluster if the probability of it being in the said cluster is highest, right?

Answer 1

The difference between the two schemes is the model of the clusters. Fuzzy C-means and K-means model their clusters as circles (spheres in n-dimensional space), EM-clustering models the clusters as probability density functions (PDFs). In Euclidean space, the latter can have elliptical shapes (using Gaussian PDFs), determined by their covariance matrices.

Thus the difference is that the assignment criterion for K-means is the distance to a centroid whereas the criterion for the EM-algorithm is the probability of a data point given the PDF of the cluster center.

The cluster centers in K-means are hard-assigned, in Fuzzy C-means they are not; but it is rather simple to get a hard decision by, as you stated, taking the cluster with the highest probability/smallest distance for a given data point.