4
$\begingroup$

I have a very specific question about semantic clustering.

I have a list of words/phrases. I want to run an intelligent semantic clustering algorithm on this list. Please let me know what the available options are. Definitely I am looking for NLP based algorithms.

Simple, open-source, easy-to-use solutions will be highly appreciated. The semantic part is extremely important here.

$\endgroup$
  • $\begingroup$ Did you already have a look at Wikipedia? Assuming this is what you mean by semantic clustering; it wouldn't hurt to clarify. $\endgroup$ – Juho Jul 27 '12 at 11:00
  • $\begingroup$ Thanks a lot, Juho. Let me try to clarify a bit. I have a file containing several titles and I want a semantic clustering of these titles. Though the dataset is essentially one dimensional, some of the titles have two or more words and/or special characters like /-"#&'()*% etc. Please let me know if there is any easy solution to this problem. Best regards, $\endgroup$ – Dibyendu Jul 27 '12 at 14:25
  • $\begingroup$ Welcome! That is by far not a "very specific question". It is also ill-posed: without a semantic metric, you can not cluster (clustering itself is no different from non-semantic clustering). You can not get to a semantic metric without stating what the semantics of your strings are, what you consider similar, etc. It is furthermore not clear what "intelligent" and "easy" should mean here. You have to give us some more to work with, and I suggest you do some research yourself. $\endgroup$ – Raphael Jul 27 '12 at 14:29
  • $\begingroup$ Thanks a lot, Raphael. Being a researcher myself, I know that the question is ill-posed. Unfortunately, I do not have enough information to add clarity to the problem description. I am yet to locate reliable software packages to deal with non-numeric data. That is why I was thinking if any standard NLP based solution is already available. The solution should consider the English meaning of the titles while clustering. Regards, $\endgroup$ – Dibyendu Jul 27 '12 at 14:51
  • $\begingroup$ More specifically, if there exists a software package, which considers the English meaning of the titles and can directly produce the clusters from the input file containing the titles, that will be ideal for my purpose. Best regards, $\endgroup$ – Dibyendu Jul 27 '12 at 14:53
6
$\begingroup$

There are a lot of different approaches that you could take. While the commenters are right, coming up with a distance metric is important, based on my own experience, finding good representations of your words/phrases is going to be significantly more important.

Most of the "semantic" clustering algorithms that immediately come to mind are document level, not word or phrase level. This would include things like the closely related LSA, PLSA, and LDA, or neural network based approaches such as Semantic Hashing. This list is by no means exhaustive, any unsupervised machine learning approach to topic modeling could probably be thought of as doing document level semantic clustering.

I'm not sure how well the above approaches would work at the phrase level. I'm not going to say they won't, but I suspect the performance would be quite poor, since you are going to have a bunch of really (really!) sparse term vectors.

At the phrase level there are several techniques that seem promising. Simple techniques, such as just clustering n-grams are unlikely to yield useful results, so we'll rule that out right off the bat. A significantly better option would be to use learned word/phrase embeddings and then run some standard clustering algorithm (such as k-means) over these. Collobert and Weston have done some really interesting work in this vein. Their paper A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning or the related work Word representations: A simple and general method for semi-supervised learning by Turian et al would be a good place to start. Turian has a number of different word embeddings available on his website for download here, which may allow you to sidestep the overhead involved in learning such embeddings yourself.

Another option is to hand engineer features or a distance metric using a resource such as Wordnet. This certainly seems like a reasonable approach and a google search for "wordnet semantic distance" yields numerous results. I can't point you in any particular direction here though. Hope this helps.

$\endgroup$
  • $\begingroup$ Hm, n-grams may be useful to check whether two words occur in similar contexts. $\endgroup$ – Raphael Jul 28 '12 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.