5
$\begingroup$

I want the conditional probability for each topic (being the word that we give as input). For example, the text being

have seen and reviewed your requirements you posted here. If you can give me the fix criteria/category of your data mining then I can do this job. If you want me to define and allot criteria and categorize it in then charges will be extra for per categorization included.

I have seen and reviewed your requirements you posted here. If you can give me the fix criteria/category of your data mining then I can do this job. If you want me to define and allot criteria and categorize it in then charges will be extra for per categorization included.

Assume that I give a word called research as an input, I want to know

What is the likelihood/probability that the text relates to research?

What algorithms we should create to get the above?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ You can use term frequency–inverse document frequency or one of its variants used in the probabillistic information retrieval models. From Wikipedia: ... A high weight in tf–idf is reached by a high term frequency (in the given document) and a low document frequency of the term in the whole collection of documents; the weights hence tend to filter out common terms. ... $\endgroup$
    – Vor
    Commented Oct 9, 2012 at 7:38
  • $\begingroup$ Are you familiar with the Natural Language Toolkit in Python? $\endgroup$
    – mac389
    Commented Nov 8, 2012 at 18:47
  • $\begingroup$ A term-frequency model is helpful in itself. The example text, it doesn't even contain the search term. Assuming the OP intends it to have some relevance, you need intermediate text/keywords. So, if you're looking for a statistical model, you'll ned to search the corpus for the term and find related terms or phrases with which to analyze the text block above. $\endgroup$
    – svidgen
    Commented Nov 9, 2012 at 22:26
  • $\begingroup$ Yes, inverse document frequency is the way to go, as @Vor described. However, this question should be in a data mining domain and not computer science. $\endgroup$
    – user4933
    Commented Dec 8, 2012 at 18:36
  • $\begingroup$ latent dirichlet allocation and topic models $\endgroup$
    – asd
    Commented Mar 9, 2013 at 23:36

2 Answers 2

Reset to default
3
$\begingroup$

You can try simple probabilistic graphical models, the simplest one being Naive Bayes.

One way to do this would be to represent a portion of text as a word frequency vector, that will be associated with a topic (the "class variable"). Then you use many such texts that are associated with topics to train your model (i.e. you model the probability of a frequency vector given a certain topic). Finally, given a new text you can ask what is the most likely topic assignment.

Naive Bayes, the simplest graphical model, would miss dependencies between the frequencies of the various words, but it is worth a shot as it is easy to implement. More complicated models could be used to capture these dependencies.

Improve this answer
$\endgroup$
2
  • $\begingroup$ @Raphael A portion of text can be represented as a word frequency vector, and will be associated with a topic (the "class variable"). Then you use many such texts that are associated with topics to train your model (i.e. you model the probability of a frequency vector given a certain topic). Finally, given a new text you can ask what is the most likely topic assignment. $\endgroup$
    – Bitwise
    Commented Mar 10, 2013 at 16:06
  • $\begingroup$ @Bitwise You should add this in your answer to make it more complete. $\endgroup$
    – Paresh
    Commented Mar 10, 2013 at 18:33
1
$\begingroup$

Try Latent Semantic Analysis or the similar Latent Semantic Indexing which reduces documents to vectors, and typically finds principal components via singular value decomposition, SVD that roughly relate or correspond to intrinsic or latent "subjects". It had major or key success in the Netflix datamining contest. Similarity is computed using vector algebra e.g. the cosine similarity or similar metrics.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.