0
$\begingroup$

I'm trying to run Gensim's LdaModel on a corpus. I load the corpus myself from two XML files. I made a class to iterate over the text pieces in the XML files. It converts every text piece to a BOW using a dictionary I built using the XML files. This all works, I've checked that. I then use this iterator to train an LDA model:

model = LdaModel(text_iterator, num_topics=10)

When I do this however, all topics are empty:

In[13]: model.print_topics()
Out[13]: 
[(0,
  '0.000*"20137" + 0.000*"4032" + 0.000*"26229" + 0.000*"20332" + 0.000*"26333" + 0.000*"317" + 0.000*"2664" + 0.000*"13626" + 0.000*"18816" + 0.000*"22420"'),
 (1,
  '0.000*"17987" + 0.000*"22313" + 0.000*"21767" + 0.000*"5261" + 0.000*"20284" + 0.000*"22027" + 0.000*"966" + 0.000*"349" + 0.000*"24857" + 0.000*"183"'),
 (2,
  '0.000*"7879" + 0.000*"518" + 0.000*"17715" + 0.000*"1493" + 0.000*"24995" + 0.000*"13789" + 0.000*"11252" + 0.000*"17447" + 0.000*"2918" + 0.000*"9131"'),
 (3,
  '0.000*"9601" + 0.000*"17656" + 0.000*"19358" + 0.000*"9690" + 0.000*"23146" + 0.000*"2264" + 0.000*"4974" + 0.000*"4022" + 0.000*"19941" + 0.000*"9122"'),
 (4,
  '0.000*"20779" + 0.000*"14551" + 0.000*"21269" + 0.000*"15422" + 0.000*"17955" + 0.000*"22860" + 0.000*"2076" + 0.000*"18358" + 0.000*"9560" + 0.000*"2977"'),
 (5,
  '0.000*"23836" + 0.000*"19651" + 0.000*"5808" + 0.000*"6156" + 0.000*"20544" + 0.000*"7681" + 0.000*"9566" + 0.000*"6945" + 0.000*"26141" + 0.000*"20062"'),
 (6,
  '0.000*"7037" + 0.000*"24278" + 0.000*"5344" + 0.000*"9728" + 0.000*"23999" + 0.000*"13210" + 0.000*"10789" + 0.000*"2823" + 0.000*"23370" + 0.000*"9440"'),
 (7,
  '0.000*"3447" + 0.000*"19113" + 0.000*"18941" + 0.000*"9960" + 0.000*"15887" + 0.000*"16232" + 0.000*"3986" + 0.000*"4990" + 0.000*"4393" + 0.000*"15064"'),
 (8,
  '0.000*"8251" + 0.000*"2122" + 0.000*"22461" + 0.000*"17461" + 0.000*"25932" + 0.000*"15056" + 0.000*"25561" + 0.000*"11463" + 0.000*"4413" + 0.000*"17334"'),
 (9,
  '0.000*"16085" + 0.000*"21046" + 0.000*"5507" + 0.000*"2292" + 0.000*"13859" + 0.000*"13897" + 0.000*"15937" + 0.000*"10399" + 0.000*"15231" + 0.000*"7106"')]

What can be the cause of this? Note that I don't know whether this is caused by the theoretical nature of the LDA model or Gensim's implementation specifically. Therefore, I posted my question here, and not on StackOverflow.

Here is a link to the LDA (Latent Dirichlet Alloction) model on Wikipedia: LDA

Improve this question
$\endgroup$
  • $\begingroup$ They way this question is phrased, it seems like you asks the reason for a specific tool do something. This is not a question that might not fit here, but your question might be on-topic on Stack Overflow, if your question is about Gensim in particular. If not, it is a good idea to rephrase the question such that it can be answered for the general Latent Dirichlet Allocation (LDA) model. Additionally, a source to the definition of the LDA model would be useful. $\endgroup$ – Discrete lizard♦ Apr 23 '17 at 14:48
  • $\begingroup$ @Discretelizard: I understand your remarks. I thought a while about where to post my question, since I don't know whether the unexpected results are caused by the theoretical LDA model, or Gensim's implementation specifically. Since I thought Gensim is used a lot in the scientific world, this felt like the best place. I will add this remark to my question to make that clear, along with a link to the LDA model. $\endgroup$ – Rugen Heidbuchel Apr 23 '17 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.