1
$\begingroup$

I hope it isn't off topic but I need to understand this example. Given the corpus 12 1 13 12 15 234 2526 and smoothing factor of k=1. The example does the following operations:

Considers OOV(out of vocabulary) words and assigns them a zero times value, after that k=1 is added to the times every words appears, to avoid zero probabilities. So the result of smoothing the bigrams probability will be:

$P(1|12)=(1+k)/(2+2+6*k)=0.2$
$P(15|12)=(1+k)/(2+2+6*k)=0.2$
$P(13|1)=(1+k)/(2+6*k)=0.25$
$P(12|13)=(1+k)/(2+6*k)=0.25$
$P(234|15)=(1+k)/(2+6*k)=0.25$
$P(2526|234)=(1+k)/(2+6*k)=0.25$

My question is, What kind of smoothing is this? shouldn't be for example like this?; $P(1|12)=(1+k)/(2+6*k)=0.25$
Besides it also says "If OOV words appear, you need to use smoothing to return a value; $P(234|12)=1/((2/7)*6+6)=0.1296$"

PS: I take this example from a small section of the translated version of this chinese webpage, it is just explaining a code implementation.

$\endgroup$

1 Answer 1

1
$\begingroup$

This looks like it might be implementing Laplace smoothing (though I can't explain the extra +2 in the first two calculations).

There are many smoothing methods, e.g., Lidstone, Good-Turing, Kneser-Tey, Witten-Bell. You can also use interpolation and backoff to handle rare or low-probability events when computing conditional probabilities like this.

$\endgroup$
3
  • $\begingroup$ But, shouldn't be (1+k)/(2+6*k)=0.25 instead of (1+k)/(2+2+6*k)=0.2? $\endgroup$ Commented Jan 27, 2020 at 17:59
  • $\begingroup$ @user2495207, I don't know where the extra +2 comes from. You'd have to ask the authors of that post why they did that. I'm not sure that reverse-engineering someone else's code or algorithm is useful. Hopefully my answer gives you an entry point into the literature on smoothing, and then you can learn more about standard methods for smoothing. $\endgroup$
    – D.W.
    Commented Jan 27, 2020 at 18:08
  • $\begingroup$ @D.W.Thank's I thought that I can't understand it becouse of I'm new in the subject. $\endgroup$ Commented Jan 27, 2020 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.