1
$\begingroup$

I am trying to build a bigram letter model.

I obtain a sequence of words in a form of ['hello','I','am','Johnny'].

Firstly, I lower all the words to obtain : ['hello','i','am','johnny'].

I am capable of building a bigram letter model, but I have read somewhere that you should provide some kind of empty strings / padding to the model.

Does anybody know why do you have to provide padding to the input data to build a proper language model? And how to use padding on this sample input to build letter model?

I was thinking about making a space in front of every word, but I am not convinced that this is the right solution - other options that I am considering are adding padding at the end of every sentence or after each sequence of 2 characters as this is a bigram letter model.

$\endgroup$
1
$\begingroup$

The padding is there since the distribution of letters at the beginning and end of words is very different from their distribution inside words. To capture that, separate words with spaces and add trailing spaces on both sides: ' hello i am johnny '.

$\endgroup$
  • $\begingroup$ If I have a method that takes a list, would you just do left padding to all of the words? So you would get: [" hello"," I", " am"," Johnny"] , or would you rather add padding to both of the sides of each word, which would result in [" hello "," I ", " am "," Johnny "] $\endgroup$ – Johhny Bravo Feb 1 '16 at 0:42
  • $\begingroup$ @JohnnyBravo This also works. $\endgroup$ – Yuval Filmus Feb 1 '16 at 6:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.