I'm a researcher working with a language that has gone through phonological changes through time. I would like to tag parts of a word (i.e. prefix, stem, suffix) and then apply those phonological changes and then see what is left or different about the stuff that I tagged.

I'm currently using Python with some regex stuff to apply the changes so if I can do this using its NPL toolkit that would be perfect. I've start to mess around with it but I haven't found anything that will work just yet. I'm also not sure if this toolkit would be the best for this.

For example, I apply the following transformation to tag re, peat and ed in repeated: $$ \begin{align} \mathtt{repeated} & \xrightarrow{\mathtt{rep} \mapsto \mathtt{rp}} \mathtt{rpeated} \\ & \xrightarrow{\mathtt{ea} \mapsto \mathtt{e}} \mathtt{rpeted} \\ & \xrightarrow{\mathtt{d}\$ \mapsto \epsilon} \mathtt{rpete} \\ \end{align} $$

I would like to be able to find out what is left of the stuff I tagged. So I'd like to see that r is all that is left of the prefix, pe is all that is left of the stem, etc. Any help or direction is greatly appreciated!


You can use sequence alignments with a suitable score. You then get something like this:

$\qquad\displaystyle \begin{array}{cccccc} r & e & p & e & a & t & e & d \\ r & - & p & e & - & t & e & - \end{array}$

Now, if you have marked the original word, you can read off that -- according to your alignment score -- only "r" is left from the prefix, "pet" from the stem and "e" from the suffix.

The score makes a difference when parts may be assigned to e.g. prefix or stem. Levenstein/edit distance and Damerau-Levenstein distance may be good starting points. Play around with the penalties a bit; note that you can assign costs depending on whether you match against prefix, stem or suffix (if you have marked the original words) and use things like affine gap costs.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.