Trying to tag parts of a word and keep track of any changes that happen to those parts

Question

I'm a researcher working with a language that has gone through phonological changes through time. I would like to tag parts of a word (i.e. prefix, stem, suffix) and then apply those phonological changes and then see what is left or different about the stuff that I tagged.

I'm currently using Python with some regex stuff to apply the changes so if I can do this using its NPL toolkit that would be perfect. I've start to mess around with it but I haven't found anything that will work just yet. I'm also not sure if this toolkit would be the best for this.

For example, I apply the following transformation to tag re, peat and ed in repeated: $$ \begin{align} \mathtt{repeated} & \xrightarrow{\mathtt{rep} \mapsto \mathtt{rp}} \mathtt{rpeated} \\ & \xrightarrow{\mathtt{ea} \mapsto \mathtt{e}} \mathtt{rpeted} \\ & \xrightarrow{\mathtt{d}\$ \mapsto \epsilon} \mathtt{rpete} \\ \end{align} $$

I would like to be able to find out what is left of the stuff I tagged. So I'd like to see that r is all that is left of the prefix, pe is all that is left of the stem, etc. Any help or direction is greatly appreciated!

Answer 1

You can use sequence alignments with a suitable score. You then get something like this:

$\qquad\displaystyle \begin{array}{cccccc} r & e & p & e & a & t & e & d \\ r & - & p & e & - & t & e & - \end{array}$

Now, if you have marked the original word, you can read off that -- according to your alignment score -- only "r" is left from the prefix, "pet" from the stem and "e" from the suffix.

The score makes a difference when parts may be assigned to e.g. prefix or stem. Levenstein/edit distance and Damerau-Levenstein distance may be good starting points. Play around with the penalties a bit; note that you can assign costs depending on whether you match against prefix, stem or suffix (if you have marked the original words) and use things like affine gap costs.