How were the GPT-2 token embeddings constructed?
The authors mention that they used Byte Pair Encoding to construct their vocabulary. But BPE is a compression algorithm that returns a list of subword tokens that would best compress the total vocabulary (and allow rare words to be encoded efficiently).
My question is: how was that list of strings turned into the vectors that they actually used for training the model? The papers they published on the original GPT and its follow-up GPT-2 don't seem to specify those details.