4
$\begingroup$

I am working on a task embedding sentences into a lower-dimensional space according to style, both grammatical and lexical. As such, I want to have as input the linear ordering of tokens in each sentence, together with its dependency parse as provided by spacy.

In particular, I'd like to find a way to tie together the representation of the linear order of tokens and the representation of the dependency parse, so the network could learn features like "this sentence used a word with an embedding close to Y as an nmod which came before and modified a word with an embedding close to Z". How could I design such a network?

Edit: The desired input to the network is a parsed sentence; the desired output is a vector which allows that sentence to be compared with others in terms of both lexical and syntactic features. I know how to use an RNN with a sequence of word-vector embeddings as input. I also know how to encode a tree of grammatical functions as a sequence of tokens starting from the root. I'm not sure how to create a unified representation of the sentence where I can determine, for instance, both the embedding and the grammatical function of the fourth word in the sentence and the embedding of the word it modifies (requiring knowledge of the edges between words as well as their linear ordering).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.