2
$\begingroup$

I am confused on the inputs of a Long-Short Term Memory (LSTM) for the slot filling task in Spoken Language Understanding.

Before I worked on this, I implemented a language model with a Recurrent Neural Network (RNN) and then with a LSTM. The input to the RNN and LSTM language models was a one hot vector, which represented each word.

Now, when moving on to the slot filling task for a LSTM, I am having trouble what the input would be. I know that a one-hot vector representation is not enough for this task because the outputs along each time step are slot labels. I have a dictionary (in Python) that maps words to indices (which I can turn into a one hot vector), and I also have a dictionary with a labels (that are used for slot filling), which I got from the ATIS data. Here is an example:

enter image description here

I know I need the above two dictionaries to accomplish the slot filling task, but I cannot figure out how to use them as inputs for the LSTM? Furthermore, I have been using the basic LSTM structure, and for the language model LSTM I build, the output at each time step went through a Softmax function. Is this what will be required for slot filling too?

I am in high school and do not have anyone to contact, so any help is really appreciated. Thank you so much.

$\endgroup$
  • $\begingroup$ Hey! I just saw this question. Why do you think one hot wont work? The input still remains one hot. The output is also one hot on the label space. The output still goes through the softmax. $\endgroup$ – Rahul Aug 30 '17 at 23:58
  • $\begingroup$ I think this is discussed in the Approach of this paper m.media-amazon.com/images/G/01/amazon.jobs/… $\endgroup$ – Sam H. Oct 10 '18 at 5:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.