Guys thanks both for the replies. I still do not understand the inputs and outputs of the attention mechanism neural network. The paper says eij = a(si−1, hj)

Guys thanks both for the replies. I still do not understand the inputs and outputs of the attention mechanism neural network. The paper says eij = a(si−1, hj) is an alignment model which scores how well the inputs around position j and the output at position i match. The score is based on the RNN hidden state si−1 (just before emitting yi) and the j-th annotation hj of the input sentence. We parametrize the alignment model a as a feedforward neural network which is jointly trained with all the other components of the proposed system. So a is a feed fw nn. What is the input to this nn ? Do we input each hj corresponding to each word of the input sentence separately and get one number for each word ? If the input sentence has 200 words do I run this 200 times ? One for each input word ?