3 ms·

I guess I'm not sure how the "feed in the whole sequence" works, if there's a single reward at the end.

• maleldil 2y ago
  It depends on the model and the problem. As an example, BERT-based models have a special [CLS] token that was pre-trained to encode information about the whole sequence. A reward model based on BERT would take the output embedding of that token from the last layer and feed it through a classification head, which would depend on your problem. You could then train this classification head on your alignment dataset like a classification problem. You can check the examples from the TRL library for more information.
  • 1024core 2y ago
    > You can check the examples from the TRL library for more information. What library is that? Thanks!