14 ms·
I would guess that semantic similarity would be the stronger training signal than distance once you go beyond a sentence or two away.
by SomewhatLikely 3y ago
I would guess that semantic similarity would be the stronger training signal than distance once you go beyond a sentence or two away.
- sweezyjeezy 3y agoI'm pretty dubious - how would the model not get absolutely swamped by the vast amount of potential context if it's not learning to ignore long range signals for the most part?