3 ms·
it's a rolling buffer, so it just upsert index % 4 in this case
by nivekkevin 3y ago
it's a rolling buffer, so it just upsert index % 4 in this case
- mhartz 3y agoThanks, so does that mean position within the buffer is irrelevant?
- nivekkevin 3y agoit does feel like so, the position eventually loses its meaning as more and more data gets crunched by the training process, eventually it's just a context of the past 4 tokens it feels like