Buttons840
searching PlanetScale…
- 1.▲
- 2.▲
- 3.▲
- 4.▲
- 5.▲
- 6.▲
87 ms·
- 1501.▲
- 1502.▲
- 1503.▲
- 1504.▲
- 1505.▲
- 1506.▲
- 1507.▲
- 1508.▲
- 1509.▲
- 1510.▲
- 1511.▲
- 1512.▲
- 1513.▲
- 1514.▲
- 1515.▲
- 1516.▲
- 1517.▲
- 1518.▲
I see, so [1, 2, 3, 4, 5, 6, 7, 8] is more similar to [5, 3, 1, 7, 8, 2, 4, 6] than with a linear layer? That's what you mean by permutation equivariant? I understand each context input is embedded with its position, but I suppose the - 1519.▲
- 1520.▲
If I want to do sequence modelling, let's say, predict the 9th element of the following sequence: [1, 2, 3, 4, 5, 6, 7, 8, 9] -- that is, I know the 8 most recent tokens as my "context", and I want to predict the next, 9 in t - 1521.▲
- 1522.▲
- 1523.▲
- 1524.▲
- 1525.▲
- 1526.▲
- 1527.▲
- 1528.▲
- 1529.▲
- 1530.▲
More ›