Y
HN Search
Hacker News Search
new
|
comments
|
top
|
jobs
orost
searching PlanetScale…
1.
▲
2.
▲
3.
▲
4.
▲
5.
▲
6.
▲
5 ms
·
31.
▲
by
orost
6y ago
I believe it's quadratic, but that still means making it 10x larger is perfectly possible just by brute force with a little motivation.
32.
▲
by
orost
6y ago
GPT-3's coherency inevitably degrades when it writes anything of length, because its "working memory" (context window) is only a few hundred words. (2048 BPE tokens, which are characters, clusters of characters, or words) So