3 ms·
>our latest large-scale dataset, encompassing up to 18 trillion tokens I remember when GPT-3 was trained on 300B tokens.
by GaggiX 2y ago
>our latest large-scale dataset, encompassing up to 18 trillion tokens
I remember when GPT-3 was trained on 300B tokens.
- imjonse 2y agoand was considered too dangerous to be released publicly.
- GaggiX 2y agoThe larger GPT-2s were also considered too dangerous to release publicly at first.
- Workaccount2 2y agoI remember be very understanding of it too after seeing the incredible (but absolutely terrible in retrospect) outputs.
- GaggiX 2y agoI wasn't really compelled at the time, nothing has changed.
- baq 2y agothey are dangerous... for folks who need to scrape the web for low background tokens to train their transformers.
- abc-1 2y agoNobody ever really believed this, the truth is rarely in vogue.
- Prbeek 2y agoI remember when Ps2 chips were considered too advanced the US government banned shipment of PlayStations to China lest the PLA gets hold of them