3 ms·
I saw the graph. Bits-per-byte is not clear for me (for example lower loss does not always mean better accuracy when you train a classifier). What I needed is t
by visarga 4y ago
I saw the graph. Bits-per-byte is not clear for me (for example lower loss does not always mean better accuracy when you train a classifier). What I needed is to know if the generated text has the same "literary" qualities, and also if it can compete in few shot mode on as many tasks as GPT-3 can do.
In other words, can we rely on it instead of GPT-3 in realistic scenarios or is it good only for low BpB?
- ShamelessC 4y agoI imagine that GPT-3's ability to complete prompts it hasn't seen will make it quite a bit more versatile than using the Pile. It's the interpolation between the data points that makes deep learning so powerful. This obviously still has plenty of use cases. Could get a batch of similar text to calculate statistics on. Or perhaps augment existing data for training something else. One realistic use case would be to simply present a search engine interface enabling you to find "interesting" text snippets alongside metadata like book author/title matching your description, perhaps for fiction enthusiasts or what have you.