2 ms·
1) Is everything really being mined already? My sense is that another round of GPT-like training on YouTube, EdX and Coursera data + some other large video arch
by samvher 4y ago
1) Is everything really being mined already? My sense is that another round of GPT-like training on YouTube, EdX and Coursera data + some other large video archives (BBC and the like) could still make quite a bit of difference. Text and images independently is one thing, having them together in context might be something else.
2) The available power seems to be growing pretty rapidly and dropping in prices. I think there are still quite some gains to be had from architectural optimizations (both in hardware and in models).
4) They were defined decades ago but I think they did actually seem to move us closer to AGI only recently.
You might be right, and there are definitely interesting times ahead! But I kind of doubt that we will have decades to sort out what we have and figure out its impact on society (which is a bit scary).