27 ms·
My main argument would be size of the good models and bandwidth, not speed of the computation. As long as it’s hard to transfer 1TB to a remote location we are
by pama 3y ago
My main argument would be size of the good models and bandwidth, not speed of the computation. As long as it’s hard to transfer 1TB to a remote location we are still moderately safe as a society and a large model of the scope of PB can take care of local security for large clusters. Maybe I am a bit off with a 20 years estimate especially given the acceleration of development with the help of AI, but I am also worried that Yoshua Bengio sounds like crying about a wolf when it is only a harmless rodent on the horizon.
- MacsHeadroom 3y agoWith no further algorithmic improvements, GPT-4 level AI should be achievable today (provided extensive compute / training on an order of magnitude more tokens than current models) in about the same file size as a typical YouTube video. Assuming some algorithmic improvement in the nearish future, I see no reason GPT-5 level AI in a GB or two isn't right around the corner. (For comparison, GPT-3.5-Turbo is likely a finetuned Curie 13B, which is just about 8GB in 4bit.)