AGI is probably overstating things here, since will probably need something more than an architectural advancement to reach the broad, transferable capabilities

AGI is probably overstating things here, since will probably need something more than an architectural advancement to reach the broad, transferable capabilities of even human problem solving abilities. But it might be worth out scaling up some older architectures that were mostly abandoned. A huge GRU/LSTM would be really interesting. There may be some emergent properties we don’t know about.