2 ms·
It matters because the hard-coded behavior is brittle and often doesn't do exactly what we want (or think). For example, GPT-2 has been ascribed nearly magical
by mattkrause 7y ago
It matters because the hard-coded behavior is brittle and often doesn't do exactly what we want (or think).
For example, GPT-2 has been ascribed nearly magical powers: it's a knowledge base, it can play chess, it does calculus, it's a dessert topping AND a floor wax!
When you look closer, however, it doesn't do any of those things particularly well. It can regurgitate something that looks like a true fact--or its negation with equal probability. It doesn't quite know the rules of chess. It needs a solver to check that the solution to an integral is, in fact, a solution.
- whatshisface 7y agoAll of those caveats apply to human intelligence, but to a lesser degree. Kids can play chess without exactly knowing the rules, and come on, everybody needs to check their integrals.
- TeMPOraL 7y agoBut kids can eventually learn proper algorithms for chess and integrals. DNNs cannot. In a sense, deep learning is like slide rules - you can squeeze some problem domains into giant, funny looking lookup tables, but generalize it does not.