And here I thought one of the big benefits of DL was that it could handle the complexities which would be too hard to specify symbolically in order to give symb

And here I thought one of the big benefits of DL was that it could handle the complexities which would be too hard to specify symbolically in order to give symbolic AI “common sense”. The following argument comes to mind, but I don’t really buy it (it just came to mind as something that one might say next): ‘ Perhaps there is an analogy between the solutions of “we just need to get better (more varied, better fitting the desired behavior, etc.) training data, and maybe better training procedures” and “we just need to add more/better inference rules and symbolic ways to encode statements, and add more facts about the world”. Similar in that both will produce the specific improvements they target, but where solving “the real/whole/big problem” that way is infeasible. If so, then maybe this indicates that a practical full-solution to artificial “common sense” would require something fundamentally different than both of them, if it is even possible at all. ‘ Again, I don’t really buy that line of reasoning, just expressing my inner GPT2 I guess, haha. Ok, but I presented an argument (or something like an argument) which I made up, and said that I don’t buy it. So, I should say why I don’t buy it, right? Like many of the things I write, it is chock-full of qualifiers like “perhaps” and “maybe”, to the point that one might say that it hardly makes any claims at all. But ignoring that part of it, one major difference is that the DL style architectures, seem to be working? And it isn’t clear what kinds of (practically speaking) hard limits it could run into. Now, on the other hand, perhaps at the time that symbolic AI was all the rage, it appeared the same way. (Is this what people mean when they talk about inside view vs outside view?). Why should these two things not be especially analogous? Well, saying “proposed solution X to the problem says to just [do more of what X is/do X better], and that is just like how proposed solution Y says to just [do more of what Y is/do Y better]” is kind of a fully generalize argument for dismissing any proposed type of solution where partial solutions of that type have been tried, but the whole problem hasn’t been solved that way yet, and another proposed kind of solution has already lost favor. This doesn’t seem like a generally valid line of reasoning. Sometimes you really do just need more dakka (spelling? I mean “more of the thing you already tried some of”). Of course, if one is convinced that it really was right for the older proposed kind of solution to be discarded, that probably should say something about the currently popular kind of solution. Especially if there have been many proposed kinds of solutions which have been discarded. But, it seems like much of what it says is just that the problem is hard. And, sure, that may mean an increased probability that the currently popular proposed kind of solution also doesn’t end up being satisfactory, that doesn’t mean one should be too quick to discard it. Tautologically: if no known alternative is currently at least as promising as the type of solution currently being considered, then, the current one is the most promising of the currently known options. Whether it is promising enough to actively pursue may be a different question, but it shouldn’t be marked as discarded until something else (perhaps something previously discarded, or something novel) becomes more promising.