5 ms·

Well said. Do you think some sort of other patterning mechanism, like that used to identify songs (thinking the Shazam algorithm where there is more along the

by sova 8y ago

Well said. Do you think some sort of other patterning mechanism, like that used to identify songs (thinking the Shazam algorithm where there is more along the lines of making a "fingerprint" and comparing to the print) will help take us to the next level? Neural Network architecture (sideways skyscrapers that get more focused as the network plays out) does not incorporate generalization much and this is sort of the point: a specific input data shall generate the resulting target, and any small variance would decidedly result in something different at the other end of the NN. Unless there were some way to incorporate a broad pattern on the data as it comes in... For example, run the NN, the NN shows results and also produces a fingerprint hash, now when you put slightly mal-aligned data into the NN, also provide the previous fingerprint result, and now (maybe) we can derive a device that will result in the same product and similar products within a range of the input data, to satisfy the case that the fingerprint stays true even when the data is slightly different. Just some ideas on the topic, if we could solve pattern generalization where the viewing window did not have to line up perfectly with the perceived pattern, that would be very powerful; we would have the ability to measure unaligned states, but an algorithm that approaches this power also must exploit some natural phenomenon, such as quantum superposition, if we are to complete our iterations in reasonable time. Without some sort of preprocessing step, such as sorting (that can ensure algorithms run quickly) it seems difficult to try and make a real neural net that is significantly more than a very specific compressed archive. What's really interesting is that as a stochastic process we could both generate two separate, functional, result-bearing neural networks (weights and layers) that achieved same or similar results while being completely different at their basis (in number of layers and the actual activation weights). And right now, there's no way to determine [elegantly] how similar our neural nets are. So, perhaps in some ways, the ability to compare neural network guts meaningfully will lead us to the ability to encode more elegant view ranges on data sets.

candiodari 8y ago
> Neural Network architecture (sideways skyscrapers that get more focused as the network plays out) does not incorporate generalization much ... Have you ever seen kids learn the tables of multiplication ? Or addition (but people don't seem to remember from their past that they learned addition from tables). Doesn't that give you pause when you're saying humans can generalize ? Because clearly, this is not how we teach our children. Humans don't learn that if 2+2=4, 3+2 must be 5. No. Humans learn that 2+2 is 4, then they learn that 2+3 is 5, then they learn ... and so on and so forth. Even when you see online players investigate a game. Whether it's chess, zelda, or starcraft. You keep seeing the same thing. Humans don't learn by generalizing. They learn how to act in every situation ... one situation at a time. So it's not just 6 year old kids doing this. The game advances because there's a few specific individuals that spend entire days making nonsensical moves, and on rare occasions they find a new move, see what happens, and tell the world (in chess we're talking something like once a year, in mario it seems possible to do it once every few months. It's scary how much tenacity is required for such a process). And for the vast majority of humans, it never even gets to that point. They just don't have the tenacity. So if you think humans are AGI, which you seem to do, then clearly such a state can be achieved with disappointing generalization capabilities.
- sova 8y ago
  >So if you think humans are AGI, which you seem to do, then clearly such a state can be achieved with disappointing generalization capabilities. Not sure what AGI means. Yeah, you make a strong point that there are axiomatic must-knows for any domain, and generalization would require some level well above -- or some layer well abstracted from -- this axiomatic knowledge.
  - candiodari 8y ago
    It took a few hours of filtering mistakes out of my kids to make me think about this. I was supportive (I hope), but internally I was thinking about it. The thing is the most difficult mistake was 5+3 being 14. I tried teaching them to approach the problem from different angles. 3+5 ? Fine. 4+3 ? Right. 6+3 ? Right. 5+3 ? 14 ... it took hours. In kids, most mistakes are corrected quickly, but a few persist. That reminded me that I have a few choice mistakes that lasted an incredibly long time despite getting corrected again and again and making no real amount of sense (meaning I know dozens of ways to calculate them, and I know how the numbers progress and I know how to use distribution to solve it, to "move the beans" (5 + 7 = 5 + 5 + 2 = 10 + 2 = 12. They present that logic, doing the tens first, as moving beans around. Which makes sense), and yet, there's a few mistakes that stuck. So you have a lot of time to think while correcting one of these sticky mistakes.

by sova 8y ago

candiodari 8y ago

> Neural Network architecture (sideways skyscrapers that get more focused as the network plays out) does not incorporate generalization much ... Have you ever seen kids learn the tables of multiplication ? Or addition (but people don't seem to remember from their past that they learned addition from tables). Doesn't that give you pause when you're saying humans can generalize ? Because clearly, this is not how we teach our children. Humans don't learn that if 2+2=4, 3+2 must be 5. No. Humans learn that 2+2 is 4, then they learn that 2+3 is 5, then they learn ... and so on and so forth. Even when you see online players investigate a game. Whether it's chess, zelda, or starcraft. You keep seeing the same thing. Humans don't learn by generalizing. They learn how to act in every situation ... one situation at a time. So it's not just 6 year old kids doing this. The game advances because there's a few specific individuals that spend entire days making nonsensical moves, and on rare occasions they find a new move, see what happens, and tell the world (in chess we're talking something like once a year, in mario it seems possible to do it once every few months. It's scary how much tenacity is required for such a process). And for the vast majority of humans, it never even gets to that point. They just don't have the tenacity. So if you think humans are AGI, which you seem to do, then clearly such a state can be achieved with disappointing generalization capabilities.

sova 8y ago
>So if you think humans are AGI, which you seem to do, then clearly such a state can be achieved with disappointing generalization capabilities. Not sure what AGI means. Yeah, you make a strong point that there are axiomatic must-knows for any domain, and generalization would require some level well above -- or some layer well abstracted from -- this axiomatic knowledge.
- candiodari 8y ago
  It took a few hours of filtering mistakes out of my kids to make me think about this. I was supportive (I hope), but internally I was thinking about it. The thing is the most difficult mistake was 5+3 being 14. I tried teaching them to approach the problem from different angles. 3+5 ? Fine. 4+3 ? Right. 6+3 ? Right. 5+3 ? 14 ... it took hours. In kids, most mistakes are corrected quickly, but a few persist. That reminded me that I have a few choice mistakes that lasted an incredibly long time despite getting corrected again and again and making no real amount of sense (meaning I know dozens of ways to calculate them, and I know how the numbers progress and I know how to use distribution to solve it, to "move the beans" (5 + 7 = 5 + 5 + 2 = 10 + 2 = 12. They present that logic, doing the tens first, as moving beans around. Which makes sense), and yet, there's a few mistakes that stuck. So you have a lot of time to think while correcting one of these sticky mistakes.