4 ms·

Is anti-fragile AI possible?

by 2019-nCoV 6y ago

cameldrv 6y ago
Yes. You don't see it too much in ad recommenders because they don't need to be correct at the tails as long as you're rightish most of the time. Elements of it show up in autonomous vehicles. For example, one thing that can cause a disconnect is when sensors detect a sequence of events that is extremely improbable. If it's improbable enough, it's probably not the world, it's probably your model. Best to hand it to a human or hit the brakes. It reminds me a bit of John Meriwhether's statement that the events that led to LTCM's collapse were a "Six Sigma Event." If you are getting a six sigma event and you've only been in business a few years, maybe it's because your probability distribution of the market is wrong when you get a little too far from the mean.
Barrin92 6y ago
yes, if you build AI on more than inductionism. Reasoning from data, by definition, doesn't work if you encounter something that is entirely novel. Time to broaden the scope again to include cybernetics, reasoning, counter-factuals and so on rather than just building end to end data driven learning systems.
rapjr9 6y ago
Much AI today is trained models, and when the training is based on behaviors that people no longer exhibit the model output is no longer sensible and the model would need to be retrained to handle it. For example if you trained a speech recognition model on mens voices and then tried to use it for womens voices it would perform poorly. There has been some work on self-training or continuous learning AI, where all new inputs are used to continuously retrain the model, but these can be difficult to control, and training is often the most computationally expensive part of AI, so it often is not feasible except on arrays of computers. For example if people realize the model "learns" they could change their behavior to make the system learn to do the opposite of what most people would expect, not a very desirable outcome. Or the "learning" might home in on some strange artifact of behavior and accept it as the preferred output, such as thinking that when people click their tongues they are saying "yes", when really it just happened that some group of people who happen to do both were using the system a lot for some reason. This is still an area of research and while progress is being made it's much more difficult than using a large collection of data to train a model. For some kinds of problems the "learning" needs to understand context, which is very difficult to do in software since it involves understanding how the world works.

Is anti-fragile AI possible?

Yes. You don't see it too much in ad recommenders because they don't need to be correct at the tails as long as you're rightish most of the time. Elements of it show up in autonomous vehicles. For example, one thing that can cause a disconnect is when sensors detect a sequence of events that is extremely improbable. If it's improbable enough, it's probably not the world, it's probably your model. Best to hand it to a human or hit the brakes. It reminds me a bit of John Meriwhether's statement that the events that led to LTCM's collapse were a "Six Sigma Event." If you are getting a six sigma event and you've only been in business a few years, maybe it's because your probability distribution of the market is wrong when you get a little too far from the mean.

Barrin92 6y ago

yes, if you build AI on more than inductionism. Reasoning from data, by definition, doesn't work if you encounter something that is entirely novel. Time to broaden the scope again to include cybernetics, reasoning, counter-factuals and so on rather than just building end to end data driven learning systems.

rapjr9 6y ago

Much AI today is trained models, and when the training is based on behaviors that people no longer exhibit the model output is no longer sensible and the model would need to be retrained to handle it. For example if you trained a speech recognition model on mens voices and then tried to use it for womens voices it would perform poorly. There has been some work on self-training or continuous learning AI, where all new inputs are used to continuously retrain the model, but these can be difficult to control, and training is often the most computationally expensive part of AI, so it often is not feasible except on arrays of computers. For example if people realize the model "learns" they could change their behavior to make the system learn to do the opposite of what most people would expect, not a very desirable outcome. Or the "learning" might home in on some strange artifact of behavior and accept it as the preferred output, such as thinking that when people click their tongues they are saying "yes", when really it just happened that some group of people who happen to do both were using the system a lot for some reason. This is still an area of research and while progress is being made it's much more difficult than using a large collection of data to train a model. For some kinds of problems the "learning" needs to understand context, which is very difficult to do in software since it involves understanding how the world works.