4 ms·

Our weird behavior during the pandemic is messing with AI models

45 points by gilad 6y ago

asquabventured 6y ago
So it turns out that humans are more complex than what the black box AI models have hyper optimized for.
- ikeyany 6y ago
  Better retrain the humans.
  - v4dok 6y ago
    Welcome to westworld
- rusticpenn 6y ago
  in AI terms, hyper optimized = overfitting, which is a known problem due to lack of data. Hopefully they will perform better in the next pandemic.
sn41 6y ago
The title is backwards. It should be that AI models are wrong. Not that a given AI model is a gold standard that human behavior is failing to live up to.
- DoreenMichele 6y ago
  "The map is not the territory."
  - deleted 6y ago
    [deleted]
- radarsat1 6y ago
  That's kind of a weird reading of the title tbh. "Messing with AI models" to me implies that the models are trained on data of a certain nature, therefore during the pandemic, they are turning out to be "wrong" in this new context. It's not drawing normative conclusions about human nature at all.
  - BerislavLopac 6y ago
    What does the word "weird" stand for exactly then?
sbmthakur 6y ago
It's high those AI models catch up with humanity.
sircastor 6y ago
As others have noted, the models don't represent the way things are anymore. And bring frank,I don't they'll represent things ever again. I very much doubt that the models will be a good match for behaviors going forward as people are likely going to have a hard, long transition towards normalcy
- deleted 6y ago
  [deleted]
deleted 6y ago
[deleted]
BIackSwan 6y ago
We disabled all ML enabled predictive features when the pandemic began. This has prevented us from the pain of having “bad” unreliable predictions which are worse than having “no” predictions. This saved us a lot of pain in maintainence and debugging.
- im3w1l 6y ago
  That seems very drastic. At least simple, explainable-logically sound models for short term dynamics should still work?
IIAOPSW 6y ago
Most models ("AI" or otherwise) behave very poorly in the situation known a "structural break" [1]. In extreme instances you can even end up with something like Simpsons paradox where the predicted correlation is the opposite of the true correlation [2]. I've had to deal with this exact problem before. The trick is to have a meta model that sits on top and tries to detect when the recent data is highly unlikely to have been produced if the existing model is true. When that happens you flush the older data. This might be what our brains do [3]. [1] https://en.wikipedia.org/wiki/Structural_break https://en.wikipedia.org/wiki/Structural_break [2] https://en.wikipedia.org/wiki/Simpson%27s_paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox [3] https://slatestarcodex.com/2017/09/06/predictive-processing-and-perceptual-control/ https://slatestarcodex.com/2017/09/06/predictive-processing-...
- enchiridion 6y ago
  Interesting idea. Why would your existing model not be able to detect a change in the distribution of the input?
  - deleted 6y ago
    [deleted]
  - IIAOPSW 6y ago
    How do you detect change in the input distribution before you even have a model of the input distribution?
- rmrfstar 6y ago
  Have you had any luck with change-point models? I always find myself falling back to hand-crafted heuristics + Snorkel [1]. Is there anything better? [1] https://www.snorkel.org/ https://www.snorkel.org/
  - IIAOPSW 6y ago
    For me its always been hand-crafted artisanal heuristics. I wouldn't know of a one size fits all solution because I've only had to deal with this scenario once.
2019-nCoV 6y ago
Is anti-fragile AI possible?
- cameldrv 6y ago
  Yes. You don't see it too much in ad recommenders because they don't need to be correct at the tails as long as you're rightish most of the time. Elements of it show up in autonomous vehicles. For example, one thing that can cause a disconnect is when sensors detect a sequence of events that is extremely improbable. If it's improbable enough, it's probably not the world, it's probably your model. Best to hand it to a human or hit the brakes. It reminds me a bit of John Meriwhether's statement that the events that led to LTCM's collapse were a "Six Sigma Event." If you are getting a six sigma event and you've only been in business a few years, maybe it's because your probability distribution of the market is wrong when you get a little too far from the mean.
- Barrin92 6y ago
  yes, if you build AI on more than inductionism. Reasoning from data, by definition, doesn't work if you encounter something that is entirely novel. Time to broaden the scope again to include cybernetics, reasoning, counter-factuals and so on rather than just building end to end data driven learning systems.
- rapjr9 6y ago
  Much AI today is trained models, and when the training is based on behaviors that people no longer exhibit the model output is no longer sensible and the model would need to be retrained to handle it. For example if you trained a speech recognition model on mens voices and then tried to use it for womens voices it would perform poorly. There has been some work on self-training or continuous learning AI, where all new inputs are used to continuously retrain the model, but these can be difficult to control, and training is often the most computationally expensive part of AI, so it often is not feasible except on arrays of computers. For example if people realize the model "learns" they could change their behavior to make the system learn to do the opposite of what most people would expect, not a very desirable outcome. Or the "learning" might home in on some strange artifact of behavior and accept it as the preferred output, such as thinking that when people click their tongues they are saying "yes", when really it just happened that some group of people who happen to do both were using the system a lot for some reason. This is still an area of research and while progress is being made it's much more difficult than using a large collection of data to train a model. For some kinds of problems the "learning" needs to understand context, which is very difficult to do in software since it involves understanding how the world works.
im3w1l 6y ago
It seems a fundamental nature of the universe that there is a tension between continuum and discrete. You see it everywhere: Wave vs particle, gradual heating vs phase change, but also soup with little chunks of meat (themselves smaller continuum), gas clouds coalescing into planets. It goes on and on. In this case model was built for the continuum and in came a big discrete shock. Something it wasn't built for. Shocks require a different modality of thought. More reasoning, logic-y than regression.
lostmsu 6y ago
To be fair, the predictive ability of humans also reduced quite a lot.
SomeoneFromCA 6y ago
Our neighborhood cats were seriously confused and disturbed by the looks of our faces in masks, no wonder AI is confused too.

45 points by gilad 6y ago

asquabventured 6y ago

So it turns out that humans are more complex than what the black box AI models have hyper optimized for.

ikeyany 6y ago
Better retrain the humans.
- v4dok 6y ago
  Welcome to westworld
rusticpenn 6y ago
in AI terms, hyper optimized = overfitting, which is a known problem due to lack of data. Hopefully they will perform better in the next pandemic.

sn41 6y ago

The title is backwards. It should be that AI models are wrong. Not that a given AI model is a gold standard that human behavior is failing to live up to.

DoreenMichele 6y ago
"The map is not the territory."
- deleted 6y ago
  [deleted]
radarsat1 6y ago
That's kind of a weird reading of the title tbh. "Messing with AI models" to me implies that the models are trained on data of a certain nature, therefore during the pandemic, they are turning out to be "wrong" in this new context. It's not drawing normative conclusions about human nature at all.
- BerislavLopac 6y ago
  What does the word "weird" stand for exactly then?

sbmthakur 6y ago

It's high those AI models catch up with humanity.

sircastor 6y ago

As others have noted, the models don't represent the way things are anymore. And bring frank,I don't they'll represent things ever again. I very much doubt that the models will be a good match for behaviors going forward as people are likely going to have a hard, long transition towards normalcy

deleted 6y ago
[deleted]

deleted 6y ago

[deleted]

BIackSwan 6y ago

We disabled all ML enabled predictive features when the pandemic began. This has prevented us from the pain of having “bad” unreliable predictions which are worse than having “no” predictions. This saved us a lot of pain in maintainence and debugging.

im3w1l 6y ago
That seems very drastic. At least simple, explainable-logically sound models for short term dynamics should still work?

IIAOPSW 6y ago

Most models ("AI" or otherwise) behave very poorly in the situation known a "structural break" [1]. In extreme instances you can even end up with something like Simpsons paradox where the predicted correlation is the opposite of the true correlation [2]. I've had to deal with this exact problem before. The trick is to have a meta model that sits on top and tries to detect when the recent data is highly unlikely to have been produced if the existing model is true. When that happens you flush the older data. This might be what our brains do [3]. [1] https://en.wikipedia.org/wiki/Structural_break https://en.wikipedia.org/wiki/Structural_break [2] https://en.wikipedia.org/wiki/Simpson%27s_paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox [3] https://slatestarcodex.com/2017/09/06/predictive-processing-and-perceptual-control/ https://slatestarcodex.com/2017/09/06/predictive-processing-...

enchiridion 6y ago
Interesting idea. Why would your existing model not be able to detect a change in the distribution of the input?
- deleted 6y ago
  [deleted]
- IIAOPSW 6y ago
  How do you detect change in the input distribution before you even have a model of the input distribution?
rmrfstar 6y ago
Have you had any luck with change-point models? I always find myself falling back to hand-crafted heuristics + Snorkel [1]. Is there anything better? [1] https://www.snorkel.org/ https://www.snorkel.org/
- IIAOPSW 6y ago
  For me its always been hand-crafted artisanal heuristics. I wouldn't know of a one size fits all solution because I've only had to deal with this scenario once.

2019-nCoV 6y ago

Is anti-fragile AI possible?

cameldrv 6y ago
Yes. You don't see it too much in ad recommenders because they don't need to be correct at the tails as long as you're rightish most of the time. Elements of it show up in autonomous vehicles. For example, one thing that can cause a disconnect is when sensors detect a sequence of events that is extremely improbable. If it's improbable enough, it's probably not the world, it's probably your model. Best to hand it to a human or hit the brakes. It reminds me a bit of John Meriwhether's statement that the events that led to LTCM's collapse were a "Six Sigma Event." If you are getting a six sigma event and you've only been in business a few years, maybe it's because your probability distribution of the market is wrong when you get a little too far from the mean.
Barrin92 6y ago
yes, if you build AI on more than inductionism. Reasoning from data, by definition, doesn't work if you encounter something that is entirely novel. Time to broaden the scope again to include cybernetics, reasoning, counter-factuals and so on rather than just building end to end data driven learning systems.
rapjr9 6y ago
Much AI today is trained models, and when the training is based on behaviors that people no longer exhibit the model output is no longer sensible and the model would need to be retrained to handle it. For example if you trained a speech recognition model on mens voices and then tried to use it for womens voices it would perform poorly. There has been some work on self-training or continuous learning AI, where all new inputs are used to continuously retrain the model, but these can be difficult to control, and training is often the most computationally expensive part of AI, so it often is not feasible except on arrays of computers. For example if people realize the model "learns" they could change their behavior to make the system learn to do the opposite of what most people would expect, not a very desirable outcome. Or the "learning" might home in on some strange artifact of behavior and accept it as the preferred output, such as thinking that when people click their tongues they are saying "yes", when really it just happened that some group of people who happen to do both were using the system a lot for some reason. This is still an area of research and while progress is being made it's much more difficult than using a large collection of data to train a model. For some kinds of problems the "learning" needs to understand context, which is very difficult to do in software since it involves understanding how the world works.

im3w1l 6y ago

It seems a fundamental nature of the universe that there is a tension between continuum and discrete. You see it everywhere: Wave vs particle, gradual heating vs phase change, but also soup with little chunks of meat (themselves smaller continuum), gas clouds coalescing into planets. It goes on and on. In this case model was built for the continuum and in came a big discrete shock. Something it wasn't built for. Shocks require a different modality of thought. More reasoning, logic-y than regression.

lostmsu 6y ago

To be fair, the predictive ability of humans also reduced quite a lot.

SomeoneFromCA 6y ago

Our neighborhood cats were seriously confused and disturbed by the looks of our faces in masks, no wonder AI is confused too.