6 ms·

Perhaps. It seems it’s still an open question whether AI is just about memorizing your data, or can it actually make reliable decisions during previously unseen

Perhaps. It seems it’s still an open question whether AI is just about memorizing your data, or can it actually make reliable decisions during previously unseen scenarios. Have we already observed, or collected, all that is possible in the “driving” world?

ipsum2 6y ago
Most self-driving companies use simulations to see how the model performs in unseen scenarios.
- maxbond 6y ago
  I'm sure that's tremendously helpful but isn't exactly an antidote, is it? You can't really simulate an unforeseen situation. If you can simulate it accurately - then you've foreseen it, and are thus able to account for all relevant variables. If you are randomly generating scenarios, something like fuzz testing or property testing, then I am sure you will discover bugs before they hit production, but you can't be sure you're simulating it accurately. For example, maybe when your car is T-boned while climbing a steep hill, the suspension behaves in a way you didn't expect and which isn't replicated by your simulation. Or maybe you're sent a batch of decals with bad adhesive, and in the hot sun the begin to slip down the windshield until they end up obscuring or otherwise interfering with a sensor. The only simulation that can account for every variable, regardless of whether you've anticipated it, is reality.
KorfmannArno 6y ago
I've yet to see an "AI" that is not just memorizing data.
- KorfmannArno 6y ago
  But then again, how does something like the following work? https://twitter.com/GoogleAI/status/1293970520753369088 https://twitter.com/GoogleAI/status/1293970520753369088 Any idea how it could be fooled?
  - antipaul 6y ago
    Via a fancy “adversarial” patch https://www.theverge.com/2019/4/23/18512472/fool-ai-surveillance-adversarial-example-yolov2-person-detection https://www.theverge.com/2019/4/23/18512472/fool-ai-surveill...
  - nl 6y ago
    What do you mean by "fooled"? I'm very familiar with the BlazeFace and FaceMesh models (which are related to this in Google's MediaPipe framework). They have weaknesses - they aren't designed for running upside down for example, so if they get data that is that way oriented they will tend to fail. They aren't designed for "life" detection, so you can show printed pictures of a face and it will detected it. But they give you confidence scores etc, so if you give it something like a caricature of a face it will return reasonable confidence numbers indicating it isn't as sure of its predictions.
- nl 6y ago
  Then you haven't really looked. Most credible machine learning systems work well on unseen data, which by definition isn't memorizing.
  - KorfmannArno 6y ago
    Can you link an example you find to generalize particularly well?
    - nl 6y ago
      Sure. Apple's Face Unlock. It generalises to almost every face, and the ones it doesn't its failure mode is safe. Or something like word embeddings. Works incredibly well, and most "failure" modes are around things like bias, where the behavior reflects the real world. Or something like AlphaZero. Not only is every new game of Go it plays brand new, it learnt to play Chess without knowing the rules. That just isn't memorization. https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go https://deepmind.com/blog/article/alphazero-shedding-new-lig...
      - KorfmannArno 6y ago
        https://arxiv.org/pdf/1805.12177.pdf https://arxiv.org/pdf/1805.12177.pdf I only read the abstract.
        nl 6y ago
        Not sure what you think this shows, but there's a lot of reasons why the results they show don't really matter much - or at least might actually reflect an accuracy approximatly what a human would also achieve. Their headline claim is a "1 pixel change reduces accuracy by 30%". The test process for that number is this: We choose a random square within the original image and resize the square to be 224x224. The size and location of the square are chosen randomly according to the distribution described in (Szegedy et al., 2015). We then shift that square by one pixel diagonally to create a second image that differs from the first one by translation by a single pixel. So... they are taking a random square, downsampling to 224, moving and then predicting on that subset of the original image, and measuring the performance against the accuracy of the original prediction. What this seems to show is that "CNNs aren't as accurate at making predictions on subsets of an image as on the whole image". This is of course to be expected, and is exactly how a human would perform. There are a bunch of other criticisms too. Read the ICLR (for which it was rejected) reviews: https://openreview.net/forum?id=HJxYwiC5tm https://openreview.net/forum?id=HJxYwiC5tm
- visarga 6y ago
  Just memorising data is simple, use a file on disk. The hard part is recognising data when it is slightly different than what the model 'memorised', deciding which of the millions of things it learned best fits the answer we desire.
nl 6y ago
> It seems it’s still an open question whether AI is just about memorizing your data, No - it's not at all, and this is a well understood problem in building machine learning systems. There are some cases where this occurs but usually this is just overfitting. Good AI systems generalize well on unseen data.
- nl 6y ago
  To the downvoters, I give you AlphaZero. Not only is every game of Go it plays and wins brand new (so no memorisation), the same system learnt to play Chess without knowing the rules, and plays in a "style .. unlike any traditional chess engine" https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go https://deepmind.com/blog/article/alphazero-shedding-new-lig...

ipsum2 6y ago

Most self-driving companies use simulations to see how the model performs in unseen scenarios.

maxbond 6y ago
I'm sure that's tremendously helpful but isn't exactly an antidote, is it? You can't really simulate an unforeseen situation. If you can simulate it accurately - then you've foreseen it, and are thus able to account for all relevant variables. If you are randomly generating scenarios, something like fuzz testing or property testing, then I am sure you will discover bugs before they hit production, but you can't be sure you're simulating it accurately. For example, maybe when your car is T-boned while climbing a steep hill, the suspension behaves in a way you didn't expect and which isn't replicated by your simulation. Or maybe you're sent a batch of decals with bad adhesive, and in the hot sun the begin to slip down the windshield until they end up obscuring or otherwise interfering with a sensor. The only simulation that can account for every variable, regardless of whether you've anticipated it, is reality.

KorfmannArno 6y ago

I've yet to see an "AI" that is not just memorizing data.

KorfmannArno 6y ago
But then again, how does something like the following work? https://twitter.com/GoogleAI/status/1293970520753369088 https://twitter.com/GoogleAI/status/1293970520753369088 Any idea how it could be fooled?
- antipaul 6y ago
  Via a fancy “adversarial” patch https://www.theverge.com/2019/4/23/18512472/fool-ai-surveillance-adversarial-example-yolov2-person-detection https://www.theverge.com/2019/4/23/18512472/fool-ai-surveill...
- nl 6y ago
  What do you mean by "fooled"? I'm very familiar with the BlazeFace and FaceMesh models (which are related to this in Google's MediaPipe framework). They have weaknesses - they aren't designed for running upside down for example, so if they get data that is that way oriented they will tend to fail. They aren't designed for "life" detection, so you can show printed pictures of a face and it will detected it. But they give you confidence scores etc, so if you give it something like a caricature of a face it will return reasonable confidence numbers indicating it isn't as sure of its predictions.
nl 6y ago
Then you haven't really looked. Most credible machine learning systems work well on unseen data, which by definition isn't memorizing.
- KorfmannArno 6y ago
  Can you link an example you find to generalize particularly well?
  - nl 6y ago
    Sure. Apple's Face Unlock. It generalises to almost every face, and the ones it doesn't its failure mode is safe. Or something like word embeddings. Works incredibly well, and most "failure" modes are around things like bias, where the behavior reflects the real world. Or something like AlphaZero. Not only is every new game of Go it plays brand new, it learnt to play Chess without knowing the rules. That just isn't memorization. https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go https://deepmind.com/blog/article/alphazero-shedding-new-lig...
    - KorfmannArno 6y ago
      https://arxiv.org/pdf/1805.12177.pdf https://arxiv.org/pdf/1805.12177.pdf I only read the abstract.
      - nl 6y ago
        Not sure what you think this shows, but there's a lot of reasons why the results they show don't really matter much - or at least might actually reflect an accuracy approximatly what a human would also achieve. Their headline claim is a "1 pixel change reduces accuracy by 30%". The test process for that number is this: We choose a random square within the original image and resize the square to be 224x224. The size and location of the square are chosen randomly according to the distribution described in (Szegedy et al., 2015). We then shift that square by one pixel diagonally to create a second image that differs from the first one by translation by a single pixel. So... they are taking a random square, downsampling to 224, moving and then predicting on that subset of the original image, and measuring the performance against the accuracy of the original prediction. What this seems to show is that "CNNs aren't as accurate at making predictions on subsets of an image as on the whole image". This is of course to be expected, and is exactly how a human would perform. There are a bunch of other criticisms too. Read the ICLR (for which it was rejected) reviews: https://openreview.net/forum?id=HJxYwiC5tm https://openreview.net/forum?id=HJxYwiC5tm
visarga 6y ago
Just memorising data is simple, use a file on disk. The hard part is recognising data when it is slightly different than what the model 'memorised', deciding which of the millions of things it learned best fits the answer we desire.

nl 6y ago

> It seems it’s still an open question whether AI is just about memorizing your data, No - it's not at all, and this is a well understood problem in building machine learning systems. There are some cases where this occurs but usually this is just overfitting. Good AI systems generalize well on unseen data.

nl 6y ago
To the downvoters, I give you AlphaZero. Not only is every game of Go it plays and wins brand new (so no memorisation), the same system learnt to play Chess without knowing the rules, and plays in a "style .. unlike any traditional chess engine" https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go https://deepmind.com/blog/article/alphazero-shedding-new-lig...