4 ms·

Would QA solve that problem? As in, "we replayed 1.4 trillion accident scenarios and our AI behaved correctly on all of them". Safety in numbers.

gccs 5y ago
"We trained a model, overfit on 1.4 trillion accident scenarios, and if behaved correctly on all of them."
- bpodgursky 5y ago
  Unless your model actually has trillions of parameters (and it doesn't, even gpt-3 only has 175 billion) it is not even possible to overfit on 1.4 trillion training inputs. You can't actually pigeonhole it.
  - deleted 5y ago
    [deleted]
  - karpierz 5y ago
    Suppose that you train a neural network to predict the next number in an arithmetic sequence (a, a+b, a+2b, a+3b, a+4b, ...). As input it gets two numbers, the last number and the current number and has to predict the next one. Suppose you had 1.4 trillion examples in the following test set (using a model with 175 billion parameters): (1,2)->3 (2,3)->4 (3,4)->5 ... Do you think it is possible to overfit and score perfect on the test set, while failing to generalize?
    - FeepingCreature 5y ago
      That isn't overfit, that's fit. Nothing can protect you if your training set just doesn't have any indication of the thing you want it to learn.
      - karpierz 5y ago
        I didn't specify that the training set wasn't representative. All this shows is that you don't need parameters anywhere close to the number of test examples to overfit.
        FeepingCreature 5y ago
        And my point is that is not what overfit is. Overfit is a specific problem where the network fails to recognize a commonality in the training set and instead interprets the irrelevant details of some subset of training samples (in the extreme, individual samples) as distinct properties. Your example training set is not filled with noise that the network is picking up on to its detriment. Your example training set is simply not representative of the function you are trying to teach.
        karpierz 5y ago
        I don't have an example training set. I don't have an example model. My exact point is that if your test set isn't representative of the underlying distribution, then accuracy on the test set doesn't mean that your model isn't overfit.
    - joshuamorton 5y ago
      I think you've specified this problem in a very strange way. But if you're saying that you're trying to train on the specific dataset where a = 1 and b = 1, then your model will fit the data perfectly with 175 billion parameters. It will also fit the data perfectly with, like, 15 parameters. If you're trying to fit to some more complex space where a and b are unknown and you're given 3 numbers in the sequence, then what you're trying to fit is `f(a, b) = a + 2(b - a)` (or 2b - a, however you want to represent it), which is a swell function, but if you only give data that can be equally represented by `f(a, b) = b + 1`, you're mis-training your model. But you could once again do that with a model with a dozen parameters. In both cases, the issue isn't overfitting, but misrepresentative data.
      - karpierz 5y ago
        I didn't specify the training set, just the test set. It's possible that your model actually models an arithmetic series. Or that it simply overfits. The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set.
        joshuamorton 5y ago
        What you need are more parameters than the complexity of the underlying distribution. If you drop to a linear function you're modelling, you only need a couple of parameters. "Overfitting" is memorizing the training data instead of generalizing. The example you're providing isn't overfitting, it's just generalizing to the wrong function. Overfitting would be if the validation set was, say, 30 random values that you got right, but didn't get other values along the same lines correct. > I didn't specify the training set, just the test set Then unless you constructed the training set with the intent of mistraining the model, I think a training set that got good accuracy on that validation set would generalize. > The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set. You can't "overfit" a validation set, unless you've done something wrong. Overfitting is, by definition, learning the training set too well such that you fail to generalize to a validation set.
        5y ago
- ALittleLight 5y ago
  What if the 1.4 trillion accident scenarios were the test set?
- egillie 5y ago
  k-fold cross validation
Invictus0 5y ago
It depends. How many of those scenarios were in fog conditions? How many where a spotlight suddenly blinds some of the cameras? How many where a power line is down, or a manhole cover is missing, or a barn owl/hippo/bear is in the road? If we estimate that it takes 10e5 to 10e6 incidents of a certain type to appropriately train the network for that situation... it's just too difficult. Reality contains infinite edge cases.
- colordrops 5y ago
  It's a long tail that goes on to virtually infinity, yes, but the collective probability of the unhandled edge cases will become vanishingly small, and just an accepted part of living in our reality, which is not perfect and impossible to fully predict and control.

gccs 5y ago

"We trained a model, overfit on 1.4 trillion accident scenarios, and if behaved correctly on all of them."

bpodgursky 5y ago
Unless your model actually has trillions of parameters (and it doesn't, even gpt-3 only has 175 billion) it is not even possible to overfit on 1.4 trillion training inputs. You can't actually pigeonhole it.
- deleted 5y ago
  [deleted]
- karpierz 5y ago
  Suppose that you train a neural network to predict the next number in an arithmetic sequence (a, a+b, a+2b, a+3b, a+4b, ...). As input it gets two numbers, the last number and the current number and has to predict the next one. Suppose you had 1.4 trillion examples in the following test set (using a model with 175 billion parameters): (1,2)->3 (2,3)->4 (3,4)->5 ... Do you think it is possible to overfit and score perfect on the test set, while failing to generalize?
  - FeepingCreature 5y ago
    That isn't overfit, that's fit. Nothing can protect you if your training set just doesn't have any indication of the thing you want it to learn.
    - karpierz 5y ago
      I didn't specify that the training set wasn't representative. All this shows is that you don't need parameters anywhere close to the number of test examples to overfit.
      - FeepingCreature 5y ago
        And my point is that is not what overfit is. Overfit is a specific problem where the network fails to recognize a commonality in the training set and instead interprets the irrelevant details of some subset of training samples (in the extreme, individual samples) as distinct properties. Your example training set is not filled with noise that the network is picking up on to its detriment. Your example training set is simply not representative of the function you are trying to teach.
        karpierz 5y ago
        I don't have an example training set. I don't have an example model. My exact point is that if your test set isn't representative of the underlying distribution, then accuracy on the test set doesn't mean that your model isn't overfit.
  - joshuamorton 5y ago
    I think you've specified this problem in a very strange way. But if you're saying that you're trying to train on the specific dataset where a = 1 and b = 1, then your model will fit the data perfectly with 175 billion parameters. It will also fit the data perfectly with, like, 15 parameters. If you're trying to fit to some more complex space where a and b are unknown and you're given 3 numbers in the sequence, then what you're trying to fit is `f(a, b) = a + 2(b - a)` (or 2b - a, however you want to represent it), which is a swell function, but if you only give data that can be equally represented by `f(a, b) = b + 1`, you're mis-training your model. But you could once again do that with a model with a dozen parameters. In both cases, the issue isn't overfitting, but misrepresentative data.
    - karpierz 5y ago
      I didn't specify the training set, just the test set. It's possible that your model actually models an arithmetic series. Or that it simply overfits. The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set.
      - joshuamorton 5y ago
        What you need are more parameters than the complexity of the underlying distribution. If you drop to a linear function you're modelling, you only need a couple of parameters. "Overfitting" is memorizing the training data instead of generalizing. The example you're providing isn't overfitting, it's just generalizing to the wrong function. Overfitting would be if the validation set was, say, 30 random values that you got right, but didn't get other values along the same lines correct. > I didn't specify the training set, just the test set Then unless you constructed the training set with the intent of mistraining the model, I think a training set that got good accuracy on that validation set would generalize. > The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set. You can't "overfit" a validation set, unless you've done something wrong. Overfitting is, by definition, learning the training set too well such that you fail to generalize to a validation set.
        5y ago
ALittleLight 5y ago
What if the 1.4 trillion accident scenarios were the test set?
egillie 5y ago
k-fold cross validation

Invictus0 5y ago

It depends. How many of those scenarios were in fog conditions? How many where a spotlight suddenly blinds some of the cameras? How many where a power line is down, or a manhole cover is missing, or a barn owl/hippo/bear is in the road? If we estimate that it takes 10e5 to 10e6 incidents of a certain type to appropriately train the network for that situation... it's just too difficult. Reality contains infinite edge cases.

colordrops 5y ago
It's a long tail that goes on to virtually infinity, yes, but the collective probability of the unhandled edge cases will become vanishingly small, and just an accepted part of living in our reality, which is not perfect and impossible to fully predict and control.