5 ms·

Context: there was this tweet [1] that showed how co-pilot generated word for word he famous inverse square root function from the quake code. The tweet generat

Context: there was this tweet [1] that showed how co-pilot generated word for word he famous inverse square root function from the quake code. The tweet generated some press and as a result copilot added the function name to the blacklist of slurs so this won’t happen again [1]: https://twitter.com/mitsuhiko/status/1410886329924194309 https://twitter.com/mitsuhiko/status/1410886329924194309

johnday 5y ago
Reminds me of the old joke "He was imprisoned for 12 years for killing his wife, but let out early for not doing it since".
josephcsible 5y ago
So instead of fixing the problem with the algorithm that makes Copilot occasionally spit back unmodified code from its training set, Microsoft just hardcoded a check to stop the one most well-known instance from happening, so that they can pretend they fixed it?
- bpodgursky 5y ago
  In the generous interpretation, they hotfixed this to resolve the immediate issue while working on a real fix on the backend during training. (I make no claim that this is actually what is happening, it's just not incompatible with behavior in the generous case).
  - josephcsible 5y ago
    But why did this "immediate issue" even need a fix? What problem was it causing, other than making Microsoft look bad by showing that the real problem exists? Imagine if a file sharing site said they'd scan all uploads for viruses, but it turns out they weren't, which someone discovered by uploading EICAR. Should the site hardcode a check to block EICAR until they actually get virus scanning working?
    - freeone3000 5y ago
      > What problem was it causing, other than making Microsoft look bad by showing that the real problem exists? That's the real problem. The issue is that Microsoft looks bad. The problem was solved by making Microsoft not look as bad. Verbatim output of inputs is not a problem, it's an understood property of the model.
      - foxfluff 5y ago
        > Verbatim output of inputs is not a problem So wait, was the obvious copyright elephant in the room solved somehow? > The issue is that Microsoft looks bad. The problem was solved by making Microsoft not look as bad. Depending on who you ask, adding a hack like this in an attempt to make them not look as bad just makes it look worse. Especially when the hack is discovered.
        freeone3000 5y ago
        Try getting a non-technical person to understand this issue beyond the level of "okay, so it output some copyrighted code, then they blocked that code from being output. Sounds like they fixed it". That's the obvious PR angle, and once the big, widely-publicized issue is fixed, it's hard to get another article with that much traction. Besides, if another snippet becomes popular, it can be blocked the same way. The issue is fundamentally a PR one, not a copyright one, since the responsibility for using or not using the code relies on the end-user. GitHub has the rights to use your code for training.
        Smaug123 5y ago
        I don't think there's any technical knowledge needed here; the concepts are common ones. "There was a big hoo-hah about how it used to output a certain well-known piece of code in possible violation of its licence. They fixed it by blocking any output that includes the name of that exact paragraph of code. In particular, it can still output other code from the same codebase with the same licence; just not that particular paragraph." The direct analogy would be trying to stop it from outputting The Lord of the Rings by blocking the phrase "a long-expected party" (the title of its first chapter).
- amoshi 5y ago
  That's standard practice afaik, it's a machine learning blackbox after all Google ‘fixed’ its racist algorithm by removing gorillas from its image-labeling tech - https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...
  - klodolph 5y ago
    Isn’t the reason here that it takes weeks or months to make changes to machine learning models of this size? You can’t “patch” a model, so you stick a simple filter on the model while you figure out how to train a new one.
- zekrioca 5y ago
  Reminds me of the way MS used to identify their software version [1]. It seems it is still how they pragmatically solve their problems :) [1] if (version.StartsWith(“Windows 9”)) { /* 95 and 98 */ } else { http://www.reddit.com/r/technology/comments/2hwlrk/new_windows_version_will_be_called_windows_10/ckwq83x http://www.reddit.com/r/technology/comments/2hwlrk/new_windo...
  - Gaelan 5y ago
    I don't think it was MS that did that, just sloppy third-party developers.
    - ChrisKnott 5y ago
      It's also baseless speculation. Better sources seem to agree it's because MS wanted a "clean break" from Windows 8, which got bad press. This makes much more sense than bending to poorly written apps from 20yrs ago.
      - bonzini 5y ago
        Bending to poorly written apps from 20yrs ago is what Microsoft has been doing with Windows all the time.
  - deleted 5y ago
    [deleted]
- bayindirh 5y ago
  > Microsoft just hardcoded a check to stop the one most well-known instance from happening, so that they can pretend they fixed it? Isn't it typical Microsoft, in some 90s and 2000s sense?

johnday 5y ago

Reminds me of the old joke "He was imprisoned for 12 years for killing his wife, but let out early for not doing it since".

josephcsible 5y ago

So instead of fixing the problem with the algorithm that makes Copilot occasionally spit back unmodified code from its training set, Microsoft just hardcoded a check to stop the one most well-known instance from happening, so that they can pretend they fixed it?

bpodgursky 5y ago
In the generous interpretation, they hotfixed this to resolve the immediate issue while working on a real fix on the backend during training. (I make no claim that this is actually what is happening, it's just not incompatible with behavior in the generous case).
- josephcsible 5y ago
  But why did this "immediate issue" even need a fix? What problem was it causing, other than making Microsoft look bad by showing that the real problem exists? Imagine if a file sharing site said they'd scan all uploads for viruses, but it turns out they weren't, which someone discovered by uploading EICAR. Should the site hardcode a check to block EICAR until they actually get virus scanning working?
  - freeone3000 5y ago
    > What problem was it causing, other than making Microsoft look bad by showing that the real problem exists? That's the real problem. The issue is that Microsoft looks bad. The problem was solved by making Microsoft not look as bad. Verbatim output of inputs is not a problem, it's an understood property of the model.
    - foxfluff 5y ago
      > Verbatim output of inputs is not a problem So wait, was the obvious copyright elephant in the room solved somehow? > The issue is that Microsoft looks bad. The problem was solved by making Microsoft not look as bad. Depending on who you ask, adding a hack like this in an attempt to make them not look as bad just makes it look worse. Especially when the hack is discovered.
      - freeone3000 5y ago
        Try getting a non-technical person to understand this issue beyond the level of "okay, so it output some copyrighted code, then they blocked that code from being output. Sounds like they fixed it". That's the obvious PR angle, and once the big, widely-publicized issue is fixed, it's hard to get another article with that much traction. Besides, if another snippet becomes popular, it can be blocked the same way. The issue is fundamentally a PR one, not a copyright one, since the responsibility for using or not using the code relies on the end-user. GitHub has the rights to use your code for training.
        Smaug123 5y ago
        I don't think there's any technical knowledge needed here; the concepts are common ones. "There was a big hoo-hah about how it used to output a certain well-known piece of code in possible violation of its licence. They fixed it by blocking any output that includes the name of that exact paragraph of code. In particular, it can still output other code from the same codebase with the same licence; just not that particular paragraph." The direct analogy would be trying to stop it from outputting The Lord of the Rings by blocking the phrase "a long-expected party" (the title of its first chapter).
amoshi 5y ago
That's standard practice afaik, it's a machine learning blackbox after all Google ‘fixed’ its racist algorithm by removing gorillas from its image-labeling tech - https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...
- klodolph 5y ago
  Isn’t the reason here that it takes weeks or months to make changes to machine learning models of this size? You can’t “patch” a model, so you stick a simple filter on the model while you figure out how to train a new one.
zekrioca 5y ago
Reminds me of the way MS used to identify their software version [1]. It seems it is still how they pragmatically solve their problems :) [1] if (version.StartsWith(“Windows 9”)) { /* 95 and 98 */ } else { http://www.reddit.com/r/technology/comments/2hwlrk/new_windows_version_will_be_called_windows_10/ckwq83x http://www.reddit.com/r/technology/comments/2hwlrk/new_windo...
- Gaelan 5y ago
  I don't think it was MS that did that, just sloppy third-party developers.
  - ChrisKnott 5y ago
    It's also baseless speculation. Better sources seem to agree it's because MS wanted a "clean break" from Windows 8, which got bad press. This makes much more sense than bending to poorly written apps from 20yrs ago.
    - bonzini 5y ago
      Bending to poorly written apps from 20yrs ago is what Microsoft has been doing with Windows all the time.
- deleted 5y ago
  [deleted]
bayindirh 5y ago
> Microsoft just hardcoded a check to stop the one most well-known instance from happening, so that they can pretend they fixed it? Isn't it typical Microsoft, in some 90s and 2000s sense?