3 ms·

The fly in the ointment being that, if such useless loops are widely adopted, ML can be trained to recognize and eliminate them too. (Don't shoot the messenger

The fly in the ointment being that, if such useless loops are widely adopted, ML can be trained to recognize and eliminate them too. (Don't shoot the messenger; the concept is obvious.)

shadowgovt 5y ago
You could train one to recognize common patterns, and you could train one to heuristically guess that an algo running on test inputs is looping forever if it takes too long, but actually dividing the world into "loops forever" and "does not" is the Halting Problem, and our solution is guaranteed to either false-positive or false-negative. (Anecdote: I heard a story once about a college that transitioned from free printing to metered printing, but since the printers didn't support metering, they implemented it by running print jobs in a virtual machine and tallying page count there. It took not very much time for some compsci student to cobble together a script that would augment a Postscript file with an "if my environment is virtual, print one blank page and exit, else..." prefix).
- ThrowawayR2 5y ago
  As you point out elsewhere, there's a human in the loop. Though the Halting Problem applies to human minds as well, they likely won't have any difficulty identifying useless code the ML doesn't pick up and adding it to the training set.
  - Qwertious 5y ago
    The next step is to integrate actual parts of the software into the poison-pill, so that if the poison-pill is not included then the software doesn't work. It isn't necessarily a good idea, but it's the next obvious step.
    - shadowgovt 5y ago
      Hard to make that work with snippets as small as Copilot is synthesizing.
sp332 5y ago
That's true if the same mitigations often show up with the same contexts, like the simplistic one in the example. Making a general mitigation-avoiding AI would require it to understand the intention of the code.

3 ms·

The fly in the ointment being that, if such useless loops are widely adopted, ML can be trained to recognize and eliminate them too. (Don't shoot the messenger

by ThrowawayR2 5y ago

The fly in the ointment being that, if such useless loops are widely adopted, ML can be trained to recognize and eliminate them too. (Don't shoot the messenger; the concept is obvious.)

shadowgovt 5y ago
You could train one to recognize common patterns, and you could train one to heuristically guess that an algo running on test inputs is looping forever if it takes too long, but actually dividing the world into "loops forever" and "does not" is the Halting Problem, and our solution is guaranteed to either false-positive or false-negative. (Anecdote: I heard a story once about a college that transitioned from free printing to metered printing, but since the printers didn't support metering, they implemented it by running print jobs in a virtual machine and tallying page count there. It took not very much time for some compsci student to cobble together a script that would augment a Postscript file with an "if my environment is virtual, print one blank page and exit, else..." prefix).
- ThrowawayR2 5y ago
  As you point out elsewhere, there's a human in the loop. Though the Halting Problem applies to human minds as well, they likely won't have any difficulty identifying useless code the ML doesn't pick up and adding it to the training set.
  - Qwertious 5y ago
    The next step is to integrate actual parts of the software into the poison-pill, so that if the poison-pill is not included then the software doesn't work. It isn't necessarily a good idea, but it's the next obvious step.
    - shadowgovt 5y ago
      Hard to make that work with snippets as small as Copilot is synthesizing.
sp332 5y ago
That's true if the same mitigations often show up with the same contexts, like the simplistic one in the example. Making a general mitigation-avoiding AI would require it to understand the intention of the code.