3 ms·

Would something like this be a good use case for copilot / gpt based coding? You have massive specs in relatively structured documents, which could be used in i

Would something like this be a good use case for copilot / gpt based coding? You have massive specs in relatively structured documents, which could be used in isolated parses as prompts. The particular tuning of the model shouldn't matter if the source specifications and good examples are in the training data already. Use few shot prompts with well put together examples, then iterating over every item in the specs, and finally running the generated code through a human review process - seems like you could shift a lot of effort towards validation, and in doubt so, devise a process subject to further automation. Crawling the resulting codebase with various prompts like "this part of the specification could be improved by..." and then review a set of 20 completions for any real value. This might work for xml,html, and so forth, but even if you only have a system that generates a baseline library for one standard, it could be powerful.

stouset 5y ago
Comprehension of the details of a security standard is necessary for semantically correct and secure implementation of that security standard. I can't personally fathom how throwing GPT at the problem will improve things, but I can trivially see how it could result in code that looks correct to the author but isn't to anyone who read the spec. Maybe that's a failure of imagination on my part. Then again, I suppose that's the situation we're already in where well-meaning authors don't think to actually read the specs and so don't actually comprehend any of the details.
ak217 5y ago
Most problems with security specs and libraries that implement them are communication problems. They involve people incompletely describing or understanding their requirements, capabilities, or threat model. Often this also involves providing/using interfaces that are not ergonomic (https://github.com/google/mundane/blob/master/DESIGN.md https://github.com/google/mundane/blob/master/DESIGN.md), which in turn can be caused by the spec trying to do too much (as XML Signature does). I don't know how GPT could help with that. If anything I would expect it to bias toward things it has already seen, which is the opposite of what you want when writing a new spec/library aiming to avoid past mistakes.
jcranmer 5y ago
To an extremely coarse approximation, a lot of AI techniques boil down to similarity matching. If you've got an algorithm that is most commonly implemented wrongly (binary search is the classic example here [1]), I would fully expect something like Copilot to happily reproduce the common wrong code instead of the rare correct code. I mean, it's already been established that Copilot happily suggests completions susceptible to SQL injection attacks, and that's a security flaw that "everyone" is aware about. And as sibling comment alludes to, security implications are very frequently areas that require very deep knowledge both of the standard and of how breaks in security protocols usually end up happening. How many commenters on this article were aware of the potential for canonicalization attacks in security before reading the article or the comments here? I'd wager relatively few. [1] If you're wondering how almost all binary searches could be wrong, the answer is they fail to account for overflow in calculating the midpoint.

stouset 5y ago

Comprehension of the details of a security standard is necessary for semantically correct and secure implementation of that security standard. I can't personally fathom how throwing GPT at the problem will improve things, but I can trivially see how it could result in code that looks correct to the author but isn't to anyone who read the spec. Maybe that's a failure of imagination on my part. Then again, I suppose that's the situation we're already in where well-meaning authors don't think to actually read the specs and so don't actually comprehend any of the details.

ak217 5y ago

Most problems with security specs and libraries that implement them are communication problems. They involve people incompletely describing or understanding their requirements, capabilities, or threat model. Often this also involves providing/using interfaces that are not ergonomic (https://github.com/google/mundane/blob/master/DESIGN.md https://github.com/google/mundane/blob/master/DESIGN.md), which in turn can be caused by the spec trying to do too much (as XML Signature does). I don't know how GPT could help with that. If anything I would expect it to bias toward things it has already seen, which is the opposite of what you want when writing a new spec/library aiming to avoid past mistakes.

jcranmer 5y ago

To an extremely coarse approximation, a lot of AI techniques boil down to similarity matching. If you've got an algorithm that is most commonly implemented wrongly (binary search is the classic example here [1]), I would fully expect something like Copilot to happily reproduce the common wrong code instead of the rare correct code. I mean, it's already been established that Copilot happily suggests completions susceptible to SQL injection attacks, and that's a security flaw that "everyone" is aware about. And as sibling comment alludes to, security implications are very frequently areas that require very deep knowledge both of the standard and of how breaks in security protocols usually end up happening. How many commenters on this article were aware of the potential for canonicalization attacks in security before reading the article or the comments here? I'd wager relatively few. [1] If you're wondering how almost all binary searches could be wrong, the answer is they fail to account for overflow in calculating the midpoint.