3 ms·
My favorite part of the paper, in the section discussing how small prompt variations affect results: > M-2: We set the Python author flag [in the prompt] to th
by MauranKilom 5y ago
My favorite part of the paper, in the section discussing how small prompt variations affect results:
> M-2: We set the Python author flag [in the prompt] to the lead author of this paper. Sadly, it increases the number of vulnerabilities.
> M-3: We changed the indentation style from spaces to
tabs and the number of vulnerable suggestions increased somewhat, as did the confidence of the vulnerable answers.
The top-scoring option remained non-vulnerable.
@authors: I think something is wrong in the phrasing for M-4 (or some text got jumbled). Was the top-scoring option vulnerable or not? The second half might belong to D-3 instead (where no assessment is given)?
- kiwih 5y agoThanks for your feedback! Yes, unfortunately Copilot doesn't seem to think much of my programming security... > @authors: I think something is wrong in the phrasing for M-4 (or some text got jumbled). Was the top-scoring option vulnerable or not? The second half might belong to D-3 instead (where no assessment is given)? Argh, you're absolutely correct. Five authors proofreading the paper and still these things slip through. Fortunately we can fix it
- comex 5y agoMore nits: - In Fig. 6 (Scenario 787-08), lines 2-4 appear to be truncated. [edit: I see you already responded to this elsewhere in the thread.] - The layout of the figure labels is confusing; at first glance it looks more like they're labeling what's under them, rather than above them. - The quotation marks are all turned into curly closing quotes. - This is more of a style thing, but the code snippets appear to be using a variable-width font laid out as if it were fixed width, which looks pretty bad. Perhaps the font isn't embedded properly.
- dlgeek 5y agoRe: M-3: Let the tab-vs-spaces war flare up again.