4 ms·

The title is missleading! The code is checked when it is pushed to the main repository... The code and the bug already exists, the developer is responsible for

The title is missleading! The code is checked when it is pushed to the main repository... The code and the bug already exists, the developer is responsible for that. Also, if you look at the video from Youtube, you can see that 60% + 30% = 100% : https://www.youtube.com/watch?v=I5C4FUvDyCc&t=50s https://www.youtube.com/watch?v=I5C4FUvDyCc&t=50s I was hoping for some kind of godlike AI to help catching bugs before I write my code, now I'm disappointed...

racl101 9y ago
> I was hoping for some kind of godlike AI to help catching bugs before I write my code, now I'm disappointed... Maybe it's not such a bad thing if AI is not godlike.
paxy 9y ago
If the "godlike AI" could find bugs before you wrote the code, it may as well just write the code instead of you.
- wavefunction 9y ago
  It might be interesting to apply machine learning to already written code, correlating common mistakes to specific developers so that the IDE might warn the developer that they're entering some area or process where they tend to make mistakes.
  - gruez 9y ago
    the problem with that is you get an opaque metric that's not very helpful, see: https://news.ycombinator.com/item?id=16569767 https://news.ycombinator.com/item?id=16569767
- lou1306 9y ago
  I mean, probably many elementary code smells (e.g. those related to null references) just pop up over and over again in a sufficiently large code base, so an AI could reasonably be trained detect them and thus prevent some of the bugs that arise from them. But for application-level bugs, we're SOL... And luckily so, otherwise we would be jobless in a few years or so.
  - RangerScience 9y ago
    > And luckily so, otherwise we would be jobless in a few years or so. I don't think so. You know those painful conversations with people who "have a great idea", but can't actually make it? It's not because they don't have technical skills (although they don't) - it's because they're not trained in thinking to the level of detail that implementation actually requires. Someone needs to "program the AI" (whatever that ends up meaning). That'll still be us, even if it doesn't require any code, because at some point what code actually is telling something what you want it to do. It's "the same" leap as from machine code to compilers, domain-specific languages, garbage collection, etc. You describe what you want to happen at some level of abstraction, and something interprets that to actually make it happen. The level of detail you, as the programmer, have to directly describe goes down over time, but you still deal with far more details than most other people.
- SatvikBeri 9y ago
  An analogy here is NP problems - it might be a lot easier to detect bugs than to generate bug-free code. Compilers actually do this on a regular basis, detecting bugs like typos and type errors.
  - eftychis 9y ago
    I think the joke here is the implication the AI can guess your code. But yeah it would be hilarious if there was a complexity class on predicting what the problem solver is going to write.
chrisfosterelli 9y ago
> Also, if you look at the video from Youtube, you can see that 60% + 30% = 100% They said it catches 60% of bugs and has 30% false alarms. Those are different measurements, the sum of true positives and false positives doesn't need to add up to 100%.
- lrem 9y ago
  So you're saying that 10% of time it says there's no bug? ;)
  - deleted 9y ago
    [deleted]
  - eftychis 9y ago
    > So you're saying that 10% of time it says there's no bug? ;) No. Recall the wording: "They said it catches 60% of bugs and has 30% false alarms. Those are different measurements, the sum of true positives and false positives doesn't need to add up to 100%." Hence (nitpicking perhaps): 100%-60%=40% of the bugs (their estimation) are not caught (false negatives) The rate of false positives is 30%, that is if a particular code excerpt has no bugs it might be tagged as buggy. Thus the final error rate I guess is (actual probability of a bug)60% + (1- (actual probability of a bug))30%.
- qbrass 9y ago
  What it does mean is that that unless 1/3 or more of your code is buggy, it will report more false positives than bugs.
  - chrisfosterelli 9y ago
    Yeah considering in terms of a binary classification the accuracy is not super impressive imo. It could just be learning statistical patterns for code that is more likely to have issues rather than identifying actual bugs. They did mention is does other things such as suggesting improvements, so maybe it does provide more tangible value in those ways. Hard to say without more information!
seanmcdirmid 9y ago
Minority report for software developers, sounds like a good recruiting startup idea.

racl101 9y ago

> I was hoping for some kind of godlike AI to help catching bugs before I write my code, now I'm disappointed... Maybe it's not such a bad thing if AI is not godlike.

paxy 9y ago

If the "godlike AI" could find bugs before you wrote the code, it may as well just write the code instead of you.

wavefunction 9y ago
It might be interesting to apply machine learning to already written code, correlating common mistakes to specific developers so that the IDE might warn the developer that they're entering some area or process where they tend to make mistakes.
- gruez 9y ago
  the problem with that is you get an opaque metric that's not very helpful, see: https://news.ycombinator.com/item?id=16569767 https://news.ycombinator.com/item?id=16569767
lou1306 9y ago
I mean, probably many elementary code smells (e.g. those related to null references) just pop up over and over again in a sufficiently large code base, so an AI could reasonably be trained detect them and thus prevent some of the bugs that arise from them. But for application-level bugs, we're SOL... And luckily so, otherwise we would be jobless in a few years or so.
- RangerScience 9y ago
  > And luckily so, otherwise we would be jobless in a few years or so. I don't think so. You know those painful conversations with people who "have a great idea", but can't actually make it? It's not because they don't have technical skills (although they don't) - it's because they're not trained in thinking to the level of detail that implementation actually requires. Someone needs to "program the AI" (whatever that ends up meaning). That'll still be us, even if it doesn't require any code, because at some point what code actually is telling something what you want it to do. It's "the same" leap as from machine code to compilers, domain-specific languages, garbage collection, etc. You describe what you want to happen at some level of abstraction, and something interprets that to actually make it happen. The level of detail you, as the programmer, have to directly describe goes down over time, but you still deal with far more details than most other people.
SatvikBeri 9y ago
An analogy here is NP problems - it might be a lot easier to detect bugs than to generate bug-free code. Compilers actually do this on a regular basis, detecting bugs like typos and type errors.
- eftychis 9y ago
  I think the joke here is the implication the AI can guess your code. But yeah it would be hilarious if there was a complexity class on predicting what the problem solver is going to write.

chrisfosterelli 9y ago

> Also, if you look at the video from Youtube, you can see that 60% + 30% = 100% They said it catches 60% of bugs and has 30% false alarms. Those are different measurements, the sum of true positives and false positives doesn't need to add up to 100%.

lrem 9y ago
So you're saying that 10% of time it says there's no bug? ;)
- deleted 9y ago
  [deleted]
- eftychis 9y ago
  > So you're saying that 10% of time it says there's no bug? ;) No. Recall the wording: "They said it catches 60% of bugs and has 30% false alarms. Those are different measurements, the sum of true positives and false positives doesn't need to add up to 100%." Hence (nitpicking perhaps): 100%-60%=40% of the bugs (their estimation) are not caught (false negatives) The rate of false positives is 30%, that is if a particular code excerpt has no bugs it might be tagged as buggy. Thus the final error rate I guess is (actual probability of a bug)60% + (1- (actual probability of a bug))30%.
qbrass 9y ago
What it does mean is that that unless 1/3 or more of your code is buggy, it will report more false positives than bugs.
- chrisfosterelli 9y ago
  Yeah considering in terms of a binary classification the accuracy is not super impressive imo. It could just be learning statistical patterns for code that is more likely to have issues rather than identifying actual bugs. They did mention is does other things such as suggesting improvements, so maybe it does provide more tangible value in those ways. Hard to say without more information!

seanmcdirmid 9y ago

Minority report for software developers, sounds like a good recruiting startup idea.