6 ms·

I wrote build scripts for my projects that tracked: 1. Syntax errors on compile (it was C). 2. Test failures. 3. Time of the run. 4. Lines of code. Then I use

by zedshaw 16y ago

I wrote build scripts for my projects that tracked: 1. Syntax errors on compile (it was C). 2. Test failures. 3. Time of the run. 4. Lines of code. Then I used R to create graphs and reports that were generated periodically. About once an hour I'd go look at them and see what synax+failure levels were per line of code at that time. I'd then plot this as a run chart and start trying to find causes of my high defect indicators. Eventually it got to where I found a bunch of the ways I wrote defective code and started to avoid them. I have a set of macros for C now that prevent it, and I use the patterns in my other coding. It was pretty insane though. I don't recommend it for anyone, and definitely not on a project you have to do for a living. Well, maybe if it's a missile command system or medical device.

JesseAldridge 16y ago
This is awesome. Do you remember any of the causes of high defect indicators? And why wouldn't you recommend it? Too much work? I made a thing that takes a screenshot every ten seconds while I work and another thing that sticks them in a window with a slider so I can look at them in sequence. Just yesterday I noticed that I had been daydreaming a lot lately. So I made something that would go through the sequence of screenshots and compare each image with its previous one and count every instance where the two images were the same (or almost the same). That gave me a pretty good indicator of how much time I spent daydreaming. If the numbers are right, then when I'm working on something boring I spend as much as 40% of my time not actively typing or anything -- just thinking about random shit. I'm thinking I'll try mindfulness meditation for a while and see if that cuts down on the daydreaming at all. We'll see how it goes...
- zedshaw 16y ago
  Yep, I wrote a bunch of C macros to prevent them and use them everywhere: curl http://zedshaw.com/dbg.h http://zedshaw.com/dbg.h The main thing that caused defects was not handling default cases in logic. Things like not filling in the dangling else of an if-statement, not trying to return a value from a function no matter what, and not having a default: in a switch. By always trying to include at least a comment there, or an assert it forces you to think about the logic of the whole statement and prevents bugs galore. Combine that with unit tests and you've got a good solid start. I don't recommend it because it's an insane amount of work above just writing code. I think it'd be something I'd force on high speed students in a kind of "Boot Camp" situation, but not on anyone else. In fact, I think most programmers couldn't handle having a giant bag of evidence staring them in the face telling them they suck. Also I think most programmers wouldn't get the stats behind the analysis and just keep screwing it up. I play guitar when I have to think about stuff. I take one to work, and when I'm stuck I bust out my guitar, a pen, and a note pad. I then noodle and practice, and as ideas come into my head I write them down without filtering. Basically, you need a hobby. :-)
  - agazso 16y ago
    In most recent C compilers you can turn on warnings for at least missing return values from functions and not having a default: in a switch. It is good practice to compile C/C++ programs with high warning level, and fix all the warnings before shipping.
- wnoise 16y ago
  When I'm writing code, a lot of the time the screen isn't going to change. This is not because I'm day dreaming, but because I'm thinking.
JoshCole 16y ago
Has anyone made a plugin or anything of that sort which abstracts away the grunt work of using this sort of system? It sounds like a great thing to try out were it not for the opportunity cost of trying it.
- zedshaw 16y ago
  Not that I know of. I imagine emacs or vim has good error-line parsing which you could steal/copy and get really good results. You could then extend it to work on any language you wanted. Biggest thing I couldn't get right was automatically connecting frequency of errors to particular lines or modules. I found that I made the same mistakes over and over, and I had to just spot them and jot down what not to do or think of a ninja way to avoid it. I also found that there was a strong correlation between syntax/test failures while writing code and how bad the code was. What would have helped is if I could calculate the stats per function, line, or file and then have some pattern analysis spit out common grammatical structures that I do which are correlated with errors. It'd be something like: function/while/if/return 24% chance of defect found in blah.c:func1 and blah.c:func2 Then it'd get closer to predicting defects and letting me go check the spots it recommends. But, never got around to it since it became really insane after a year already without that level of analysis.
- windsurfer 16y ago
  Hudson AKA Jenkins? I'm not sure if that's exactly what you want, but it worked wonders at the job I had.
- torpor 16y ago
  Coverity?
yters 16y ago
Have you tried delta debugging? http://en.wikipedia.org/wiki/Delta_Debugging http://en.wikipedia.org/wiki/Delta_Debugging
- zedshaw 16y ago
  Yes, I have. We used to call it "debugging", but I guess everything has to have a name before the cool kids will use it. However, this wasn't about find a bug but preventing defects by using evidence to retrain my behavior. The approach is to assume nothing, then use simple graphs and data to look for places where you can change how you do stuff to improve your code. It's not hard, just really tedious.
  - yters 16y ago
    It's automated debugging.

by zedshaw 16y ago

JesseAldridge 16y ago

This is awesome. Do you remember any of the causes of high defect indicators? And why wouldn't you recommend it? Too much work? I made a thing that takes a screenshot every ten seconds while I work and another thing that sticks them in a window with a slider so I can look at them in sequence. Just yesterday I noticed that I had been daydreaming a lot lately. So I made something that would go through the sequence of screenshots and compare each image with its previous one and count every instance where the two images were the same (or almost the same). That gave me a pretty good indicator of how much time I spent daydreaming. If the numbers are right, then when I'm working on something boring I spend as much as 40% of my time not actively typing or anything -- just thinking about random shit. I'm thinking I'll try mindfulness meditation for a while and see if that cuts down on the daydreaming at all. We'll see how it goes...

zedshaw 16y ago
Yep, I wrote a bunch of C macros to prevent them and use them everywhere: curl http://zedshaw.com/dbg.h http://zedshaw.com/dbg.h The main thing that caused defects was not handling default cases in logic. Things like not filling in the dangling else of an if-statement, not trying to return a value from a function no matter what, and not having a default: in a switch. By always trying to include at least a comment there, or an assert it forces you to think about the logic of the whole statement and prevents bugs galore. Combine that with unit tests and you've got a good solid start. I don't recommend it because it's an insane amount of work above just writing code. I think it'd be something I'd force on high speed students in a kind of "Boot Camp" situation, but not on anyone else. In fact, I think most programmers couldn't handle having a giant bag of evidence staring them in the face telling them they suck. Also I think most programmers wouldn't get the stats behind the analysis and just keep screwing it up. I play guitar when I have to think about stuff. I take one to work, and when I'm stuck I bust out my guitar, a pen, and a note pad. I then noodle and practice, and as ideas come into my head I write them down without filtering. Basically, you need a hobby. :-)
- agazso 16y ago
  In most recent C compilers you can turn on warnings for at least missing return values from functions and not having a default: in a switch. It is good practice to compile C/C++ programs with high warning level, and fix all the warnings before shipping.
wnoise 16y ago
When I'm writing code, a lot of the time the screen isn't going to change. This is not because I'm day dreaming, but because I'm thinking.

JoshCole 16y ago

Has anyone made a plugin or anything of that sort which abstracts away the grunt work of using this sort of system? It sounds like a great thing to try out were it not for the opportunity cost of trying it.

zedshaw 16y ago
Not that I know of. I imagine emacs or vim has good error-line parsing which you could steal/copy and get really good results. You could then extend it to work on any language you wanted. Biggest thing I couldn't get right was automatically connecting frequency of errors to particular lines or modules. I found that I made the same mistakes over and over, and I had to just spot them and jot down what not to do or think of a ninja way to avoid it. I also found that there was a strong correlation between syntax/test failures while writing code and how bad the code was. What would have helped is if I could calculate the stats per function, line, or file and then have some pattern analysis spit out common grammatical structures that I do which are correlated with errors. It'd be something like: function/while/if/return 24% chance of defect found in blah.c:func1 and blah.c:func2 Then it'd get closer to predicting defects and letting me go check the spots it recommends. But, never got around to it since it became really insane after a year already without that level of analysis.
windsurfer 16y ago
Hudson AKA Jenkins? I'm not sure if that's exactly what you want, but it worked wonders at the job I had.
torpor 16y ago
Coverity?

yters 16y ago

Have you tried delta debugging? http://en.wikipedia.org/wiki/Delta_Debugging http://en.wikipedia.org/wiki/Delta_Debugging

zedshaw 16y ago
Yes, I have. We used to call it "debugging", but I guess everything has to have a name before the cool kids will use it. However, this wasn't about find a bug but preventing defects by using evidence to retrain my behavior. The approach is to assume nothing, then use simple graphs and data to look for places where you can change how you do stuff to improve your code. It's not hard, just really tedious.
- yters 16y ago
  It's automated debugging.