4 ms·

How developers search for code: a case study

103 points by pramodbiligiri 11y ago

boyter 11y ago
Agree with pretty much everything inside this. The big difference I note however is that my experiences with running searchcode.com suggest that for public search engines that closer to 60% of searches appear to be "How to use a function". By contrast 60% of API calls appear to be looking for AWS keys, passwords and exploits. This is probably totally different to search working over internal codebases however. I must confess... I was originally looking from a vanity point of view and have mixed feelings to see that searchcode was mentioned in the references but not linked.
nautical 11y ago
25% of the total is documentation related .. Thats really interesting ... A 'smartly designed' documentation website can cater to 25% of total development queries ?
- noir_lord 11y ago
  Good documentation can do more than that I think, take http://knockoutjs.com/ http://knockoutjs.com/ there documentation is incredible, I went from not been able to use it at all to using it comfortably in a week or two and most of the questions I've had since have been answered by the docs. It's pretty much the standard I'll aim in my own documentation in the future, lean, precise and covers the main cases with useful examples.
- xyzzyz 11y ago
  The issue I think is that the documentation which is separated from code can easily go out of sync with the actual code, so it's worth to make writing documentation alongside the code as easy as possible, while trying not to obscure the code itself too much.
  - acveilleux 11y ago
    I'd argue that for public APIs (defined as API users don't site next to/are the same as the API devs) this concern with "getting out of date" is moot compared to the benefit of detailed docs. If anything, it's the API signatures that get out of sync with the user's needs. And as anyone who's ever tried to deprecate public APIs will know, it's a real pain, frustrating to users and devs alike.
  - shawn-butler 11y ago
    Whenever I see this comment I can't resist urge to make reference to Knuth: https://en.wikipedia.org/wiki/Literate_programming https://en.wikipedia.org/wiki/Literate_programming No language developer ever seems to take it seriously as a design challenge to solve though. You get tools like rustdoc as "good enough"
debacle 11y ago
Kind of funny that this is coming from Google considering how bad Google has become for code search. I wish there was a way to turn off the "I'm ignoring what you searched for and returning what I think you meant." feature.
- noir_lord 11y ago
  Indeed and the "I know you put it in quotes but I'm going to return none-literal matches as well". I guess the fundamental problem here is that optimising for one group sometimes comes at the expense of another and statistically we (meaning developers) just aren't a big enough group to matter.
- xyzzyz 11y ago
  Yeah, I personally don't like this behaviour either. However, paper specifically talks about internal code search tool, which is really pretty amazing, especially when it comes to finding and jumping to references.
- tetraodonpuffer 11y ago
  these days when doing code searches I often just add site:github.com to my search criteria, more often than not there is some project somewhere that is using the API / toolkit I am interested in, with this or site:stackoverflow / stackexchange I can usually find what I need fairly quickly
- simonw 11y ago
  Have you tried "Verbatim" mode? It's available from any Google results page as Search Tools -> All Results -> Verbatim.
fizixer 11y ago
I imagine typing a code comment in my text-editor before I start writing the code (e.g., calculate fibonacci sequence) and my editor (on a machine connected to the internet) populates with sample open-source code in my language (inferred from the file extension, or shebang, etc), that I can use as a boilerplate and make changes. The populated code is one search result, with the option to look at second most relevant result, third-most relevant and so on. And it shows the source url where I can go for details. I think something like this with rosetta-code snippets is very doable (a weekend project, assuming you're good with your editor's programmability).
- TheQwerty 11y ago
  It's unfortunately only for Visual Studio but Microsoft created an extension called "Developer Assistant" which does precisely that using code from MSDN, StackOverflow, GitHub, and also local sources. They boast "over 21 million code samples". http://blogs.msdn.com/b/onecode/p/devassistant.aspx http://blogs.msdn.com/b/onecode/p/devassistant.aspx Sadly they also appear to be keeping it closed source.

103 points by pramodbiligiri 11y ago

Agree with pretty much everything inside this. The big difference I note however is that my experiences with running searchcode.com suggest that for public search engines that closer to 60% of searches appear to be "How to use a function". By contrast 60% of API calls appear to be looking for AWS keys, passwords and exploits. This is probably totally different to search working over internal codebases however. I must confess... I was originally looking from a vanity point of view and have mixed feelings to see that searchcode was mentioned in the references but not linked.

nautical 11y ago

25% of the total is documentation related .. Thats really interesting ... A 'smartly designed' documentation website can cater to 25% of total development queries ?

noir_lord 11y ago
Good documentation can do more than that I think, take http://knockoutjs.com/ http://knockoutjs.com/ there documentation is incredible, I went from not been able to use it at all to using it comfortably in a week or two and most of the questions I've had since have been answered by the docs. It's pretty much the standard I'll aim in my own documentation in the future, lean, precise and covers the main cases with useful examples.
xyzzyz 11y ago
The issue I think is that the documentation which is separated from code can easily go out of sync with the actual code, so it's worth to make writing documentation alongside the code as easy as possible, while trying not to obscure the code itself too much.
- acveilleux 11y ago
  I'd argue that for public APIs (defined as API users don't site next to/are the same as the API devs) this concern with "getting out of date" is moot compared to the benefit of detailed docs. If anything, it's the API signatures that get out of sync with the user's needs. And as anyone who's ever tried to deprecate public APIs will know, it's a real pain, frustrating to users and devs alike.
- shawn-butler 11y ago
  Whenever I see this comment I can't resist urge to make reference to Knuth: https://en.wikipedia.org/wiki/Literate_programming https://en.wikipedia.org/wiki/Literate_programming No language developer ever seems to take it seriously as a design challenge to solve though. You get tools like rustdoc as "good enough"

debacle 11y ago

Kind of funny that this is coming from Google considering how bad Google has become for code search. I wish there was a way to turn off the "I'm ignoring what you searched for and returning what I think you meant." feature.

noir_lord 11y ago
Indeed and the "I know you put it in quotes but I'm going to return none-literal matches as well". I guess the fundamental problem here is that optimising for one group sometimes comes at the expense of another and statistically we (meaning developers) just aren't a big enough group to matter.
xyzzyz 11y ago
Yeah, I personally don't like this behaviour either. However, paper specifically talks about internal code search tool, which is really pretty amazing, especially when it comes to finding and jumping to references.
tetraodonpuffer 11y ago
these days when doing code searches I often just add site:github.com to my search criteria, more often than not there is some project somewhere that is using the API / toolkit I am interested in, with this or site:stackoverflow / stackexchange I can usually find what I need fairly quickly
simonw 11y ago
Have you tried "Verbatim" mode? It's available from any Google results page as Search Tools -> All Results -> Verbatim.

fizixer 11y ago

I imagine typing a code comment in my text-editor before I start writing the code (e.g., calculate fibonacci sequence) and my editor (on a machine connected to the internet) populates with sample open-source code in my language (inferred from the file extension, or shebang, etc), that I can use as a boilerplate and make changes. The populated code is one search result, with the option to look at second most relevant result, third-most relevant and so on. And it shows the source url where I can go for details. I think something like this with rosetta-code snippets is very doable (a weekend project, assuming you're good with your editor's programmability).

TheQwerty 11y ago
It's unfortunately only for Visual Studio but Microsoft created an extension called "Developer Assistant" which does precisely that using code from MSDN, StackOverflow, GitHub, and also local sources. They boast "over 21 million code samples". http://blogs.msdn.com/b/onecode/p/devassistant.aspx http://blogs.msdn.com/b/onecode/p/devassistant.aspx Sadly they also appear to be keeping it closed source.