14 ms·

Grepping for symbols like function names and class names feels so anemic compared to using a tool that has a syntactic understanding of the code. Just "go to de

• wglb 2y ago
  A bit on the other side of the argument, I use grep plus find plus some shell work to do source code analysis for security reviews. grep doesn't really understand the syntax of languages, and that is mostly OK. I've used this technique on auditing many code bases including the C family, perl, Visual Basic, C# and SQL. With this sort of tool, I don't need to look for language-particular parsers--so long as the source is in a text file, this works well.
• umvi 2y ago
  Interface-heavy languages break IDEs. In .NET at least, "go to definition" jumps you to the interface definition which you probably aren't interested in (vs. the specific implementation you are trying to dig into). Also with .NET specifically XAML breaks IDE traceability as well.
• ilrwbwrkhv 2y ago
  I tried a good IDE recently: Jetbrains IntelliJ and Webstorm. Considered the topdog of IDEs. Was working on a typescript project which uses npm link to symlink another local project into the node_modules of current project. The great IDEs IntelliJ and Webstorm stopped autosuggesting completions from the symlinked project. Open up Sublime Text again. Worked perfectly. That is why Jetbrains and their behemoth IDEs are utter shite. Write your code to have symmetry and make it easy to grep.
  • 71bw 2y ago
    >I tried a good IDE recently: Jetbrains IntelliJ Having dealt with IntelliJ for 3 years due to education stuff - I laughed out here. Even VS is better than ideaj.
• mihaaly 2y ago
  "A good IDE" I am also waiting for world peace! ; )
• groby_b 2y ago
  Working on a 32MLOC project, text search is still the quickest way to find a hook that gets you to the deeper investigation. From there, finding definitions/usage definitely matters. You can maybe skip the greppability if the code base is of a size that you can hold the rough shape and names in your head, but a "get a list of things that sound like they might be related to my problem" operation is still extremely helpful. And it's also worth keeping in mind that greppability matters to onboarding. Does that mean it should be an overriding design concern? No. But it does mean that if it's cheap to build greppable, you probably should, because it's a net positive.
• dangsux 2y ago
  [dead]
• jakub_g 2y ago
  Your observation does not help with the majority of the points in the article. How do you find all usages of a parameter value literal?
  • troupo 2y ago
    This is what the article starts with: "Even in projects exclusively written by myself, I have to search a lot: function names, error messages, class names, that kind of thing." All of that is trivial to search for with a tool that understands the language.
    • renewiltord 2y ago
      I actually don't think there's a tool that handles usages when using PHP varvars or when using example number one there which is parametrically choosing a table name. When you string interpolate to build the name you lose searchability.
      • troupo 2y ago
        Yes, full-text search is a great fallback when everything else fails. But in the use cases listed at the beginning of the article it's usually not needed if you have proper tools
    • nosianu 2y ago
      > All of that is trivial to search for with a tool that understands the language. Isn't string search, or grepping for patterns, even more trivial? So what is your argument? You found an alternative method, good, but how is it any better? In my own case, I wrote a library that we used in many projects, and I often wanted to know where and how functions from my lib were used in those projects. For example, to be able to tell how much of an effort it would be for the users to refactor when I changed something. However, your method of choice at least with my IDE (Webstorm) only worked locally within the project. Only string search would let me reliably and easily search all projects. I actually experimented creating a "meta" project of all projects, but while it worked that lead to too many problems, and the main method to find anything still was string search (CTRL-SHIFT-F Find dialog in IDEA IDEs is string search and it's a wonderful dialog in that IDE family). I also had to open that meta project. Instead, I created a gitignored folder with symlinks to the sources of all the other projects and created a search scope for that folder, in which the search dialog let me string-search all projects' sources at once right from within the library project and still being able to use the excellent Find dialog. In addition, I found that sometimes the IDE would not find a usage even within the project. I only noticed because I used both methods, and string search showed me one or two places more than the method that relied on the underlying code-parsing. Unfortunately IDEs have bugs, and the method you suggests relies on much more work of the IDE in parsing and indexing compared to the much more mundane string or string pattern search.
• brain5ide 2y ago
  I think the first sentence of the author counters your comment. What you described works best in a familiar codebase where the organizing principles have been maintained well and are familiar to the reader and the tools are just the extension of those organizing principles. Even then a deviation from those rules might produce gaps in understanding of what the codebase does. And grep cuts right through that in a pretty universal way. What the post describes are just ways to not work against grep to optimize for something ephemeral.
  • ricardo81 2y ago
    Agree. Not just because it's unfamiliar code, you can also get a feel for how the program/programmer(s) structured the whole thing.
• heisenbit 2y ago
  A good IDE can be so much better iff it understands the code. However this requires the IDE to be able to understand the project structure, dependencies etc. which can be considerable effort. In a codebase with many projects employing several different languages it becomes hard to get and maintain the IDE understands everything state.
  • amichal 2y ago
    And an IDE would also fail to find references for most of the cases described in the article: name composition/manipulation, naming consistency across language barriers, and flat namespaces in serialization. And file/path folder naming seems to be irrelevant to the smart IDE argument. "Naming things is hard"
  • carlmr 2y ago
    And especially in large monorepos anything that understands the code can become quite sluggish. While ripgrep remains fast. A kind of in-between I've found for some search and replace action is comby (https://comby.dev/ https://comby.dev/). Having a matching braces feature is a godsend for doing some kind of replacements properly.
• zarzavat 2y ago
  Go to definition and find usages only work one symbol at a time. I use both, but I still use global find/replace for groups of symbols sharing the same concept. For example if I want to rename all “Dog” (DogModel, DogView, DogController) symbols to “Wolf”, find/replace is much better at that because it will tell me about symbols I had forgotten about.
  • gugagore 2y ago
    I am familiar with the situation you describe, and it's a good point. However, it does suggest that there is an opportunity for factoring "Dog" out in the code, at least by name spacing (e.g. Dog.Model).
    • f1shy 2y ago
      That really depends on the context, and specific situation.
    • zarzavat 2y ago
      That gets to the core of the issue doesn’t it? There are two cultures: Do you prefer to refactor DogView into Dog.View, or do you prefer to refactor Dog.View into DogView. Personally I value uniqueness/canonicalness over conciseness. I would rather have DogView because then there is one name for the symbol regardless of where I am in the codebase. If the same symbol is used with differently qualified names it is confusing - I want the least qualified name to be more descriptive than “View”. The other culture is to lean heavily on namespaces and to not worry about uniqueness. In this case you have View and Dog.View that may be used interchangeably in different files. This is the dominant culture in Java and C#.
      • kccqzy 2y ago
        The second culture that you describe happens also to be how OCaml structures things in modules. It's quite a turnoff for me.
  • f1shy 2y ago
    For that use case I think you can use treesitter[1] you can find Dog.* but only if it is a variable name, for example. Avoiding replacement inside of say literals. [1] https://www.youtube.com/watch?v=MZPR_SC9LzE https://www.youtube.com/watch?v=MZPR_SC9LzE
• underdeserver 2y ago
  Unfortunately in larger codebases or dynamic languages these tools are just not good enough today. At least not those I and my employers have tried. They're either incomplete (you don't get ALL references or you get false references) or way too slow (>10 seconds when rg takes 1-2). Recommendations are most welcome.
  • jimmaswell 2y ago
    Only thing I can recommend is using C# (obviously not always possible). Never had an issue with these functions in Visual Studio proper no matter how big the project.
• aa-jv 2y ago
  On the flipside, IDE's can turn you into lazy, inefficient programmers by doing all the hand-holding for you. If your feelings are anemic when tasked with doing a grep, its because you have lost a very valuable skill by delegating it to a computer. There are some things the IDE is never going to be able to find - lest it becomes the development environment - so keeping your grep fu sharpened is wise beyond the decades. (Disclaimer: 40 years of software development, and vim+cscope+grep/silversearcher are all I really need, next to my compiler..)
  • HdS84 2y ago
    Huh? I have an old hand-powered drill from my Grandpa in my workshop. I used it once for fun. For all other tasks I use a powered drill. Same for IDEs. They help your refactor and reason about code - both properties I value. Sure, I could print it and use a textmarker, but I'm not Grandpa
    • trashtester 2y ago
      Knowing the bash ecosystem translates better to how you use the knife in the kitchen. Sure you can replace most uses of a knife with power tools, but there is a reason why most top chefs still rely on that knife for most of those tasks. A hand powered drill is more like a hand powered meatgrinder. It has the same limitation as the powered versions, and is simply a more primitive version.
  • winwang 2y ago
    I count the IDE and stuff like LSP as natural extensions of the compiler. For sure I grep (or equivalent) for stuff, but I highly prefer statically typed languages/ecosystems. At the end of the day, I'm here to solve problems, and there's no end to them -- might as well get a head start.
  • high_na_euv 2y ago
    Leveraging technology is good thing
  • throwaway2037 2y ago
• IshKebab 2y ago
  Definitely true when you can use static typing. Unfortunately sometimes you can't, and sometimes you can but people can't be arsed, so this is still a consideration.
• a_e_k 2y ago
  I've come to really like language servers for big personal and work projects where I already have my tools configured and tuned for efficiently working with it. But being able to grep is really nice when trying to figure out something out about a source tree that I don't yet have set up to compile, nor am I a developer of. I.e., I've downloaded the source for a tool I've been using pre-built binaries of and am now trying to trace why I might be getting a particular error.
• leni536 2y ago
  I can't use an IDE on my entire git history, but git can grep.
• citrin_ru 2y ago
  Not everything you need to look for is a language identifier. I often grep for configuration option names in the code to see what the option actually does - sometimes it is easy to grep, sometimes there are too many matches, sometimes they cannot be found because option name composed in the code from separate unrepeatable (because of too many matches) parts. It's not hard to make config options greppable but some coders just don't care about this property.
• k__ 2y ago
  Honestly, in my 18 years of software development, I haven't "greped" code once. I only use grep to filter the output of CLI tools. For code, I use my IDE or repository features.
  • yCombLinks 2y ago
    Do you use the find feature in your IDE? IE not find by reference, just text matching? That's the same as greppability.
• laserbeam 2y ago
  Scenarios where an IDE with full syntactic understanding is better: - It's your day to day project and you expect to be working in it for a long time. Scenarios where grepping is more useful: - Your language has #ifdef or equivalent syntax which does conditional compilation making syntactic tools incomplete. - You just opened the project for the first time. - It's in a language you don't daily drive (you write backend but have to delve in frontend code, it's a 3rd party library, it's configuration files, random json/xml files or data) - You're editing or searching through documentation. - You haven't even downloaded the project and are checking things out in github (or some similar site for your project). - You're providing remote assistance to someone and you are not at your main development machine. - You're remoting via SSH and have access to code there (say it's a python server). Yes, an IDE will save you time daily driving. But there's no reason to sabotage all the other usecases.
  • codedokode 2y ago
    "Go to definition" often doesn't work in dynamic languages like Python without type hints; it might not work when the code is dynamically generated.
    • neves 2y ago
      It always work in VSCode if your environment is correctly configured.
      • codedokode 2y ago
        No; Python is a dynamic language and when you see a method call like x.so_something() you might not know what is the type of x and what method does do_something refers to.
  • beeboobaa3 2y ago
    > - Your language has #ifdef or equivalent syntax which does conditional compilation making syntactic tools incomplete. You need a better IDE. > - You just opened the project for the first time. Go grab a coffee > - It's in a language you don't daily drive Jetbrains all products pack, baby. > - You haven't even downloaded the project and are checking things out in github (or some similar site for your project). On GitHub, press `.` to open it in a web-based vscode. Download it & open it in your IDE while you are doing this. > - You're remoting via SSH and have access to code there (say it's a python server). Don't do this. Check the git hash that was deployed and checkout the code locally.
• PhilipRoman 2y ago
  IDEs are cool and all, but there is no way I'm gonna let VSCode index my 80GB yocto tmp directory. Ctags can crunch the whole thing in a few minutes, and so can grep. Plus there are cases where grep is really what you need, for example after updating a particular command line tool whose output changed, I was able to find all scripts which grepped the output of the tool in a way that was broken.
• phyrex 2y ago
  This breaks down at scale and across languages. All the FAANGs make heavy use of the equivalent of grepping in their code base
• gregjor 2y ago
  I abandoned VSCode and went back to vim + ctags + ripgrep after a year with the most popular IDE. I miss some features but it didn’t give me a 10x or even 1.5x improvement in my own work along any dimension. I attribute that mostly to my several decades of experience with vi(m) and command line tools, not to anything inherently bad about VSCode. What counts as “better” tools has a lot of subjectivity and circumstances implied. No one set of tools works for everyone. I very often have to work over ssh on servers that don’t allow installing anything, much less Node and npm for VSCode, so I invest my time in the tools that always work everywhere, for the work I do. The main project I’ve worked on for the last few years has a little less than 500,000 lines of code. VSCode’s LSP takes a few seconds fairly often to maintain the LSP indexes. Running ctags over the same code takes about a second and I can control when that happens. vim has no delays at all, and ripgrep can search all of the files in a second or two.
  • wrasee 2y ago
    Did you consider Neovim? You get the benefit of vim while also being able to mix in as much LSP tooling as you like. The tradeoff is that it takes some time to set up, although that is getting easier. That won’t make LSP go any faster though. There’s still something interesting in the fact that a ripgrep of every line in the codebase can still be faster than a dedicated tool.
    • gen220 2y ago
      You can use a tool like ALE (the Asynchronous Linting Engine) to run LSPs in normal-Vim; I've been doing it for years and have no complaints! It's rapid.
    • gregjor 2y ago
      Considered it and have tried repeatedly to get it to work with mixed success. As you wrote, it takes "some time" to set up. In my case it would only offer marginal improvements over plain vim, since I'm not that interested in the LSP integration (and vim has that too, through a plugin). In the environments I often work in I can't install anything or run processes like node. I ssh into a server and have to use whatever came with the Linux distro, which means sticking with the tools I will find everywhere. I can't copy the code from the server either. If I get lucky they used version control. I know not everyone works with those constraints. I specialize in working on abandoned and legacy code.
• sauercrowd 2y ago
  strongly disagree here. This works if - your IDE/language server is performant - all the tools are fully set up - you know how to query the specific semantic entity you're looking for (remembering shortcuts) - you are only interested in a single specific semantic entity - mixing entities is rarely supported I dont map out projects in terms of semantics, I map out projects in files and code - That makes querying intuitive and I can easily compose queries that match the specificity of what I care about (e.g. I might want to find a `Server` but I want to show both classes, interfaces and abstract classes). For the specific toolchain I'm using - typescript - the symbol search is also unusable once it hits a certain project size, it's just way too slow for it to be part of my core workflow
• db48x 2y ago
  True, but IDEs are fragile tools. Sometimes you want to fall back to simpler tools that will always work, and grep is not fragile.
  • cxr 2y ago
    The basis if this article (and its forebear "Too DRY - The Grep Test"[1]) is that grep is fragile. It's just fragile in a way that's different from the way that IDEs are fragile. 1. <http://jamie-wong.com/2013/07/12/grep-test/ http://jamie-wong.com/2013/07/12/grep-test/>
• kragen 2y ago
  posts like this sound like the author routinely solves harder problems than you are, because the solutions you suggest don't work in the cases the post is about. we've had 'go to definition' since 01978 and 'find usages' since 01980, and you should definitely use them for the cases where they work
  • mjr00 2y ago
    From the article, - dynamically built identifiers is 100% correct, never do this. Breaks both text search and symbol search, results in complete garbage code. I had to deal with bugs in early versions of docker-compose because of this. - same name for things across the stack? Shouldn't matter, just use find usages on `getAddressById`. Also easy way to bait yourself because database fields aren't 1:1 with front-end fields in anything but the simplest of CRUD webshit. - translation example: the fundamental problem is using strings as keys when they should be symbols. Flat vs nested is irrelevant here because you should be using neither. - react component example: As I mentioned in another comment, trivially managed with Find Usages. Nothing in here strikes me as "routinely solves harder problems," it's just standard web dev.
    • kragen 2y ago
      yes, i agree that standard web dev is full of these problems, which can't be solved with go-to-definition and find-usages. it's a mess. i wasn't claiming that these messy, hard problems where grep is more helpful than etags are exotic; they are in fact very common. they are harder than the problems lucumo is evidently accustomed to dealing with because they don't have correct, complete solutions, so we have to make do with heuristics advice to the effect of 'you should not make a mess' is obviously correct but also, in many situations, unhelpful. sometimes i'm not smart enough to figure out how to solve a problem without making a mess, and sometimes i inherit other people's messes. in those situations that advice decays into 'you should not try to solve hard problems'
      • lucumo 2y ago
        > they are harder than the problems lucumo is evidently accustomed to dealing with because they don't have correct, complete solutions, so we have to make do with heuristics Funny. But since you asked. The hardest problems I've solved haven't been technical problems for years. Not that I stopped solving technical problems, or that I started solving only the easier problems. I just learned to solve people problems more. People problems are much harder than technical problems. The author showed a simple people problem: someone who needs to know about better tooling. If we were working together, showing them some tricks wouldn't take much time and would improve their productivity. An example of a harder problem is when someone tries to play aggressive little word games with you. For example, trying to put you down by loudly making assumptions about your career and skills. One way to deal with that is to just laugh it off. Maybe even make a self-deprecating joke. And then continuing as if nothing happened. But that assumes you want or have to continue working productively with them. If you don't, it can be quite enjoyable to just laugh in their face. After all, it's never the sharpest tool in the shed, or the brightest light that does that. In fact, it's usually the least useful person around, who is just trying to hide that fact. Of course, once you realize that, it becomes hard to laugh, because it's no longer funny. Just sad and pitiful.
• hyperpape 2y ago
  I can run rg over my project faster than I can do anything in my IDE. Both tools have their places.
• mjr00 2y ago
  > Honestly, posts like this sound like the author needs to invest some time in learning about better tools for his language. A good IDE alone will save you so much time. Completely agreed. The React component example in the article is trivial solvable with any modern IDE; right click on class name, "Find Usages" (or use the appropriate hotkey, of course). Trying to grep for a class name when you could just do that is insane. I mainly see this from juniors who don't know any better, but as seen in this thread and the article, there are also experienced engineers who are stubborn and refuse to use tools made after 1990 for some reason.
  • gregjor 2y ago
    > experienced engineers who are stubborn and refuse to use tools made after 1990 for some reason. Before calling people stubborn or assuming they got left behind out of ignorance, consider your assumptions. 40+ years experience, senior in both experience and age at this point. Long-term vim + command line tools user. Do you have any evidence that shows "A good IDE alone will save you so much time?" Have you seen studies comparing productivity or code quality or any metric written by people using IDEs vs those using a plain editor with grep? By "so much faster" what do you mean exactly? I have decades of experience with vim + ctags + grep (rg these days, because I don't want to get called a stubborn stick in the mud). I can find and change things in large codebases pretty fast. I used VSCode for a year on the same codebases and I didn't feel "so much faster," and I committed to it and watched numerous how-to videos and learned the tool well enough to train other programmers on it. No 10x improvement, not even 1.5x. For most tasks I would call it close to the same in terms of time taken to write code. After getting burned a couple times with "Replace symbol" in VSCode I stopped trusting it. After noticing the LSP failed to find some references I trusted it less. I know grep/ack/rg/ctags aren't perfect, but I also know their weaknesses and how to work with them to get them to do what I want. After a year I went back to vim + ctags + rg. We might have more productive (and friendly) interactions as programmers if we remembered that not everyone works the same way, or on the same kind of code and projects. What we call "best practices" or "modern tools" largely come down to familiarity, received wisdom, opinion, and fashion -- almost never from rigorous metrics and testing. You like your IDE? Great! I like my tools too. Would either of us get "so much faster" using a different set of tools? Probably not. Trying to find the silver bullet that reduces accidental complexity in software development presents an ongoing challenge, but history shows that editors and IDEs don't do much because if they did programmers today would outperform old guys like me by 10x in a measurable way. At the last full-time job I had, at an educational software company with 30+ programmers, everyone used Eclipse. My first day I got a new desktop with two big monitors, Eclipse installed, ready to go. I installed vim and the CLI subversion client and some other stuff and worked from the command line, as I usually do. I left one of the monitors off, I don't need that much screen space, and I don't have Twitter and Facebook and other junk running on a second monitor all day like most of the other people did. I got made fun of, old man using old tools. Then once a week, like clockwork, Eclipse would auto-install some updates and everyone came to a halt trying to resolve plugin version conflicts, getting the team in sync. Hours and hours wasted regularly just getting the IDE to work. That didn't affect me, I never opened Eclipse. Watching the other programmers it seemed really slow. So just maybe Eclipse could jump to a definition faster than vim + ctags (I doubt it), but amortized over a month Eclipse all by itself wasted more time than anyone possibly saved with the more powerful tool. Anecdote, I know, but I've seen this play out in similar ways at more than one shop. Just last year a new hire at a place I freelance for spent days trying to get Jetbrains PHPStorm working on a shared remote dev server. Like VSCode it runs a heavy process on the server (including the LSP). Unlike VSCode, PHPStorm can actually kill the whole server, wasting everyone's time and maybe losing work. I have never seen vim or grep bring a whole server down. I could add up how much "faster" PHPStorm might turn out compared to vim, but it will have to recoup the days lost trying to get it to work at all first.
• EasyMark 2y ago
  It seems like the law of diminishing returns; while I'm sure in a few cases this characteristic of a code writing style is extremely useful, it cuts into other things such as readability and conciseness. Fewer lines can mean fewer bugs, within reason, if you aren't in lisp and are using more than 3 parentheses, you might want to split it up because the compiler/JIT/interpreter is going to anyway.
• brooke2k 2y ago
  with all due respect, it sounds like you have the privilege of working in some relatively tidy codebases (and I'm jealous!) with a legacy codebase, or a fork of a dependency that had to be patched which uses an incompatible buildsystem, or any C/C++/obj-c/etc that heavily uses the preprocessor or nonstandard build practices, or codebases that mix lots of different languages over awkward FFI boundaries and so on and so forth -- there are so many situations where sometimes an IDE just can't get you 100% of the way there and you have to revert to grepping to do any real work that being said, I don't fully support the idea of handcuffing your code in the name of greppability, but I think dismissing it as a metric under the premise that IDEs make grepping "obsolete" is a little bit hasty
  • lucumo 2y ago
    > with all due respect, it sounds like you have the privilege of working in some relatively tidy codebases (and I'm jealous!) I wish, but no. I've found people will make a mess of everything. Which is why I don't trust solutions that rely on humans having more discipline, like what this article advocates. In any situation where grep is your last saviour, you cannot rely on the greppability of the code. You'll have to check and double check everything, and still accept the risk of errors.
• jmmv 2y ago
  Sure, if you have the luxury of having a functional IDE for all of your code. You can't imagine how much faster I was than everybody else at answering questions about a large codebase just because I knew how to use ripgrep (on Windows). "Knowing how to grep" is a superpower.
• kelnos 2y ago
  Even with IDEs, I find that I grep through source trees fairly often. Sometimes it's because I don't completely trust the IDE to find everything I'm interested in (justifiably; sometimes it doesn't). Sometimes it's because I'm not looking to dive into the code and do serious work on it; I'm just doing a quick drive-by check/lookup for something. Sometimes it's because I'm ssh'd into another machine and I don't have the ability to easily open the sources in an IDE.