5 ms·

If you read/scan 10 lines a second, you still need over 24 hours non-stop to read a 1M+ code base. I doubt a random file ordering is helpful! Especially if you

If you read/scan 10 lines a second, you still need over 24 hours non-stop to read a 1M+ code base. I doubt a random file ordering is helpful! Especially if you lose code navigation features like "go to definition".

sillysaurusx 6y ago
You'd be surprised. Suppose I want to "go to definition" for a class named Saver. /^class Saver\> 19 times out of 20, this works. It's also instant; my vim will likely get me there faster than your IDE's go to definition functionality. (Looking at you, pycharm!) Here's my flow. >>> import tensorflow as tf >>> tf.train.Saver <class 'tensorflow.python.training.saver.Saver'> >>> from tensorflow.python.training import saver >>> saver <module 'tensorflow.python.training.saver' from '/usr/local/lib/python3.7/site-packages/tensorflow_core/python/training/saver.py'> Then I open /usr/local/lib/python3.7/site-packages/tensorflow_core/python/training/saver.py. Suppose I want to know: Where are all the places that Saver is used in all of tensorflow? time find . -type f -name '*.py' | xargs merge | ft py Vim: Reading from stdin... real 0m0.930s user 0m0.243s sys 0m0.414s /\<Saver( :%v//d Boop: https://i.imgur.com/EJ5bZSW.png https://i.imgur.com/EJ5bZSW.png Literally every usage of Saver in all of Tensorflow. So let's say you're interested in a specific line. This one, for example: # being added to the GLOBAL_VARIABLES collection, so that Saver() Boop: https://i.imgur.com/YSAzhCJ.png https://i.imgur.com/YSAzhCJ.png I did that by highlighting "being added to the GLOBAL_VARIABLES", ctrl-c, then pressing "u" to undo the :%v//d, then / followed by ctrl-v. That might sound hard, but with muscle memory I don't even think about it -- it's like explaining how you open a can of food. Do you really think about where you place your fingers, or the pressure of your nail on the flap of the can? No, you just open it up. Same thing here; it's automatic. Way faster than IDEs, and I get just as much (or more) info. I'd love to use pycharm, but the slowness keeps pushing me back to this technique.
- fraculus 6y ago
  I agree that things like "Go To Definition" can be pretty bad (especially when a codebase has code that is auto-generated, but you haven't figured out how yet). But I'm curious, what benefits do you see in your approach over just doing a grep in the folder (or e.g. Ctrl+Shift+F "Find in Files" in Pycharm)?
  - disgruntledphd2 6y ago
    grep type approaches are really good when part of your application is generated as SQL strings to make tables (which is a pathology sadly common in most data science codebases).
- mejutoco 6y ago
  Nothing wrong with this (big fan of vim myself). I can see the appeal and simplicity. I just want to add that in Intellij you can do Ctrl + Shift + R "Saver" and it will search in all files (as dumb text matches, not usages), plus optional checks on extensions, etc. This is pretty fast too, and quite convenient, since it has a preview for each file. Not saying it is better, but it is an alternative.
  - sillysaurusx 6y ago
    I'm happy you mentioned that, because this highlights a very important difference: Your IDE would show you all "Saver" files in some checkout of Tensorflow, right? But you usually don't want to see the latest version of Tensorflow. You want to see the current version that's installed, which is somewhere under /usr/lib. I haven't found any IDE that can easily and effortlessly do that.
- minusf 6y ago
  i do something similar but with tags and without the merging. vim's tag navigation is very powerful. split a window with the target tag and see side by side the class and its descendant for example. i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions. it's also possible to have the system libraries show up in the tag database, it's just a matter of telling uctags which path's, files to include/exclude or alternatively use another tag file for that, vim can use multiple tag files. the real pain point is keeping the tag file update. gutentags makes this a bit more easy for me.
  - sillysaurusx 6y ago
    i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions. it's also possible to have the system libraries show up in the tag database, it's just a matter of telling uctags which path's, files to include/exclude or alternatively use another tag file for that, vim can use multiple tag files. Oh? Yours is the first system I've found that has this very important feature -- the whole reason I do it my way is because I can dill down into the actual installed libraries, whereas IDEs almost always fail. (It's hit or miss. Yeah, theoretically you can configure the IDE properly if you spend your life becoming an IDE master and have a 96-core workstation, but it never seems to "just work.") If you ever do a writeup of how precisely you've set up your environment, do ping me! I'm https://twitter.com/theshawwn https://twitter.com/theshawwn. I'd be very interested and would happily retweet it.
    - 00N8 6y ago
      I use pycharm and as long as I set the virtualenv I'm using as the Project Interpreter in Preferences it lets me "drill down" to the library code as well
- amw-zero 6y ago
  How do you handle finding occurrences of a specific variable with the same name as another one? Simple text search has the power equivalence of a dull butter knife.
disgruntledphd2 6y ago
With the language server protocol, this works reasonably well in Python code bases for me in Emacs.
- jidiculous 6y ago
  Yep, I didn't see why the author decided to snipe at Emacs and Vim at the end of the piece. Maybe 4 or 5 years ago, that criticism of Emacs would have been valid, but packages like flycheck and lsp-mode have brought Emacs at least to parity with many IDEs. In terms of Git alone, Magit has catapulted Emacs past any other editor, IDE, or Git GUI client.

sillysaurusx 6y ago

You'd be surprised. Suppose I want to "go to definition" for a class named Saver. /^class Saver\> 19 times out of 20, this works. It's also instant; my vim will likely get me there faster than your IDE's go to definition functionality. (Looking at you, pycharm!) Here's my flow. >>> import tensorflow as tf >>> tf.train.Saver <class 'tensorflow.python.training.saver.Saver'> >>> from tensorflow.python.training import saver >>> saver <module 'tensorflow.python.training.saver' from '/usr/local/lib/python3.7/site-packages/tensorflow_core/python/training/saver.py'> Then I open /usr/local/lib/python3.7/site-packages/tensorflow_core/python/training/saver.py. Suppose I want to know: Where are all the places that Saver is used in all of tensorflow? time find . -type f -name '*.py' | xargs merge | ft py Vim: Reading from stdin... real 0m0.930s user 0m0.243s sys 0m0.414s /\<Saver( :%v//d Boop: https://i.imgur.com/EJ5bZSW.png https://i.imgur.com/EJ5bZSW.png Literally every usage of Saver in all of Tensorflow. So let's say you're interested in a specific line. This one, for example: # being added to the GLOBAL_VARIABLES collection, so that Saver() Boop: https://i.imgur.com/YSAzhCJ.png https://i.imgur.com/YSAzhCJ.png I did that by highlighting "being added to the GLOBAL_VARIABLES", ctrl-c, then pressing "u" to undo the :%v//d, then / followed by ctrl-v. That might sound hard, but with muscle memory I don't even think about it -- it's like explaining how you open a can of food. Do you really think about where you place your fingers, or the pressure of your nail on the flap of the can? No, you just open it up. Same thing here; it's automatic. Way faster than IDEs, and I get just as much (or more) info. I'd love to use pycharm, but the slowness keeps pushing me back to this technique.

fraculus 6y ago
I agree that things like "Go To Definition" can be pretty bad (especially when a codebase has code that is auto-generated, but you haven't figured out how yet). But I'm curious, what benefits do you see in your approach over just doing a grep in the folder (or e.g. Ctrl+Shift+F "Find in Files" in Pycharm)?
- disgruntledphd2 6y ago
  grep type approaches are really good when part of your application is generated as SQL strings to make tables (which is a pathology sadly common in most data science codebases).
mejutoco 6y ago
Nothing wrong with this (big fan of vim myself). I can see the appeal and simplicity. I just want to add that in Intellij you can do Ctrl + Shift + R "Saver" and it will search in all files (as dumb text matches, not usages), plus optional checks on extensions, etc. This is pretty fast too, and quite convenient, since it has a preview for each file. Not saying it is better, but it is an alternative.
- sillysaurusx 6y ago
  I'm happy you mentioned that, because this highlights a very important difference: Your IDE would show you all "Saver" files in some checkout of Tensorflow, right? But you usually don't want to see the latest version of Tensorflow. You want to see the current version that's installed, which is somewhere under /usr/lib. I haven't found any IDE that can easily and effortlessly do that.
minusf 6y ago
i do something similar but with tags and without the merging. vim's tag navigation is very powerful. split a window with the target tag and see side by side the class and its descendant for example. i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions. it's also possible to have the system libraries show up in the tag database, it's just a matter of telling uctags which path's, files to include/exclude or alternatively use another tag file for that, vim can use multiple tag files. the real pain point is keeping the tag file update. gutentags makes this a bit more easy for me.
- sillysaurusx 6y ago
  i use `venv`s for every project inside the project folder so uctags also generates tags for all libraries installed and i can "drill all the way up" to classes and definitions. it's also possible to have the system libraries show up in the tag database, it's just a matter of telling uctags which path's, files to include/exclude or alternatively use another tag file for that, vim can use multiple tag files. Oh? Yours is the first system I've found that has this very important feature -- the whole reason I do it my way is because I can dill down into the actual installed libraries, whereas IDEs almost always fail. (It's hit or miss. Yeah, theoretically you can configure the IDE properly if you spend your life becoming an IDE master and have a 96-core workstation, but it never seems to "just work.") If you ever do a writeup of how precisely you've set up your environment, do ping me! I'm https://twitter.com/theshawwn https://twitter.com/theshawwn. I'd be very interested and would happily retweet it.
  - 00N8 6y ago
    I use pycharm and as long as I set the virtualenv I'm using as the Project Interpreter in Preferences it lets me "drill down" to the library code as well
amw-zero 6y ago
How do you handle finding occurrences of a specific variable with the same name as another one? Simple text search has the power equivalence of a dull butter knife.

disgruntledphd2 6y ago

With the language server protocol, this works reasonably well in Python code bases for me in Emacs.

jidiculous 6y ago
Yep, I didn't see why the author decided to snipe at Emacs and Vim at the end of the piece. Maybe 4 or 5 years ago, that criticism of Emacs would have been valid, but packages like flycheck and lsp-mode have brought Emacs at least to parity with many IDEs. In terms of Git alone, Magit has catapulted Emacs past any other editor, IDE, or Git GUI client.