7 ms·

If you need to do something once, use a GUI (Excel). If you need to do something ten times, use hotkeys and shortcuts. If you need to do something a hundred t

by Zhyl 5y ago

If you need to do something once, use a GUI (Excel). If you need to do something ten times, use hotkeys and shortcuts. If you need to do something a hundred times, write a script (R). I usually use the command line as the example for why writing code and scripts are better than the more intuitive and lower-learning-curve GUIs. If I want to move a file from one folder to another then I just drag it across. Easy. If I want to move a thousand files from one folder to another, I will benefit from learning `CTRL-A` or shift-clicking (slightly more obscure than the 'intuitive' drag each file across individually or drag a large box around them all to select them). If I want to move a thousand files beginning with 'UTR-77' and ending with '.csv' then I would benefit from learning `mv UTR-77*.csv $folder`, but I could still do it manually if I didn't know that was an option. If I want to move a thousand files beginning with 'URT-77' to another folder at a moment's notice or at Thursday 1am, then the only options I really have are scripting. I almost feel like before people learn the 'basic' stuff as outlined at the beginning of the article, they should be shown some 'magic' that is only really possible with scripting so that it's clear from the outset why you wouldn't 'just use excel'.

DantesKite 5y ago
This is a great comment and the line of reasoning for work at different scales makes so much sense.
danuker 5y ago
> I want to move a thousand files beginning with 'UTR-77' and ending with '.csv' 1. Use a file manager with stable sorting [1] (I use Thunar which does this, but I suspect lots of file managers keep the sort order stable). 2. Sort by type. 3. Sort by name. 4. Select the first file named UTR-77. 5. Scroll to the last file, and Shift+Click it. 6. Cut then paste to your desired directory. [1] - https://en.wikipedia.org/wiki/Sorting_algorithm#Stability https://en.wikipedia.org/wiki/Sorting_algorithm#Stability
- bicolao 5y ago
  > 5. Scroll to the last file, and Shift+Click it. With thousands of files, this won't be particularly fast or easy.
  - Stratoscope 5y ago
    If you know how to use a scrollbar, it is fast and easy. Grab the thumb and drag it down. If you overshoot or have trouble landing on the last file, use the page up/down areas of the scrollbar or its up/down arrow to fine tune the scroll position. Should only take a few seconds. Another way is to hold down the Ctrl key and use the PgDn/PgUp keys to scroll. If you hold down Ctrl+PgDn you can whip through thousands of files quickly. Because you have the Ctrl key down, it won't affect your selection of the first file. If all you know is two finger touchpad scrolling, then it will be very tedious. Unfortunately, from observing a number of people - and apps that hide the scrollbars - it seems to me that two finger scrolling has largely taken over from scrollbar scrolling or the scroll keys.
    - dredmorbius 5y ago
      There are many circumstances in which it's not. - Remotely accessing a system (terminal or GUI). - An overloaded system (GUI response is ... inconsisstent) - Remotely accessing a system from a touch-based device. An increasingly common scenario. - Walking someone through a process (text is unambiguous). - Repeated operations (something that has to be done multiple times, in multiple directories, on an ongoing basis, on a scheduled basis, reliably, consistently, provably, with documentation and debuggability). Additionally, the scrollbar seems to be increasingly unpopular. I'm on record as not being happy about this. https://ello.co/dredmorbius/post/0hgfswmoti3fi5zgftjecq https://ello.co/dredmorbius/post/0hgfswmoti3fi5zgftjecq (HN discussion: https://news.ycombinator.com/item?id=21356511 https://news.ycombinator.com/item?id=21356511)
      - Stratoscope 5y ago
        I enjoyed your flippin' scrollbar rant! I couldn't have put it better myself.
- SirSourdough 5y ago
  This feels pretty squarely in the realm of "but you could still do it manually". The CLI method still seems cleaner and faster, with less likelihood of error since you don't select the files manually.
  - wutbrodo 5y ago
    I had the same initial reaction, and I think conversations like this often include desperate justification of GUI workflows driven by an inferiority complex. But the parent comment was responding to a portion of its parent that specifically said "there's no option other than scripting". It sounds like he's responding to that, not making some general claim about the GUI being a better option.
  - bosie 5y ago
    How is not selecting it manually but implementing code that could select the wrong files any better?
corty 5y ago
rstudio is an excellent gui for R. So you can already start with R.
- a5seo 5y ago
  So is https://exploratory.io https://exploratory.io - I almost always start with it and export the R script once I have some intuitive confidence in my data.
tomrod 5y ago
Agreed.
glormph 5y ago
Also, if you need to do something (even once) and have to know how you did it three months later, write a script.
- iamgopal 5y ago
  I usually save my command history to version control every time I git commit. It helps a lot for less frequently visited projects.
  - jbverschoor 5y ago
    You can set your HIST_FILE per project. That’s what I do in vscode (terminal env), so I have a history per project. Other than that, I usually document some ops/quick read me about how to use or prepare the project
- dwater 5y ago
  Yes, the lack of reproducibility is really Excel’s biggest weakness in data analysis. Is this 5th workbook that is a 4th generation derivation from the original data statistically valid? Can you go back to each step and verify it was valid? A script that does all processing and analysis starting from raw data can be validated and audited.
- trenchgun 5y ago
  This!
rcthompson 5y ago
I often write a script even if I only have to do something twice. In this case, the point is not to save time, it's to be secure in the knowledge that I did the thing in exactly the same way both times, because it was the same script that did it both times. (And then it saves time anyway, because it turns out I had to do the thing 20 more times after all.)
- orzig 5y ago
  It’s not full proof, but the heuristic that your stakeholder will ask you for the same thing next week (or immediately ask you to do the same thing but slightly differently) has held pretty true for me
  - ACow_Adonis 5y ago
    yes, I've got a little heuristic I use to tell the beginner analysts at work when to think about automating their work or putting it into a script. if you're doing something once and once only, maybe it's a candidate for the manual or gui way. if you think you might do it twice, it's almost certainly time to automate or start programming it. Anything encountered in business that you encounter more than 1 time is likely to be encountered N times more.
    - fho 5y ago
      The good old "allow it never, once or n times" :-)
- gspr 5y ago
  Exactly. A little {insert favorite language} snippet is immediately inspectable, also weeks or months or years in the future.
  - commandlinefan 5y ago
    You underestimate my ability to write code that I myself can’t make sense of years (or sometimes even months or weeks) in the future.
wutbrodo 5y ago
> If I want to move a file from one folder to another then I just drag it across. Easy. Even this is easier with scripting, especially if you're already used to thinking in wildcards and tab-completion. Scrolling, hunting for files, dragging, clicking: these are inherently clumsier and slower steps, optimized for new-user intuitiveness and simplicity over efficiency. The upfront investment of making your brain think in CLI is fairly high, but once you've done it, there's vanishingly little reason to bother with file browsers. I don't think I've used one in a decade, even with (eg) Nautilus's ability to match wildcards with ctrl+s.
- sdenton4 5y ago
  For those who haven't used it, ctrl-r in the command line is the best thing in the world. It does a reverse-search over previous commands. Set your max history size to unlimited, and your CLI will remember /all/ commands you ever type, making it very easy to find long, strange incantations that you figured out once, so long as you can remember some small part of the command. For example, How did I do that magical ffmpeg thing last time? Just hit ctrl-r, type ffmpeg, and keep hitting ctrl-r to find previous examples until I find what I'm after.
  - sideshowb 5y ago
    For more complex commands I use often I sometimes comment the end of my line with a keyword I can search on
  - jamiek88 5y ago
    Does this work on Mac OS? I’ve got a bunch of saved notes with various incantations that would become redundant if so!
    - sdenton4 5y ago
      Seems like it should! https://stackoverflow.com/questions/41780746/searching-your-command-history-on-macos-terminal https://stackoverflow.com/questions/41780746/searching-your-... Good to double check how to turn off the history size limit before throwing away the post it notes entirely.
deleted 5y ago
[deleted]
BrandoElFollito 5y ago
I found that "Automation"[1] by xkcd (R Munroe) closely captures the lifecycle of my "clever" scripts. I have a few of them glancing at me from the corner of my desktop and I hope to do something "over the summer". [1] https://xkcd.com/1319/ https://xkcd.com/1319/
Ekaros 5y ago
And then there is the fun tasks, which I have always considered to pit the CLI guys against the GUI: Copy all cat pictures to different directory. (No metadata or names)
Const-me 5y ago
> I would benefit from learning `mv UTR-77.csv $folder` In far manager (freeware, open source) and total commander (commercial, not too expensive, trial available), numpad `+` key open “expand selection” popup, where you can type “utr-77.csv”, and it will select just these files. Unlike the command line, you can inspect what had been selected before moving these files. You can also use insert/numpad +/numpad - keys to modify the set of files going to be moved/copied/deleted/zipped/etc. Most file systems and file managers don’t have undo support. It can be important to review what going to happen before actually moving any files.
dm319 5y ago
Not sure I agree with this. Most statistical analysis is a 'do once' thing, but using Excel would not be a good choice. Reproducibility and immutability of the source data are two good reasons to write a script.
perl4ever 5y ago
But why not use the scripting languages built in to Excel?

by Zhyl 5y ago

DantesKite 5y ago

This is a great comment and the line of reasoning for work at different scales makes so much sense.

danuker 5y ago

> I want to move a thousand files beginning with 'UTR-77' and ending with '.csv' 1. Use a file manager with stable sorting [1] (I use Thunar which does this, but I suspect lots of file managers keep the sort order stable). 2. Sort by type. 3. Sort by name. 4. Select the first file named UTR-77. 5. Scroll to the last file, and Shift+Click it. 6. Cut then paste to your desired directory. [1] - https://en.wikipedia.org/wiki/Sorting_algorithm#Stability https://en.wikipedia.org/wiki/Sorting_algorithm#Stability

bicolao 5y ago
> 5. Scroll to the last file, and Shift+Click it. With thousands of files, this won't be particularly fast or easy.
- Stratoscope 5y ago
  If you know how to use a scrollbar, it is fast and easy. Grab the thumb and drag it down. If you overshoot or have trouble landing on the last file, use the page up/down areas of the scrollbar or its up/down arrow to fine tune the scroll position. Should only take a few seconds. Another way is to hold down the Ctrl key and use the PgDn/PgUp keys to scroll. If you hold down Ctrl+PgDn you can whip through thousands of files quickly. Because you have the Ctrl key down, it won't affect your selection of the first file. If all you know is two finger touchpad scrolling, then it will be very tedious. Unfortunately, from observing a number of people - and apps that hide the scrollbars - it seems to me that two finger scrolling has largely taken over from scrollbar scrolling or the scroll keys.
  - dredmorbius 5y ago
    There are many circumstances in which it's not. - Remotely accessing a system (terminal or GUI). - An overloaded system (GUI response is ... inconsisstent) - Remotely accessing a system from a touch-based device. An increasingly common scenario. - Walking someone through a process (text is unambiguous). - Repeated operations (something that has to be done multiple times, in multiple directories, on an ongoing basis, on a scheduled basis, reliably, consistently, provably, with documentation and debuggability). Additionally, the scrollbar seems to be increasingly unpopular. I'm on record as not being happy about this. https://ello.co/dredmorbius/post/0hgfswmoti3fi5zgftjecq https://ello.co/dredmorbius/post/0hgfswmoti3fi5zgftjecq (HN discussion: https://news.ycombinator.com/item?id=21356511 https://news.ycombinator.com/item?id=21356511)
    - Stratoscope 5y ago
      I enjoyed your flippin' scrollbar rant! I couldn't have put it better myself.
SirSourdough 5y ago
This feels pretty squarely in the realm of "but you could still do it manually". The CLI method still seems cleaner and faster, with less likelihood of error since you don't select the files manually.
- wutbrodo 5y ago
  I had the same initial reaction, and I think conversations like this often include desperate justification of GUI workflows driven by an inferiority complex. But the parent comment was responding to a portion of its parent that specifically said "there's no option other than scripting". It sounds like he's responding to that, not making some general claim about the GUI being a better option.
- bosie 5y ago
  How is not selecting it manually but implementing code that could select the wrong files any better?

corty 5y ago

rstudio is an excellent gui for R. So you can already start with R.

a5seo 5y ago
So is https://exploratory.io https://exploratory.io - I almost always start with it and export the R script once I have some intuitive confidence in my data.

tomrod 5y ago

Agreed.

glormph 5y ago

Also, if you need to do something (even once) and have to know how you did it three months later, write a script.

iamgopal 5y ago
I usually save my command history to version control every time I git commit. It helps a lot for less frequently visited projects.
- jbverschoor 5y ago
  You can set your HIST_FILE per project. That’s what I do in vscode (terminal env), so I have a history per project. Other than that, I usually document some ops/quick read me about how to use or prepare the project
dwater 5y ago
Yes, the lack of reproducibility is really Excel’s biggest weakness in data analysis. Is this 5th workbook that is a 4th generation derivation from the original data statistically valid? Can you go back to each step and verify it was valid? A script that does all processing and analysis starting from raw data can be validated and audited.
trenchgun 5y ago
This!

rcthompson 5y ago

I often write a script even if I only have to do something twice. In this case, the point is not to save time, it's to be secure in the knowledge that I did the thing in exactly the same way both times, because it was the same script that did it both times. (And then it saves time anyway, because it turns out I had to do the thing 20 more times after all.)

orzig 5y ago
It’s not full proof, but the heuristic that your stakeholder will ask you for the same thing next week (or immediately ask you to do the same thing but slightly differently) has held pretty true for me
- ACow_Adonis 5y ago
  yes, I've got a little heuristic I use to tell the beginner analysts at work when to think about automating their work or putting it into a script. if you're doing something once and once only, maybe it's a candidate for the manual or gui way. if you think you might do it twice, it's almost certainly time to automate or start programming it. Anything encountered in business that you encounter more than 1 time is likely to be encountered N times more.
  - fho 5y ago
    The good old "allow it never, once or n times" :-)
gspr 5y ago
Exactly. A little {insert favorite language} snippet is immediately inspectable, also weeks or months or years in the future.
- commandlinefan 5y ago
  You underestimate my ability to write code that I myself can’t make sense of years (or sometimes even months or weeks) in the future.

wutbrodo 5y ago

> If I want to move a file from one folder to another then I just drag it across. Easy. Even this is easier with scripting, especially if you're already used to thinking in wildcards and tab-completion. Scrolling, hunting for files, dragging, clicking: these are inherently clumsier and slower steps, optimized for new-user intuitiveness and simplicity over efficiency. The upfront investment of making your brain think in CLI is fairly high, but once you've done it, there's vanishingly little reason to bother with file browsers. I don't think I've used one in a decade, even with (eg) Nautilus's ability to match wildcards with ctrl+s.

sdenton4 5y ago
For those who haven't used it, ctrl-r in the command line is the best thing in the world. It does a reverse-search over previous commands. Set your max history size to unlimited, and your CLI will remember /all/ commands you ever type, making it very easy to find long, strange incantations that you figured out once, so long as you can remember some small part of the command. For example, How did I do that magical ffmpeg thing last time? Just hit ctrl-r, type ffmpeg, and keep hitting ctrl-r to find previous examples until I find what I'm after.
- sideshowb 5y ago
  For more complex commands I use often I sometimes comment the end of my line with a keyword I can search on
- jamiek88 5y ago
  Does this work on Mac OS? I’ve got a bunch of saved notes with various incantations that would become redundant if so!
  - sdenton4 5y ago
    Seems like it should! https://stackoverflow.com/questions/41780746/searching-your-command-history-on-macos-terminal https://stackoverflow.com/questions/41780746/searching-your-... Good to double check how to turn off the history size limit before throwing away the post it notes entirely.

deleted 5y ago

[deleted]

BrandoElFollito 5y ago

I found that "Automation"[1] by xkcd (R Munroe) closely captures the lifecycle of my "clever" scripts. I have a few of them glancing at me from the corner of my desktop and I hope to do something "over the summer". [1] https://xkcd.com/1319/ https://xkcd.com/1319/

Ekaros 5y ago

And then there is the fun tasks, which I have always considered to pit the CLI guys against the GUI: Copy all cat pictures to different directory. (No metadata or names)

Const-me 5y ago

> I would benefit from learning `mv UTR-77.csv $folder` In far manager (freeware, open source) and total commander (commercial, not too expensive, trial available), numpad `+` key open “expand selection” popup, where you can type “utr-77.csv”, and it will select just these files. Unlike the command line, you can inspect what had been selected before moving these files. You can also use insert/numpad +/numpad - keys to modify the set of files going to be moved/copied/deleted/zipped/etc. Most file systems and file managers don’t have undo support. It can be important to review what going to happen before actually moving any files.

dm319 5y ago

Not sure I agree with this. Most statistical analysis is a 'do once' thing, but using Excel would not be a good choice. Reproducibility and immutability of the source data are two good reasons to write a script.

perl4ever 5y ago

But why not use the scripting languages built in to Excel?