6 ms·

Go Replace - simple and fast search and replace tool for command line

70 points by piranha 13y ago

gglanzani 13y ago
Are 32bit binaries coming? For the rest, great job!
- piranha 13y ago
  Hah, haven't thought about them (I use osx too much it seems). They will appear as soon as I figure out how to add them to Makefile without making it too ugly. :)
- piranha 13y ago
  Here we go, links are in readme. They work for me on 64-bit OS X and 64-bit Linux and `file` reports they are indeed 32-bit, but I haven't tested them on real 32-bit OS (I don't have any ATM).
porker 13y ago
I have to break out the manual every time I use sed. Or awk. If this works reliably without any edge cases, it'll be fab!
- piranha 13y ago
  It does for me. :) I partially did it because I was too tired of looking up sed's regexp syntax and was afraid I'm going to fuck up everything.
M4v3R 13y ago
Wow, it's super fast AND the output looks great. Good job, piranha! Copied to /usr/bin, I will use this a lot.
- piranha 13y ago
  Thanks!
- nitrogen 13y ago
  If you're the only user on your box, you can just create ~/bin and some distributions will automatically add it to your personal PATH. That way you don't lose track of what you've copied into /usr/bin, leaving it to be maintained by packages.
  - pekk 13y ago
    you can also very easily add ~/bin to PATH in e.g. your .bashrc
va1en0k 13y ago
I really like Facebook's codemod.py for stuff like that: https://github.com/facebook/codemod https://github.com/facebook/codemod It has something like automatic and manual modes, so you can check (and fix) every diff during the replacement
- piranha 13y ago
  Well you see... It's in Python, which breaks the deal for me (I despise slow startup). But interactive mode sounds interesting, that's something I'll consider implementing.
  - pekk 13y ago
    By slow startup do you mean 0.01s of wall time? This sounds more like a prejudice than a decision with a strong technical basis.
    - piranha 13y ago
      By slow startup I mean that 0.1s is the fastest small Python program using optparse, os, sys, re will start up on my system (i7 2.0, ssd). Search and replace tool will be slower. I constantly get 0.02s-0.3s results from 'gr' on my small-to-medium repositories (up to ~80k sloc). 'ack' takes more than a second usually. Also it was written when I had HDD (not SSD) and codebase was on an encrypted image. It made enormous difference to me.
      - cfrss 13y ago
        btw, here is ag, which also looks .xignore files: https://github.com/ggreer/the_silver_searcher https://github.com/ggreer/the_silver_searcher
        piranha 13y ago
        Yes, I've mentioned it in my post. :) It's nice, but it does not perform replaces. I could add .xignore support though.
madisp 13y ago
For x86 binaries: go get github.com/piranha/goreplace go install github.com/piranha/goreplace is another way to get it if you have GO and GOPATH set up. I personally aliased it to 'gor' in my zshrc as the git plugin already occupied the 'gr' (short for git remote).
- piranha 13y ago
  Ha-ha, I've got an issue asking to add some notes about conflict with oh-my-zsh alias. :) I hastily suggested `gp`, but it's already taken by same plugin, so I changed it to `gor` there, thanks for idea.
  - alexchamberlain 13y ago
    Is tab completion not enough?
    - piranha 13y ago
      It's often is. But somehow people end up with 'git remote' aliased to 'gr', and I ask the same question. :) I use gr very often and so not have to tab complete is a plus.
JoeAcchino 13y ago
find ~/my/docs/ -type f -name '*.txt' -exec sed -i.bak 's/inheritance/composition/g' {} + This will search all files ending in `.txt` and will exec `sed 's/inheritance/composition/g'` on it. Sed modifies files in-place and saves a backup file with .bak extension (-i.bak) and is called the minimum necessary times (-exec +). My best advice to someone using the command line is to learn `find` + `xargs` or, even better, `find` + `parallel`.
- gladimdim 13y ago
  Compare with "gr what-is-it -r here-you-go". I do not have to remember all that argument hell.
  - anonymouz 13y ago
    Well, one can write an one-line shell alias (or script) instead of reinventing the wheel by rewriting the whole thing anew in Go. (Which is certainly good as learning experience for Go, but the end result is easier to achieve with the shell)
    - piranha 13y ago
      When you start adding coloring, nice output, .hg/.gitignore support, it becomes just a big mess. Been there, done that.
      - andreypopp 13y ago
        coloring is already supported by grep, .hg/.gitignore support is just a few lines
        piranha 13y ago
        Just do it then.
  - dhamidi 13y ago
    Probably slower than the go version, but more flexible since you have more over the files processed. # bash # find/replace function fr { local pattern=$1 local replacement=$2 local program if [[ -z "$replacement" ]] then program="grep $pattern" else program="sed -i s/$pattern/$replacement/g" fi while read line do $program "$line" done } # usage find . | fr find-this find . | fr replace-this with-that
lazyjones 13y ago
Nice! svn support would be useful (ignore .svn directories) and the -r option is easy to remember, but perhaps not so much in line with the already messy standard Unix options (-r or -R for recursive operation). Note that specifically for Go source code, there is "go fix -r" for syntax/context-aware code replacement (see "godoc fix").
- piranha 13y ago
  In case of go fix `-r` does something absolutely unrelated. To be honest, I was just lazy inventing a command line key and it's recursive by design, so it happened this way. Plus, as you say, it's easy to remember because of mnemonics. I haven't touched svn repos in years, but yeah, that could be added. I know they exist, it just that I haven't felt the need to support them.
  - piranha 13y ago
    Btw, it ignores .svn and a bit more common stuff if it doesn't find itself in a git or hg repo.
- thameera 13y ago
  Exactly. gr can be for find/sed like ack is for grep.
swah 13y ago
This isn't working for me. I get: C:\Users\swah\docs>gr TEST panic: Given path should be anchored at / goroutine 1 [running]: main.NewIgnorer(0xc08005bcf0, 0x29, 0x0, 0x0, 0x140110, ...) /Users/piranha/dev/go/src/goreplace/ignore.go:32 +0xa4 main.main() /Users/piranha/dev/go/src/goreplace/goreplace.go:62 +0x1b7 goroutine 2 [runnable]: [edit] New version is working!
- piranha 13y ago
  Argh! I see, I think I can solve this. Give me a bit of time please. :)
- piranha 13y ago
  Should be fixed, can you download new version please?
  - swah 13y ago
    Working! Do you use colors? They don't work the same in DOS, I believe. I'm seeing some escape sequences. ←[0;32;49mRS\terms.tex ←[0m←[1;39;49m←[0;33;49m25:←[0m ←[0;39;43mOEM←[0m & \\ \hline
    - piranha 13y ago
      Oh, it's the library I use has no idea about colors in DOS. :( I'll have to do some research on how to fix this... :\
    - piranha 13y ago
      Eh, it's not very easy to fix that unfortunately. I opened an issue and will try to work on this a bit, but then in a meanwhile... Maybe I can strip colors for outputting things on windows. Edit: argh, outputting colors on windows means making system calls. Which means cgo and good-bye cross-compiling. I'll probably just strip all colors...
      - 13y ago
zokier 13y ago
what's the rationale behind 'gr foo -r bar' instead of simply 'gr foo bar'?
- piranha 13y ago
  Just happened so, plus if you forgot to put quotes around argument you could easily end up with a lot of problems. :)
artagnon 13y ago
Okay, so what I learnt from this: 1. You've imported droundy/goopt, wsxiaoys/terminal really easily. The first provides option-parsing, and the second provides helpers for emitting ANSI escape codes. 2. Practically everything is overloaded. You can == to strcmp(), + to concatenate strings. 3. path/filepath gives you some nice goodies for path manipulation. But you have to deal with the fallouts of making naive assumptions like whether or not to recurse symlinks (why isn't this a flag in filepath.Walk?). This has resulted in ugliness in your walkFunc callback. 4. You have been able to abstract quite a bit without using Java-like factories, although your inconsistent error passing has me somewhat confused. 5. Since it's so strongly typed and there are no raw pointers, it's not going to be hard to debug at all. Overall, I'd say Go is an effective language. But not beautiful, or novel in the slightest. It resembles Algol-68 (yes, a 1968 language) quite strongly, and I don't think it will drive compiler technology. Their syntax is specifically engineered to be parseable quickly (and hence super-fast AOT compilation): what the final program looks like is secondary concern. I'm not sure I see the appeal. [edit: clarified that the note on compiler technology was an opinion]
- gngeal 13y ago
  and proudly declares that it will not use any modern advances to compiler technology Where do they say anything of the kind?
- piranha 13y ago
  > your inconsistent error passing has me somewhat confused. That was my first somewhat big program in Go plus coming from Python made a bit of hustle here. I still believe exceptions to be much better than this error-returning. :\ Never learned to do it nicely. :\ > Overall, I'd say Go is an effective language. But not beautiful, or novel in the slightest. Yes, this is more or less my feelings nowadays. It's good for certain cases and if I ever need something fast, I'll use Go and not C. > I'm not sure I see the appeal. There is some. It's still much simpler than C (GC + stricter compiler), plus it has goroutines (not much in this app, though), plus interfaces are really wonderful (you don't need to declare that you implement interface, just implement it and you're done).
  - 13y ago

70 points by piranha 13y ago

gglanzani 13y ago

Are 32bit binaries coming? For the rest, great job!

piranha 13y ago
Hah, haven't thought about them (I use osx too much it seems). They will appear as soon as I figure out how to add them to Makefile without making it too ugly. :)
piranha 13y ago
Here we go, links are in readme. They work for me on 64-bit OS X and 64-bit Linux and `file` reports they are indeed 32-bit, but I haven't tested them on real 32-bit OS (I don't have any ATM).

porker 13y ago

I have to break out the manual every time I use sed. Or awk. If this works reliably without any edge cases, it'll be fab!

piranha 13y ago
It does for me. :) I partially did it because I was too tired of looking up sed's regexp syntax and was afraid I'm going to fuck up everything.

M4v3R 13y ago

Wow, it's super fast AND the output looks great. Good job, piranha! Copied to /usr/bin, I will use this a lot.

piranha 13y ago
Thanks!
nitrogen 13y ago
If you're the only user on your box, you can just create ~/bin and some distributions will automatically add it to your personal PATH. That way you don't lose track of what you've copied into /usr/bin, leaving it to be maintained by packages.
- pekk 13y ago
  you can also very easily add ~/bin to PATH in e.g. your .bashrc

va1en0k 13y ago

I really like Facebook's codemod.py for stuff like that: https://github.com/facebook/codemod https://github.com/facebook/codemod It has something like automatic and manual modes, so you can check (and fix) every diff during the replacement

piranha 13y ago
Well you see... It's in Python, which breaks the deal for me (I despise slow startup). But interactive mode sounds interesting, that's something I'll consider implementing.
- pekk 13y ago
  By slow startup do you mean 0.01s of wall time? This sounds more like a prejudice than a decision with a strong technical basis.
  - piranha 13y ago
    By slow startup I mean that 0.1s is the fastest small Python program using optparse, os, sys, re will start up on my system (i7 2.0, ssd). Search and replace tool will be slower. I constantly get 0.02s-0.3s results from 'gr' on my small-to-medium repositories (up to ~80k sloc). 'ack' takes more than a second usually. Also it was written when I had HDD (not SSD) and codebase was on an encrypted image. It made enormous difference to me.
    - cfrss 13y ago
      btw, here is ag, which also looks .xignore files: https://github.com/ggreer/the_silver_searcher https://github.com/ggreer/the_silver_searcher
      - piranha 13y ago
        Yes, I've mentioned it in my post. :) It's nice, but it does not perform replaces. I could add .xignore support though.

madisp 13y ago

For x86 binaries: go get github.com/piranha/goreplace go install github.com/piranha/goreplace is another way to get it if you have GO and GOPATH set up. I personally aliased it to 'gor' in my zshrc as the git plugin already occupied the 'gr' (short for git remote).

piranha 13y ago
Ha-ha, I've got an issue asking to add some notes about conflict with oh-my-zsh alias. :) I hastily suggested `gp`, but it's already taken by same plugin, so I changed it to `gor` there, thanks for idea.
- alexchamberlain 13y ago
  Is tab completion not enough?
  - piranha 13y ago
    It's often is. But somehow people end up with 'git remote' aliased to 'gr', and I ask the same question. :) I use gr very often and so not have to tab complete is a plus.

JoeAcchino 13y ago

find ~/my/docs/ -type f -name '*.txt' -exec sed -i.bak 's/inheritance/composition/g' {} + This will search all files ending in `.txt` and will exec `sed 's/inheritance/composition/g'` on it. Sed modifies files in-place and saves a backup file with .bak extension (-i.bak) and is called the minimum necessary times (-exec +). My best advice to someone using the command line is to learn `find` + `xargs` or, even better, `find` + `parallel`.

gladimdim 13y ago
Compare with "gr what-is-it -r here-you-go". I do not have to remember all that argument hell.
- anonymouz 13y ago
  Well, one can write an one-line shell alias (or script) instead of reinventing the wheel by rewriting the whole thing anew in Go. (Which is certainly good as learning experience for Go, but the end result is easier to achieve with the shell)
  - piranha 13y ago
    When you start adding coloring, nice output, .hg/.gitignore support, it becomes just a big mess. Been there, done that.
    - andreypopp 13y ago
      coloring is already supported by grep, .hg/.gitignore support is just a few lines
      - piranha 13y ago
        Just do it then.
- dhamidi 13y ago
  Probably slower than the go version, but more flexible since you have more over the files processed. # bash # find/replace function fr { local pattern=$1 local replacement=$2 local program if [[ -z "$replacement" ]] then program="grep $pattern" else program="sed -i s/$pattern/$replacement/g" fi while read line do $program "$line" done } # usage find . | fr find-this find . | fr replace-this with-that

lazyjones 13y ago

Nice! svn support would be useful (ignore .svn directories) and the -r option is easy to remember, but perhaps not so much in line with the already messy standard Unix options (-r or -R for recursive operation). Note that specifically for Go source code, there is "go fix -r" for syntax/context-aware code replacement (see "godoc fix").

piranha 13y ago
In case of go fix `-r` does something absolutely unrelated. To be honest, I was just lazy inventing a command line key and it's recursive by design, so it happened this way. Plus, as you say, it's easy to remember because of mnemonics. I haven't touched svn repos in years, but yeah, that could be added. I know they exist, it just that I haven't felt the need to support them.
- piranha 13y ago
  Btw, it ignores .svn and a bit more common stuff if it doesn't find itself in a git or hg repo.
thameera 13y ago
Exactly. gr can be for find/sed like ack is for grep.

swah 13y ago

This isn't working for me. I get: C:\Users\swah\docs>gr TEST panic: Given path should be anchored at / goroutine 1 [running]: main.NewIgnorer(0xc08005bcf0, 0x29, 0x0, 0x0, 0x140110, ...) /Users/piranha/dev/go/src/goreplace/ignore.go:32 +0xa4 main.main() /Users/piranha/dev/go/src/goreplace/goreplace.go:62 +0x1b7 goroutine 2 [runnable]: [edit] New version is working!

piranha 13y ago
Argh! I see, I think I can solve this. Give me a bit of time please. :)
piranha 13y ago
Should be fixed, can you download new version please?
- swah 13y ago
  Working! Do you use colors? They don't work the same in DOS, I believe. I'm seeing some escape sequences. ←[0;32;49mRS\terms.tex ←[0m←[1;39;49m←[0;33;49m25:←[0m ←[0;39;43mOEM←[0m & \\ \hline
  - piranha 13y ago
    Oh, it's the library I use has no idea about colors in DOS. :( I'll have to do some research on how to fix this... :\
  - piranha 13y ago
    Eh, it's not very easy to fix that unfortunately. I opened an issue and will try to work on this a bit, but then in a meanwhile... Maybe I can strip colors for outputting things on windows. Edit: argh, outputting colors on windows means making system calls. Which means cgo and good-bye cross-compiling. I'll probably just strip all colors...
    - 13y ago

zokier 13y ago

what's the rationale behind 'gr foo -r bar' instead of simply 'gr foo bar'?

piranha 13y ago
Just happened so, plus if you forgot to put quotes around argument you could easily end up with a lot of problems. :)

artagnon 13y ago

Okay, so what I learnt from this: 1. You've imported droundy/goopt, wsxiaoys/terminal really easily. The first provides option-parsing, and the second provides helpers for emitting ANSI escape codes. 2. Practically everything is overloaded. You can == to strcmp(), + to concatenate strings. 3. path/filepath gives you some nice goodies for path manipulation. But you have to deal with the fallouts of making naive assumptions like whether or not to recurse symlinks (why isn't this a flag in filepath.Walk?). This has resulted in ugliness in your walkFunc callback. 4. You have been able to abstract quite a bit without using Java-like factories, although your inconsistent error passing has me somewhat confused. 5. Since it's so strongly typed and there are no raw pointers, it's not going to be hard to debug at all. Overall, I'd say Go is an effective language. But not beautiful, or novel in the slightest. It resembles Algol-68 (yes, a 1968 language) quite strongly, and I don't think it will drive compiler technology. Their syntax is specifically engineered to be parseable quickly (and hence super-fast AOT compilation): what the final program looks like is secondary concern. I'm not sure I see the appeal. [edit: clarified that the note on compiler technology was an opinion]

gngeal 13y ago
and proudly declares that it will not use any modern advances to compiler technology Where do they say anything of the kind?
piranha 13y ago
> your inconsistent error passing has me somewhat confused. That was my first somewhat big program in Go plus coming from Python made a bit of hustle here. I still believe exceptions to be much better than this error-returning. :\ Never learned to do it nicely. :\ > Overall, I'd say Go is an effective language. But not beautiful, or novel in the slightest. Yes, this is more or less my feelings nowadays. It's good for certain cases and if I ever need something fast, I'll use Go and not C. > I'm not sure I see the appeal. There is some. It's still much simpler than C (GC + stricter compiler), plus it has goroutines (not much in this app, though), plus interfaces are really wonderful (you don't need to declare that you implement interface, just implement it and you're done).
- 13y ago