6 ms·

I’ve been working on Makefiles lately and its funny how many compatibility issues there are. I was doing Sed substitutions and found I needed to change the -i f

I’ve been working on Makefiles lately and its funny how many compatibility issues there are. I was doing Sed substitutions and found I needed to change the -i flag to make it work between Mac and Linux. I was really hoping for ultimate portability but ended up learning lots of hacks that lead to portability

pjmlp 4y ago
GNU and POSIX differences, https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/
- DougMerritt 4y ago
  The second link is the same as the first; presumably you meant to post a different second link?
sacnoradhq 4y ago
sed -i '' .... is compatible with both. It's not a big deal.
- asicsp 4y ago
  Doesn't work with `GNU sed`: $ sed -i '' 's/foo/FOO/' ip.txt sed: can't read s/foo/FOO/: No such file or directory See also: https://stackoverflow.com/questions/5694228/sed-in-place-flag-that-works-both-on-mac-bsd-and-linux https://stackoverflow.com/questions/5694228/sed-in-place-fla...
  - tempodox 4y ago
    But this works: $ sed -e 's/foo/FOO/' -i 'ip.txt'
    - asicsp 4y ago
      That's same as `sed -i 's/foo/FOO/' ip.txt` on GNU sed. Question is about inplace editing command that works on both GNU and MacOS (without needing a backup, with backup `-i.bkp` works on both). I don't have a Mac to check if `sed -e 's/foo/FOO/' -i 'ip.txt'` works there.
      - tempodox 4y ago
        I checked, it doesn't. I see your point.
psyclobe 4y ago
This is why powershell is gaining traction
- pxc 4y ago
  PowerShell has platform-specific modules and ships by default with a ton of compatibility-breaking aliases. It doesn't really solve this problem
  - psyclobe 4y ago
    Sure it does, all the standard calls are universal, hell I wrote a _universal_ directory monitoring shell script. Try and do that with bash on windows/linux/osx.
    - cf100clunk 4y ago
      >I wrote a _universal_ directory monitoring shell script That might be a good Show HN example.
      - pxc 4y ago
        Seconded!
    - pxc 4y ago
      If you're going to use PowerShell like a shell (i.e., to call external programs), you still have to worry about distributing them or variations between what's preinstalled. If you're only going to use PowerShell functions and not external programs anyway, then PowerShell is more in competition with languages like Python, Perl, Ruby, etc., than it is with Bash. PowerShell also falls down when it comes to the other distinguishing feature of a shell, which is that you gradually learn your scripting language just by using your computer, because PowerShell just falls flat as a daily login shell. This is mainly because it's unbelievably dog slow for a shell and lacks basic functionality like job control, but arguably also because of smaller things like its verbosity, extremely complicated profile sourcing behavior, etc. PowerShell is still sometimes a good choice for cross-platform scripting, but I wouldn't say that's because of its compatibility.
bdhcuidbebe 4y ago
Differences in GNU/BSD sed is hardly a bug in Make.
- dspillett 4y ago
  No, correct. But it is a good example of something that you might expect to be more standard than it is between platforms, because of its age and relative ubiquity, where the differences get highlighted when trying to create (not just with make but other tools too) a consistent build process for a project.
  - czx4f4bd 4y ago
    Yeah, variance between different system utilities is one of the biggest underrated perils of shell scripting. For a long time, I worked on a team that mostly used Macs to administer Linux servers, which led to some confusing issues when people tested certain commands locally and then tried to use them on a server. It would be nice if shellcheck had checks to detect common BSD/GNU incompatibilities and warn about them so you don't have to catch them in the moment.
    - hulitu 4y ago
      I think that if one aims for compatibility , then it shall use POSIX.
- arp242 4y ago
  It wasn't claimed to be a bug.
- SAI_Peregrinus 4y ago
  But it is a bug in your Makefile to assume a particular version of a tool (or path to a tool, etc) without first checking (or otherwise forcing the correct thing to be used). For forcing the existence of tools, you can start your Makefile by setting up $PATH with the dependencies your script has (everything not guaranteed by POSIX, for example). E.g. with Nix `export PATH := $(shell nix-shell -p DEPENDENCY1 DEPENDENCY2 --run 'echo $$PATH')` Will cause `make` to include DEPENDENCY1 and DEPENDENCY2 from the Nix store in $PATH for the rest of the Makefile. You can get more complex and pin particular versions where it matters. Other cross-platform package management tools presumably have similar tricks that can be used, or you can bundle binaries for the tools you need. Then your build instructions only need to tell the user to have Make and the package management tool installed, and everything else can be set up in the Makefile. Of course this is a PITA.

pjmlp 4y ago

GNU and POSIX differences, https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/

DougMerritt 4y ago
The second link is the same as the first; presumably you meant to post a different second link?

sacnoradhq 4y ago

sed -i '' .... is compatible with both. It's not a big deal.

asicsp 4y ago
Doesn't work with `GNU sed`: $ sed -i '' 's/foo/FOO/' ip.txt sed: can't read s/foo/FOO/: No such file or directory See also: https://stackoverflow.com/questions/5694228/sed-in-place-flag-that-works-both-on-mac-bsd-and-linux https://stackoverflow.com/questions/5694228/sed-in-place-fla...
- tempodox 4y ago
  But this works: $ sed -e 's/foo/FOO/' -i 'ip.txt'
  - asicsp 4y ago
    That's same as `sed -i 's/foo/FOO/' ip.txt` on GNU sed. Question is about inplace editing command that works on both GNU and MacOS (without needing a backup, with backup `-i.bkp` works on both). I don't have a Mac to check if `sed -e 's/foo/FOO/' -i 'ip.txt'` works there.
    - tempodox 4y ago
      I checked, it doesn't. I see your point.

psyclobe 4y ago

This is why powershell is gaining traction

pxc 4y ago
PowerShell has platform-specific modules and ships by default with a ton of compatibility-breaking aliases. It doesn't really solve this problem
- psyclobe 4y ago
  Sure it does, all the standard calls are universal, hell I wrote a _universal_ directory monitoring shell script. Try and do that with bash on windows/linux/osx.
  - cf100clunk 4y ago
    >I wrote a _universal_ directory monitoring shell script That might be a good Show HN example.
    - pxc 4y ago
      Seconded!
  - pxc 4y ago
    If you're going to use PowerShell like a shell (i.e., to call external programs), you still have to worry about distributing them or variations between what's preinstalled. If you're only going to use PowerShell functions and not external programs anyway, then PowerShell is more in competition with languages like Python, Perl, Ruby, etc., than it is with Bash. PowerShell also falls down when it comes to the other distinguishing feature of a shell, which is that you gradually learn your scripting language just by using your computer, because PowerShell just falls flat as a daily login shell. This is mainly because it's unbelievably dog slow for a shell and lacks basic functionality like job control, but arguably also because of smaller things like its verbosity, extremely complicated profile sourcing behavior, etc. PowerShell is still sometimes a good choice for cross-platform scripting, but I wouldn't say that's because of its compatibility.

bdhcuidbebe 4y ago

Differences in GNU/BSD sed is hardly a bug in Make.

dspillett 4y ago
No, correct. But it is a good example of something that you might expect to be more standard than it is between platforms, because of its age and relative ubiquity, where the differences get highlighted when trying to create (not just with make but other tools too) a consistent build process for a project.
- czx4f4bd 4y ago
  Yeah, variance between different system utilities is one of the biggest underrated perils of shell scripting. For a long time, I worked on a team that mostly used Macs to administer Linux servers, which led to some confusing issues when people tested certain commands locally and then tried to use them on a server. It would be nice if shellcheck had checks to detect common BSD/GNU incompatibilities and warn about them so you don't have to catch them in the moment.
  - hulitu 4y ago
    I think that if one aims for compatibility , then it shall use POSIX.
arp242 4y ago
It wasn't claimed to be a bug.
SAI_Peregrinus 4y ago
But it is a bug in your Makefile to assume a particular version of a tool (or path to a tool, etc) without first checking (or otherwise forcing the correct thing to be used). For forcing the existence of tools, you can start your Makefile by setting up $PATH with the dependencies your script has (everything not guaranteed by POSIX, for example). E.g. with Nix `export PATH := $(shell nix-shell -p DEPENDENCY1 DEPENDENCY2 --run 'echo $$PATH')` Will cause `make` to include DEPENDENCY1 and DEPENDENCY2 from the Nix store in $PATH for the rest of the Makefile. You can get more complex and pin particular versions where it matters. Other cross-platform package management tools presumably have similar tricks that can be used, or you can bundle binaries for the tools you need. Then your build instructions only need to tell the user to have Make and the package management tool installed, and everything else can be set up in the Makefile. Of course this is a PITA.