5 ms·

I’ve seen packages that do ”internet-detection” by calling out to icanhazip.com, and I just thought that was so irresposnible. What if your package got popular,

by OskarS 5y ago

I’ve seen packages that do ”internet-detection” by calling out to icanhazip.com, and I just thought that was so irresposnible. What if your package got popular, how much money are you costing the hoster? For services like this, people just don’t consider the fact that there’s someone on the other side.

ljm 5y ago
I feel the same about dependency steps in CI, without a cache or any similar structure. Package repos like Rubygems, NPM and PyPi get utterly rinsed by the continual downloading and redownloading of stuff the client should have already stored.
- trulyme 5y ago
  This. And both with GitHub and GitLab it takes quite a bit of an extra effort to setup caching. It hurts to see 'npm ci' download half the internet every time a developer pushes to dev server.
  - yellow_lead 5y ago
    Would be interesting to speculate about the greenhouse effect of all these repeated downloads
    - bombcar 5y ago
      It used to be possible to have a squid or even a backwards varnish cache handle lots of this but https everything has made that much harder to do. Still possible, however.
    - ljm 5y ago
      If nothing else, it is patently wasteful and, as a user, you don't really see CI billed in terms of network bandwidth. Just indirectly through the equivalent of mainframe minutes. And even then, that's not enough to discourage anyone from building a suboptimal pipeline.
- Seirdy 5y ago
  That's why the first step of CI for me, when possible, is to rsync a .tar.gz file from the server I'm deploying to. The tarball contains statically-linked binaries and other stuff I'll need for the build. It's also a good reason for CI providers to mirror package repositories.
- SilverRed 5y ago
  Nothing is going to change until the hosters make it a pain to abuse. Rubygems could require an api key to download from and rate limit that key. Sure you could attempt to generate a bunch of keys and cycle them but it would be easier to just cache your gems.
kortilla 5y ago
The article was about abusive floods accounting for 90% of the traffic. The author was happy with legitimate use cases like packages doing detection, contrary to your comment.
Seirdy 5y ago
If you want, you can set up a similar service yourself by adding the following lines to an NGINX config: location = /ip { default_type text/plain; return 200 '$remote_addr'; } Requesting "yoursite.tld/ip" will then return your IP address. I set up something like this on all my servers and recommend that others do the same. It's easy to do the same for Apache and Caddy configs. That should help spread the load. I'm curious as to what other overused utilities can be trivially done with pure server configs.
- jacobmischka 5y ago
  Is it easy to do the same for Apache? The best solution I found was some hacky way with an ErrorDocument directive which seems pretty gross.
  - Ayesh 5y ago
    You can use SSI, and echo the remote IP: https://httpd.apache.org/docs/2.4/howto/ssi.html https://httpd.apache.org/docs/2.4/howto/ssi.html ```  ```
- lsiebert 5y ago
  Doing this for HN asks you to login as an administrator.
- zie 5y ago
  If you want JSON instead: location /ip { add_header Content-Type "application/json"; return 200 '{"host":"$server_name","ip":"$remote_addr","port":"$remote_port","server_ip":"$server_addr","server_port":"$server_port"}\n'; }

by OskarS 5y ago

ljm 5y ago

I feel the same about dependency steps in CI, without a cache or any similar structure. Package repos like Rubygems, NPM and PyPi get utterly rinsed by the continual downloading and redownloading of stuff the client should have already stored.

trulyme 5y ago
This. And both with GitHub and GitLab it takes quite a bit of an extra effort to setup caching. It hurts to see 'npm ci' download half the internet every time a developer pushes to dev server.
- yellow_lead 5y ago
  Would be interesting to speculate about the greenhouse effect of all these repeated downloads
  - bombcar 5y ago
    It used to be possible to have a squid or even a backwards varnish cache handle lots of this but https everything has made that much harder to do. Still possible, however.
  - ljm 5y ago
    If nothing else, it is patently wasteful and, as a user, you don't really see CI billed in terms of network bandwidth. Just indirectly through the equivalent of mainframe minutes. And even then, that's not enough to discourage anyone from building a suboptimal pipeline.
Seirdy 5y ago
That's why the first step of CI for me, when possible, is to rsync a .tar.gz file from the server I'm deploying to. The tarball contains statically-linked binaries and other stuff I'll need for the build. It's also a good reason for CI providers to mirror package repositories.
SilverRed 5y ago
Nothing is going to change until the hosters make it a pain to abuse. Rubygems could require an api key to download from and rate limit that key. Sure you could attempt to generate a bunch of keys and cycle them but it would be easier to just cache your gems.

kortilla 5y ago

The article was about abusive floods accounting for 90% of the traffic. The author was happy with legitimate use cases like packages doing detection, contrary to your comment.

Seirdy 5y ago

If you want, you can set up a similar service yourself by adding the following lines to an NGINX config: location = /ip { default_type text/plain; return 200 '$remote_addr'; } Requesting "yoursite.tld/ip" will then return your IP address. I set up something like this on all my servers and recommend that others do the same. It's easy to do the same for Apache and Caddy configs. That should help spread the load. I'm curious as to what other overused utilities can be trivially done with pure server configs.

jacobmischka 5y ago
Is it easy to do the same for Apache? The best solution I found was some hacky way with an ErrorDocument directive which seems pretty gross.
- Ayesh 5y ago
  You can use SSI, and echo the remote IP: https://httpd.apache.org/docs/2.4/howto/ssi.html https://httpd.apache.org/docs/2.4/howto/ssi.html ```  ```
lsiebert 5y ago
Doing this for HN asks you to login as an administrator.
zie 5y ago
If you want JSON instead: location /ip { add_header Content-Type "application/json"; return 200 '{"host":"$server_name","ip":"$remote_addr","port":"$remote_port","server_ip":"$server_addr","server_port":"$server_port"}\n'; }