9 ms·

GNU Wget2 1.99.2 (Beta)

68 points by lelf 7y ago

saagarjha 7y ago
I didn't even know that there was a Wget2…what's the development progress on this? Can (or should) I be using it instead of Wget?
- msla 7y ago
  It can download faster due to using more modern features (parallel downloading, If-Modified-Since) but the main focus is to make a cleaner codebase which is built around a reusable library, so it's easier to add features and make other code which uses those features, and so the libcurl monoculture gets a little fire under its ass.
  - paulddraper 7y ago
    Does libcurl need a fire under its ass?
    - yjftsjthsd-h 7y ago
      Every monoculture does.
- darnir 7y ago
  Wget2 has been in development for a few years now. It is meant to be a drop in replacement for Wget with nearly full backward compatibility with all the old switches. In most cases you can replace wget with wget2 and not really realize the difference. For the known differences, please refer to the documentation in the gitlab repo. There's still a few rough edges though. And we very much welcome any criticisms / bug reports about the same.
msla 7y ago
> In many cases Wget2 downloads much faster than Wget1.x due to HTTP2, HTTP compression, parallel connections and use of If-Modified-Since HTTP header. This appears to be offical: https://gitlab.com/gnuwget/wget2 https://gitlab.com/gnuwget/wget2 Which has this wiki: https://gitlab.com/gnuwget/wget2/wikis/home https://gitlab.com/gnuwget/wget2/wikis/home > The idea is to have a fresh and maintainable codebase with features like multithreaded downloads, HTTP2, OCSP, HSTS, Metalink, IDNA2008, Public Suffix List, Multi-Proxies, Sitemaps, Atom/RSS Feeds, compression (gzip, deflate, lzma, bzip2), support for local filenames, etc. > Some of these feature have been built into Wget in the meantime, but some other are really hard to implement into the old codebase.
- darnir 7y ago
  Another feature not mentioned in there is support for verifying GPG signatures of binaries you download. If a gpg signature file exists, Wget2 will automatically verify the binary against that signature file
  - rurban 7y ago
    With a .sig or. asc extension? gnu differs from gpg here.
    - darnir 7y ago
      Configurable. Default extension type is sig. However you can specify a list of extensions and it will attempt to verify against the first it finds.
chocolatkey 7y ago
It looks like the biggest new feature is support for http2. Will http3 be in wget2 or a "wget3" though when it eventually happens?
- layoutIfNeeded 7y ago
  Does it refuse to support plaintext HTTP2 like browsers do?
  - darnir 7y ago
    Yes. It does not support plaintext http2.
- darnir 7y ago
  Nope. Wget2 was required since we wanted to break some backward compatibility with Wget 1.x releases. So we decided that a major version bump is warranted. It is also an entire rewrite of Wget in order to support HTTP/2. But this won't be required again when HTTP/3 or other protocols come out
  - marvy 7y ago
    > we wanted to break some backward compatibility examples?
pram 7y ago
so will you explicitly need to specify 'wget2' in a package manager, or is this replacing wget. i certainly hope it's not the former!
- chubot 7y ago
  wget() { wget2 "$@"; } or alias wget=wget2
  - johnday 7y ago
    Those are solutions for the shell, not for the package manager.
- peterwwillis 7y ago
  If they replaced 'wget', and you need HTTP2 support, you might be using the old 'wget' without HTTP2 support, or the new one with it. You would have to wrap all your commands with a version check, and ask people to install the newer, different software package, to get HTTP2 support. Or, call 'wget2' to get HTTP2 support. You can then either install both 'wget' and 'wget2', or the package maintainers could install a symlink for 'wget' backwards compatibility, so you'd only need to install the newer 'wget2' package. From the package maintainer's perspective, they're literally different software projects, and changing an established package name to be different software would break many systems.
  - Fnoord 7y ago
    This could be solved with wget1 for Wget 1.x, and wget/wget2 binaries (linked) for Wget 2.x. Only thing that does is break scripts.
    - peterwwillis 7y ago
      > Only thing that does is break scripts That's not really an acceptable change... Would you like all the scripts you run on your Linux distros to break every time you do package updates? I wouldn't touch such a distro with a 100 foot pole
farisjarrah 7y ago
Anyone know if/when this will filter down to the distros?
- einpoklum 7y ago
  FYI - Debian 10 Buster (and thus also Devuan 3) uses wget2 1.99.1, so not far behind.
- darnir 7y ago
  Debian packages the alpha version. Arch AUR has it as well. Other distros will follow soon
molticrystal 7y ago
I see a MingW64 build in the pipelines for continuous integration builds [1], but they only seem to be making Debian artifacts. Besides using wsl, are there any releases for officially anywhere, or are there any unofficial places? https://gitlab.com/gnuwget/wget2/pipelines/80747921 https://gitlab.com/gnuwget/wget2/pipelines/80747921

68 points by lelf 7y ago

I didn't even know that there was a Wget2…what's the development progress on this? Can (or should) I be using it instead of Wget?

msla 7y ago
It can download faster due to using more modern features (parallel downloading, If-Modified-Since) but the main focus is to make a cleaner codebase which is built around a reusable library, so it's easier to add features and make other code which uses those features, and so the libcurl monoculture gets a little fire under its ass.
- paulddraper 7y ago
  Does libcurl need a fire under its ass?
  - yjftsjthsd-h 7y ago
    Every monoculture does.
darnir 7y ago
Wget2 has been in development for a few years now. It is meant to be a drop in replacement for Wget with nearly full backward compatibility with all the old switches. In most cases you can replace wget with wget2 and not really realize the difference. For the known differences, please refer to the documentation in the gitlab repo. There's still a few rough edges though. And we very much welcome any criticisms / bug reports about the same.

msla 7y ago

> In many cases Wget2 downloads much faster than Wget1.x due to HTTP2, HTTP compression, parallel connections and use of If-Modified-Since HTTP header. This appears to be offical: https://gitlab.com/gnuwget/wget2 https://gitlab.com/gnuwget/wget2 Which has this wiki: https://gitlab.com/gnuwget/wget2/wikis/home https://gitlab.com/gnuwget/wget2/wikis/home > The idea is to have a fresh and maintainable codebase with features like multithreaded downloads, HTTP2, OCSP, HSTS, Metalink, IDNA2008, Public Suffix List, Multi-Proxies, Sitemaps, Atom/RSS Feeds, compression (gzip, deflate, lzma, bzip2), support for local filenames, etc. > Some of these feature have been built into Wget in the meantime, but some other are really hard to implement into the old codebase.

darnir 7y ago
Another feature not mentioned in there is support for verifying GPG signatures of binaries you download. If a gpg signature file exists, Wget2 will automatically verify the binary against that signature file
- rurban 7y ago
  With a .sig or. asc extension? gnu differs from gpg here.
  - darnir 7y ago
    Configurable. Default extension type is sig. However you can specify a list of extensions and it will attempt to verify against the first it finds.

chocolatkey 7y ago

It looks like the biggest new feature is support for http2. Will http3 be in wget2 or a "wget3" though when it eventually happens?

layoutIfNeeded 7y ago
Does it refuse to support plaintext HTTP2 like browsers do?
- darnir 7y ago
  Yes. It does not support plaintext http2.
darnir 7y ago
Nope. Wget2 was required since we wanted to break some backward compatibility with Wget 1.x releases. So we decided that a major version bump is warranted. It is also an entire rewrite of Wget in order to support HTTP/2. But this won't be required again when HTTP/3 or other protocols come out
- marvy 7y ago
  > we wanted to break some backward compatibility examples?

pram 7y ago

so will you explicitly need to specify 'wget2' in a package manager, or is this replacing wget. i certainly hope it's not the former!

chubot 7y ago
wget() { wget2 "$@"; } or alias wget=wget2
- johnday 7y ago
  Those are solutions for the shell, not for the package manager.
peterwwillis 7y ago
If they replaced 'wget', and you need HTTP2 support, you might be using the old 'wget' without HTTP2 support, or the new one with it. You would have to wrap all your commands with a version check, and ask people to install the newer, different software package, to get HTTP2 support. Or, call 'wget2' to get HTTP2 support. You can then either install both 'wget' and 'wget2', or the package maintainers could install a symlink for 'wget' backwards compatibility, so you'd only need to install the newer 'wget2' package. From the package maintainer's perspective, they're literally different software projects, and changing an established package name to be different software would break many systems.
- Fnoord 7y ago
  This could be solved with wget1 for Wget 1.x, and wget/wget2 binaries (linked) for Wget 2.x. Only thing that does is break scripts.
  - peterwwillis 7y ago
    > Only thing that does is break scripts That's not really an acceptable change... Would you like all the scripts you run on your Linux distros to break every time you do package updates? I wouldn't touch such a distro with a 100 foot pole

farisjarrah 7y ago

Anyone know if/when this will filter down to the distros?

einpoklum 7y ago
FYI - Debian 10 Buster (and thus also Devuan 3) uses wget2 1.99.1, so not far behind.
darnir 7y ago
Debian packages the alpha version. Arch AUR has it as well. Other distros will follow soon

molticrystal 7y ago

I see a MingW64 build in the pipelines for continuous integration builds [1], but they only seem to be making Debian artifacts. Besides using wsl, are there any releases for officially anywhere, or are there any unofficial places? https://gitlab.com/gnuwget/wget2/pipelines/80747921 https://gitlab.com/gnuwget/wget2/pipelines/80747921