6 ms·

Xdm – Powerfull download accelerator and video downloader

41 points by maydemir 5y ago

deleted 5y ago
[deleted]
charcircuit 5y ago
Software can't give you faster download speeds. If I get 10 Mbps down from my ISP I'm not going to somehow download at 50 Mbps. yt-dlp, steam, and firefox already max out my connection when they download something.
- buo 5y ago
  What they _might_ mean to say is that it gets around throttling at the source, probably by attempting to download multiple pieces in parallel.
- quesera 5y ago
  To nitpick: Software can give you faster effective download speeds. Compression is software. Parallelization across multiple hosts is achieved in software. It really depends on the cause of the bottleneck. Sometimes you need capabilities on both ends though. Also, while I'm nitpicking: "xdm" is and will always be the X Display Manager. :(
  - dsr_ 5y ago
    The biggest problem facing technology in the early 22nd century is that there are only 17,576 TLAs, and they're all overloaded already.
    - beagle3 5y ago
      That’s why we also use ETLAs
      - dspillett 5y ago
        I've ahead of the game, I jumped directly to FETLAs.
- RobertMiller 5y ago
  yt-dlp supports this sort of thing too.. > Multi-threaded fragment downloads: Download multiple fragments of m3u8/mpd videos in parallel. Use --concurrent-fragments (-N) option to set the number of threads used Not all sites are willing to max out your pipe with a single connection. Making multiple connections to those sites often gets around that.
- thomastjeffery 5y ago
  It can't get you faster than your ISP's throttling, but it can get you faster than the server you download from's throttling. I've been stuck with a throttled 10mbit fiber connection before, and still had times where sites would throttle me more than that. Now that I have a gigabit connection, it's a much more common problem.
- toyg 5y ago
  The bottleneck is typically at the server level, not the pipe. Steam uses BitTorrent, which is built to parallelize downloads - effectively implementing what download managers do. I think Firefox these days also does clever stuff behind the scenes. The result is downloads that complete faster than before, even though the theoretical max speed of any given connection is the same. Download managers are somewhat old-fashioned simply because modern software learnt from them and now tries their tricks out of the box.
  - charcircuit 5y ago
    >Steam uses BitTorrent, No, it downloads encrypted chunks from Steam's CDN.
- steerablesafe 5y ago
  Sometimes the bottleneck is on the server end, and often they throttle throughput per connection. Some download mangers allow you to download the same file through multiple connections and range request. Having said that I think it's a little bit rude to work around the intended throughput limitation.
_benj 5y ago
I used a lot of download managers and accelerators back in the dial-up times, they seemed like magic to me, plus being able to resume a download after somebody picked up the phone and dropped my internet connection was a plus! How is it that they work, that is the accelerator part? My guess is an algo to find the optimal amount of parallel downloads? Edit: also with parallel downloads, is it possible to do a HTTP GET with an bytes offset? With video I guess it’s possible to select what part to stream, but with regular http download I’m not sure
- charcircuit 5y ago
  You want to use the Range header. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Ra...
  - _benj 5y ago
    so that's how `wget -c` works! the more you know :)
    - eatonphil 5y ago
      Since flags are hard to google, what does `wget -c` do?
      - loeg 5y ago
        Resume partial download.
        eatonphil 5y ago
        Gotcha, thanks
      - elsherbini 5y ago
        My favorite way to find out what flags do is explainshell: e.g. https://explainshell.com/explain?cmd=wget+-c https://explainshell.com/explain?cmd=wget+-c
throw0101a 5y ago
So not X Display Manager? * https://en.wikipedia.org/wiki/XDM_(display_manager) https://en.wikipedia.org/wiki/XDM_(display_manager) * https://en.wikipedia.org/wiki/X_display_manager https://en.wikipedia.org/wiki/X_display_manager Not sure the Latin alphabet is wide enough for all the acronyms we need in recent years. :) Edit: reminds me of a story. Supposedly (!) the reason "Xkcd" is named so is because at the time of creation there were no Google results for that letter combination.
- moltke 5y ago
  "Unix is running out of two letter acronyms."
- calvinmorrison 5y ago
  I snagged a 4 letter domain name about 10 years ago and will never abandon it. Even if it's a .org
  - bryanrasmussen 5y ago
    hmm, a.org is incubating the future, good snag.
    - ASalazarMX 5y ago
      It's a shame that b.org has been domain squatted since 1993. Not even a placeholder website.
      - bryanrasmussen 5y ago
        I wonder how much it would cost to assimilate that site.
warmwaffles 5y ago
One tool I remember using a lot of was "DownThemAll". It was great to download quickly using a regex for files.
1MachineElf 5y ago
Together with the browser extension, I've successfully used it to download hundreds of videos. There are many other popular video downloader browser extensions, but often they contain spyware. Being open source gives me slightly more confidence in XDM.
- schleck8 5y ago
  How high are the chances it uses a yt-dlp backend?
  - jszymborski 5y ago
    It written in Java, so while not impossible, I'd say unlikely.
- Isthatablackgsd 5y ago
  I uses Video DownloadHelper (Firefox addon) with JDownloader2 (Java standalone app), I only use Video DownloadHelper to grab the video header and send it off to JD2 to do the download and conversion (via FFmpeg). The video platform that I couldn't find a way to download is JW Player. I tried various extensions (Firefox/Chrome) and standalone apps, they couldn't download the video stream from JW Player. The only thing they could grab is m3u/m2u8 file and that it. Nice to see there are another Java downloader.
steerablesafe 5y ago
I recommend aria2 for a similar purpose. Just `apt install aria2`. I'm just a happy user.
- jbourne 5y ago
  +1 for aria2! It does one job and does it very well. Want to add a UI? Take a look at WebUI-Aria2: https://github.com/ziahamza/webui-aria2 https://github.com/ziahamza/webui-aria2 It is also very easy to integrate with othersoftware via the RPC interface https://aria2.github.io/manual/en/html/aria2c.html?highlight=all%20proxy#rpc-interface https://aria2.github.io/manual/en/html/aria2c.html?highlight.... You can find wrappers in some languages e.g. the JS bindings: https://github.com/sonnyp/aria2.js https://github.com/sonnyp/aria2.js
Mathnerd314 5y ago
Last code commit in December 2020: https://github.com/subhra74/xdm/commit/953765940ae63107e8cfbf4925ad49294b8c470e https://github.com/subhra74/xdm/commit/953765940ae63107e8cfb... Apparently it is being rewritten from Java to .Net: https://github.com/subhra74/xdm/discussions/768 https://github.com/subhra74/xdm/discussions/768
lyton 5y ago
The UI has always seemed "off." I'll stick to Free Download Manager
gaws 5y ago
How does this compare to youtube-dl or yt-dlp?

41 points by maydemir 5y ago

deleted 5y ago

[deleted]

Software can't give you faster download speeds. If I get 10 Mbps down from my ISP I'm not going to somehow download at 50 Mbps. yt-dlp, steam, and firefox already max out my connection when they download something.

buo 5y ago
What they _might_ mean to say is that it gets around throttling at the source, probably by attempting to download multiple pieces in parallel.
quesera 5y ago
To nitpick: Software can give you faster effective download speeds. Compression is software. Parallelization across multiple hosts is achieved in software. It really depends on the cause of the bottleneck. Sometimes you need capabilities on both ends though. Also, while I'm nitpicking: "xdm" is and will always be the X Display Manager. :(
- dsr_ 5y ago
  The biggest problem facing technology in the early 22nd century is that there are only 17,576 TLAs, and they're all overloaded already.
  - beagle3 5y ago
    That’s why we also use ETLAs
    - dspillett 5y ago
      I've ahead of the game, I jumped directly to FETLAs.
RobertMiller 5y ago
yt-dlp supports this sort of thing too.. > Multi-threaded fragment downloads: Download multiple fragments of m3u8/mpd videos in parallel. Use --concurrent-fragments (-N) option to set the number of threads used Not all sites are willing to max out your pipe with a single connection. Making multiple connections to those sites often gets around that.
thomastjeffery 5y ago
It can't get you faster than your ISP's throttling, but it can get you faster than the server you download from's throttling. I've been stuck with a throttled 10mbit fiber connection before, and still had times where sites would throttle me more than that. Now that I have a gigabit connection, it's a much more common problem.
toyg 5y ago
The bottleneck is typically at the server level, not the pipe. Steam uses BitTorrent, which is built to parallelize downloads - effectively implementing what download managers do. I think Firefox these days also does clever stuff behind the scenes. The result is downloads that complete faster than before, even though the theoretical max speed of any given connection is the same. Download managers are somewhat old-fashioned simply because modern software learnt from them and now tries their tricks out of the box.
- charcircuit 5y ago
  >Steam uses BitTorrent, No, it downloads encrypted chunks from Steam's CDN.
steerablesafe 5y ago
Sometimes the bottleneck is on the server end, and often they throttle throughput per connection. Some download mangers allow you to download the same file through multiple connections and range request. Having said that I think it's a little bit rude to work around the intended throughput limitation.

_benj 5y ago

I used a lot of download managers and accelerators back in the dial-up times, they seemed like magic to me, plus being able to resume a download after somebody picked up the phone and dropped my internet connection was a plus! How is it that they work, that is the accelerator part? My guess is an algo to find the optimal amount of parallel downloads? Edit: also with parallel downloads, is it possible to do a HTTP GET with an bytes offset? With video I guess it’s possible to select what part to stream, but with regular http download I’m not sure

charcircuit 5y ago
You want to use the Range header. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Ra...
- _benj 5y ago
  so that's how `wget -c` works! the more you know :)
  - eatonphil 5y ago
    Since flags are hard to google, what does `wget -c` do?
    - loeg 5y ago
      Resume partial download.
      - eatonphil 5y ago
        Gotcha, thanks
    - elsherbini 5y ago
      My favorite way to find out what flags do is explainshell: e.g. https://explainshell.com/explain?cmd=wget+-c https://explainshell.com/explain?cmd=wget+-c

throw0101a 5y ago

So not X Display Manager? * https://en.wikipedia.org/wiki/XDM_(display_manager) https://en.wikipedia.org/wiki/XDM_(display_manager) * https://en.wikipedia.org/wiki/X_display_manager https://en.wikipedia.org/wiki/X_display_manager Not sure the Latin alphabet is wide enough for all the acronyms we need in recent years. :) Edit: reminds me of a story. Supposedly (!) the reason "Xkcd" is named so is because at the time of creation there were no Google results for that letter combination.

moltke 5y ago
"Unix is running out of two letter acronyms."
calvinmorrison 5y ago
I snagged a 4 letter domain name about 10 years ago and will never abandon it. Even if it's a .org
- bryanrasmussen 5y ago
  hmm, a.org is incubating the future, good snag.
  - ASalazarMX 5y ago
    It's a shame that b.org has been domain squatted since 1993. Not even a placeholder website.
    - bryanrasmussen 5y ago
      I wonder how much it would cost to assimilate that site.

warmwaffles 5y ago

One tool I remember using a lot of was "DownThemAll". It was great to download quickly using a regex for files.

1MachineElf 5y ago

Together with the browser extension, I've successfully used it to download hundreds of videos. There are many other popular video downloader browser extensions, but often they contain spyware. Being open source gives me slightly more confidence in XDM.

schleck8 5y ago
How high are the chances it uses a yt-dlp backend?
- jszymborski 5y ago
  It written in Java, so while not impossible, I'd say unlikely.
Isthatablackgsd 5y ago
I uses Video DownloadHelper (Firefox addon) with JDownloader2 (Java standalone app), I only use Video DownloadHelper to grab the video header and send it off to JD2 to do the download and conversion (via FFmpeg). The video platform that I couldn't find a way to download is JW Player. I tried various extensions (Firefox/Chrome) and standalone apps, they couldn't download the video stream from JW Player. The only thing they could grab is m3u/m2u8 file and that it. Nice to see there are another Java downloader.

steerablesafe 5y ago

I recommend aria2 for a similar purpose. Just `apt install aria2`. I'm just a happy user.

jbourne 5y ago
+1 for aria2! It does one job and does it very well. Want to add a UI? Take a look at WebUI-Aria2: https://github.com/ziahamza/webui-aria2 https://github.com/ziahamza/webui-aria2 It is also very easy to integrate with othersoftware via the RPC interface https://aria2.github.io/manual/en/html/aria2c.html?highlight=all%20proxy#rpc-interface https://aria2.github.io/manual/en/html/aria2c.html?highlight.... You can find wrappers in some languages e.g. the JS bindings: https://github.com/sonnyp/aria2.js https://github.com/sonnyp/aria2.js

Mathnerd314 5y ago

Last code commit in December 2020: https://github.com/subhra74/xdm/commit/953765940ae63107e8cfbf4925ad49294b8c470e https://github.com/subhra74/xdm/commit/953765940ae63107e8cfb... Apparently it is being rewritten from Java to .Net: https://github.com/subhra74/xdm/discussions/768 https://github.com/subhra74/xdm/discussions/768

lyton 5y ago

The UI has always seemed "off." I'll stick to Free Download Manager

gaws 5y ago

How does this compare to youtube-dl or yt-dlp?