6 ms·

The question I had is answered by this line of code: xhr.setRequestHeader("Range", "bytes=" + from + "-" + to); I am a little surprised you can just do that

by tlb 5y ago

The question I had is answered by this line of code: xhr.setRequestHeader("Range", "bytes=" + from + "-" + to); I am a little surprised you can just do that. In https://github.com/phiresky/sql.js-httpvfs/blob/master/src/lazyFile.ts https://github.com/phiresky/sql.js-httpvfs/blob/master/src/l...

crazygringo 5y ago
Not all webservers support/enable it, so YMMV. But as long as you're dealing with a known server that does, then gravy!
- altfredd 5y ago
  > Not all webservers support/enable it Could you provide an example of server that does not? AFAIK, Range is supported by all major CDNs, so not supporting it in web server would be a death knell for it's real-world adoption.
  - enneff 5y ago
    Depends on what’s being served. Any decent static file server should support it, but if the content is at all dynamically produced then the authors would have to think to implement it and rarely do.
  - crazygringo 5y ago
    I can't think of a specific one, but a decent proportion (maybe a quarter?) of HTTP downloads I attempt don't support resuming a partial download, therefore don't support Range. (I.e. resuming always starts from the beginning.) I would assume this is often because the site in question isn't using Apache etc. to serve a file directly, but is either essentially proxying it to some custom-built file serving service, or a script that processes/authenticates the file in some way, and they just never bothered to implement Range.
- stabbles 5y ago
  nginx enables it by default. Another ingenious use of range request is zsync, it allows you to diff compressed binaries on a remote with local ones, so that you only have to download what has changed on an update. AppImage uses this
yread 5y ago
I've learnt about this by using https://www.biodalliance.org https://www.biodalliance.org It's an embedded genome viewer, you can just point it at a multigigabyte reference files and .seg files and it loads super quick
- punnerud 5y ago
  Here is direct link to GitHub with the usage: https://github.com/dasmoth/dalliance/search?q=bytes%3D%27&type= https://github.com/dasmoth/dalliance/search?q=bytes%3D%27&ty...
dathinab 5y ago
Range headers are a pretty standard tools to e.g. continue interrupted downloads and similar. Any well designed system, especially if it has static sources and is server cached should support it. Surprisingly many web-frameworks don't support it out of the box, or don't support it well. Either way gh-pages are static content and probably with some server side regional caches, so I'm not surprised it works.
- hinkley 5y ago
  Range headers are also how DownloadThemAll works. If you're pulling a single TCP stream across a crowded network, you get maybe 1/40th of the available bandwidth. If you do four range requests instead, you might see >3/40th of the bandwidth. This place we were contracting at, the managers were widely rumored to stream sports games at their desks, and our release cycle happened to fall on a game day. My poor coworker was downloading our installer every week, and the ETA was over 40 minutes. "Are you using DTA?" "No, what's that?" <fiddle fiddle> ETA: 12 minutes. 12 minute pauses in the middle of a manual process are a lot easier to stomach than 40 minutes. Especially if something goes wrong and you have to do it twice.
fanf2 5y ago
One of the heaviest users of range requests is (or was) the Adobe Acrobat PDF plugin.
- iudqnolq 5y ago
  I'm surprised that works, iirc pdf isn't defined in order and can't be parsed streaming
  - kindall 5y ago
    I think that's what the "optimize for Web" checkbox does
    - IshKebab 5y ago
      Wow so it actually does something! I wish programs would use such vague descriptions. (Or more of them had helpful instant tooltips.)
      - maskros 5y ago
        Linearized (a.k.a. web optimized) PDF files only help for displaying the first page quickly. The rest of the file is still in pretty much random access order.
        e12e 5y ago
        For what it's worth, there's a format for content like pdfs that's optimized for viewing on screens (as opposed to printing), djvu: https://en.m.wikipedia.org/wiki/DjVu https://en.m.wikipedia.org/wiki/DjVu Oh, wow initial release 1998,now I'm feeling a bit old...
  - tonyedgecombe 5y ago
    Linearised PDF files are defined in order and can be read from the start. https://blog.idrsolutions.com/2010/02/linearized-pdf-files/ https://blog.idrsolutions.com/2010/02/linearized-pdf-files/
e12e 5y ago
I guess "all" that's needed for write would be a webdav server with support for PATCH x-update-range? :) https://tools.ietf.org/html/rfc5789 https://tools.ietf.org/html/rfc5789 https://sabre.io/dav/http-patch/ https://sabre.io/dav/http-patch/ Unfortunately, solid, stand-alone webdav servers are harder to come by than decent http2/1.1 servers.

by tlb 5y ago

crazygringo 5y ago

Not all webservers support/enable it, so YMMV. But as long as you're dealing with a known server that does, then gravy!

altfredd 5y ago
> Not all webservers support/enable it Could you provide an example of server that does not? AFAIK, Range is supported by all major CDNs, so not supporting it in web server would be a death knell for it's real-world adoption.
- enneff 5y ago
  Depends on what’s being served. Any decent static file server should support it, but if the content is at all dynamically produced then the authors would have to think to implement it and rarely do.
- crazygringo 5y ago
  I can't think of a specific one, but a decent proportion (maybe a quarter?) of HTTP downloads I attempt don't support resuming a partial download, therefore don't support Range. (I.e. resuming always starts from the beginning.) I would assume this is often because the site in question isn't using Apache etc. to serve a file directly, but is either essentially proxying it to some custom-built file serving service, or a script that processes/authenticates the file in some way, and they just never bothered to implement Range.
stabbles 5y ago
nginx enables it by default. Another ingenious use of range request is zsync, it allows you to diff compressed binaries on a remote with local ones, so that you only have to download what has changed on an update. AppImage uses this

yread 5y ago

I've learnt about this by using https://www.biodalliance.org https://www.biodalliance.org It's an embedded genome viewer, you can just point it at a multigigabyte reference files and .seg files and it loads super quick

punnerud 5y ago
Here is direct link to GitHub with the usage: https://github.com/dasmoth/dalliance/search?q=bytes%3D%27&type= https://github.com/dasmoth/dalliance/search?q=bytes%3D%27&ty...

dathinab 5y ago

Range headers are a pretty standard tools to e.g. continue interrupted downloads and similar. Any well designed system, especially if it has static sources and is server cached should support it. Surprisingly many web-frameworks don't support it out of the box, or don't support it well. Either way gh-pages are static content and probably with some server side regional caches, so I'm not surprised it works.

hinkley 5y ago
Range headers are also how DownloadThemAll works. If you're pulling a single TCP stream across a crowded network, you get maybe 1/40th of the available bandwidth. If you do four range requests instead, you might see >3/40th of the bandwidth. This place we were contracting at, the managers were widely rumored to stream sports games at their desks, and our release cycle happened to fall on a game day. My poor coworker was downloading our installer every week, and the ETA was over 40 minutes. "Are you using DTA?" "No, what's that?" <fiddle fiddle> ETA: 12 minutes. 12 minute pauses in the middle of a manual process are a lot easier to stomach than 40 minutes. Especially if something goes wrong and you have to do it twice.

fanf2 5y ago

One of the heaviest users of range requests is (or was) the Adobe Acrobat PDF plugin.

iudqnolq 5y ago
I'm surprised that works, iirc pdf isn't defined in order and can't be parsed streaming
- kindall 5y ago
  I think that's what the "optimize for Web" checkbox does
  - IshKebab 5y ago
    Wow so it actually does something! I wish programs would use such vague descriptions. (Or more of them had helpful instant tooltips.)
    - maskros 5y ago
      Linearized (a.k.a. web optimized) PDF files only help for displaying the first page quickly. The rest of the file is still in pretty much random access order.
      - e12e 5y ago
        For what it's worth, there's a format for content like pdfs that's optimized for viewing on screens (as opposed to printing), djvu: https://en.m.wikipedia.org/wiki/DjVu https://en.m.wikipedia.org/wiki/DjVu Oh, wow initial release 1998,now I'm feeling a bit old...
- tonyedgecombe 5y ago
  Linearised PDF files are defined in order and can be read from the start. https://blog.idrsolutions.com/2010/02/linearized-pdf-files/ https://blog.idrsolutions.com/2010/02/linearized-pdf-files/

e12e 5y ago

I guess "all" that's needed for write would be a webdav server with support for PATCH x-update-range? :) https://tools.ietf.org/html/rfc5789 https://tools.ietf.org/html/rfc5789 https://sabre.io/dav/http-patch/ https://sabre.io/dav/http-patch/ Unfortunately, solid, stand-alone webdav servers are harder to come by than decent http2/1.1 servers.