4 ms·

I once tried sending a 3 GB file through it, then kept wondering why my entire system became so sluggish. Turns out it loads the entire thing into RAM... The fi

I once tried sending a 3 GB file through it, then kept wondering why my entire system became so sluggish. Turns out it loads the entire thing into RAM... The file didn't go through either. I hope this one isn't like that.

f1refly 7y ago
Same experience here. Every website I tried so far died when sending large files because it apparently loaded the whole thing into ram and then some. That might be fine on a system with 32gb memory, but my laptop with 8gb dies when trying to send a 7gb file.
h-cobordism 7y ago
Does HTML5 even let you read a file without loading it all into memory? Edit: Looks like this is it. https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream https://developer.mozilla.org/en-US/docs/Web/API/ReadableStr... Edit 2: And yes, this is using it. https://github.com/saljam/webwormhole/blob/master/web/main.js#L96-L105 https://github.com/saljam/webwormhole/blob/master/web/main.j...
- anderspitman 7y ago
  You can also do this manually (and on older browsers) by creating a FileReader and only loading new chunks after old ones have been transferred. With async APIs like WebSockets and WebRTC this typically requires implementing your own backpressure to avoid blowing up browser memory. See for example how omnistreams does it[0]. [0]: https://github.com/omnistreams/omnistreams-filereader-js/blob/master/index.mjs https://github.com/omnistreams/omnistreams-filereader-js/blo...
- lxgr 7y ago
  That's great! Inconsistent large file support has also been what has prevented me from using the various predecessors to this so far.
- lxgr 7y ago
  Hm, apparently what you found is something else (i.e. https://github.com/saljam/webwormhole/issues/5 https://github.com/saljam/webwormhole/issues/5). There is still an open bug for large file transfers: https://github.com/saljam/webwormhole/issues/4 https://github.com/saljam/webwormhole/issues/4
mr_toad 7y ago
The JavaScript implementation is creating a blob with a URL pointing to it so the user can save the file. I might be wrong, but I think that the all the data is in browser memory before it is saved. Browsers are pretty restrictive about writing to the file system.

f1refly 7y ago

Same experience here. Every website I tried so far died when sending large files because it apparently loaded the whole thing into ram and then some. That might be fine on a system with 32gb memory, but my laptop with 8gb dies when trying to send a 7gb file.

h-cobordism 7y ago

Does HTML5 even let you read a file without loading it all into memory? Edit: Looks like this is it. https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream https://developer.mozilla.org/en-US/docs/Web/API/ReadableStr... Edit 2: And yes, this is using it. https://github.com/saljam/webwormhole/blob/master/web/main.js#L96-L105 https://github.com/saljam/webwormhole/blob/master/web/main.j...

anderspitman 7y ago
You can also do this manually (and on older browsers) by creating a FileReader and only loading new chunks after old ones have been transferred. With async APIs like WebSockets and WebRTC this typically requires implementing your own backpressure to avoid blowing up browser memory. See for example how omnistreams does it[0]. [0]: https://github.com/omnistreams/omnistreams-filereader-js/blob/master/index.mjs https://github.com/omnistreams/omnistreams-filereader-js/blo...
lxgr 7y ago
That's great! Inconsistent large file support has also been what has prevented me from using the various predecessors to this so far.
lxgr 7y ago
Hm, apparently what you found is something else (i.e. https://github.com/saljam/webwormhole/issues/5 https://github.com/saljam/webwormhole/issues/5). There is still an open bug for large file transfers: https://github.com/saljam/webwormhole/issues/4 https://github.com/saljam/webwormhole/issues/4

mr_toad 7y ago

The JavaScript implementation is creating a blob with a URL pointing to it so the user can save the file. I might be wrong, but I think that the all the data is in browser memory before it is saved. Browsers are pretty restrictive about writing to the file system.