4 ms·

Webassembly is going to be fantastic for building a lot of similar apps like this that avoid the server. It also keeps costs of the developer to a minimum becau

Webassembly is going to be fantastic for building a lot of similar apps like this that avoid the server. It also keeps costs of the developer to a minimum because huge data files are not being transferred to/fro.

josefrichter 5y ago
I’ve actually done this 10 years ago with HTML canvas - resizing images on client side before upload, to save bandwidth and speed everything up: https://github.com/josefrichter/resize https://github.com/josefrichter/resize
- zamber 5y ago
  This uses WebAssembly so it can preserve PNG transparency for example, which the canvas method loses.
  - josefrichter 5y ago
    yes of course, wasm is far superior. we are 10 years farther, after all. it just reminded me that the problem of resizing before upload exists for at least a decade, even solutions exist for at least a decade, yet it’s still rarely being solved. maybe it’s not the biggest problem, so devs focus on other stuff.
ramraj07 5y ago
I want to make an app that allows folks to upload an excel file and I want to extract just the top 10k rows of the first two columns to plot them. I’m afraid of allowing an upload option for a free service, can I parse an xls file using wasm and avoid uploading anything?
- akmittal 5y ago
  Technically that's possible, you can compile any C, Rust or Go Excel parcer and use that.
- zamber 5y ago
  For client-side stuff like that, there is sheetjs which has support for XLSX but there are file size limitations. XLSX is a glorified zip file and to parse it you 1st have to unzip it and then build the XLSX file. I found that it starts to hang on XLSX files ~50MB because the spreadsheet model it produces get's a bit too big and sheetjs dies. Doing this in a web worker is a good idea because otherwise it will clog the main thread. For CSV the story is different, because you can read a plain text file in chunks (even in JS) with FileReader and readAsArrayBuffer(file.slice(byteOffset, byteOffset + chunkSize, file.type)).
deleted 5y ago
[deleted]

josefrichter 5y ago

I’ve actually done this 10 years ago with HTML canvas - resizing images on client side before upload, to save bandwidth and speed everything up: https://github.com/josefrichter/resize https://github.com/josefrichter/resize

zamber 5y ago
This uses WebAssembly so it can preserve PNG transparency for example, which the canvas method loses.
- josefrichter 5y ago
  yes of course, wasm is far superior. we are 10 years farther, after all. it just reminded me that the problem of resizing before upload exists for at least a decade, even solutions exist for at least a decade, yet it’s still rarely being solved. maybe it’s not the biggest problem, so devs focus on other stuff.

ramraj07 5y ago

I want to make an app that allows folks to upload an excel file and I want to extract just the top 10k rows of the first two columns to plot them. I’m afraid of allowing an upload option for a free service, can I parse an xls file using wasm and avoid uploading anything?

akmittal 5y ago
Technically that's possible, you can compile any C, Rust or Go Excel parcer and use that.
zamber 5y ago
For client-side stuff like that, there is sheetjs which has support for XLSX but there are file size limitations. XLSX is a glorified zip file and to parse it you 1st have to unzip it and then build the XLSX file. I found that it starts to hang on XLSX files ~50MB because the spreadsheet model it produces get's a bit too big and sheetjs dies. Doing this in a web worker is a good idea because otherwise it will clog the main thread. For CSV the story is different, because you can read a plain text file in chunks (even in JS) with FileReader and readAsArrayBuffer(file.slice(byteOffset, byteOffset + chunkSize, file.type)).

deleted 5y ago

[deleted]