3 ms·

The fact that it needs a server at all seems unnecessary. It's all written in JavaScript, and isn't doing anything that couldn't be done in a browser, I see no

by Mogzol 4y ago

The fact that it needs a server at all seems unnecessary. It's all written in JavaScript, and isn't doing anything that couldn't be done in a browser, I see no reason why this can't be an entirely client-side application.

mholt 4y ago
Processing multi-GB files in the browser is... fun. Doing that kind of thing on a server is easier. *I'm not justifying doing it on the server, especially for an application like this where yes: it can be done in the client.* But I do sympathize because I know from experience why it's easier to do it server-side, without any conspiracies. I wrote Papa Parse[0] about 10 years ago, and back then at least, it was extremely difficult to stream large files in an efficient, reliable way. Web Workers make things slightly better, but there's so many issues with large-scale local compute in a browser tab. A few examples: - https://stackoverflow.com/questions/24708649/why-does-web-worker-performance-sharply-decline-after-30-seconds https://stackoverflow.com/questions/24708649/why-does-web-wo... (the answer actually came from Google+ which is still linked to, but no longer available; fortunately I summarized it in my post) - https://stackoverflow.com/questions/27081858/how-can-i-make-a-really-long-string-using-indexeddb-without-crashing-the-browser https://stackoverflow.com/questions/27081858/how-can-i-make-... You get deep enough into the weeds and eventually you realize you can make it work cross-browser if you know which browser you're using (YES, User-Agent does matter for things like this) and call you crazy for trying to find out: - https://stackoverflow.com/questions/27084036/how-can-i-reliably-detect-the-browser-without-using-window-navigator https://stackoverflow.com/questions/27084036/how-can-i-relia... Despite all this, I *100%* agree and local-only processing is also a hard-rule for me as well. (That's why JSON-to-Go[1] does it all client-side. `go fmt` event compiles to WASM and runs in the browser!) [0]: https://www.papaparse.com/ https://www.papaparse.com/ [1]: https://mholt.github.io/json-to-go/ https://mholt.github.io/json-to-go/
- rob74 4y ago
  Ok, I think I get what you're saying - this is a VS Code extension, but because VS Code is an Electron application, it's still "running in a browser"?
- simplotek 4y ago
  > Processing multi-GB files in the browser is... fun. Doing that kind of thing on a server is easier. This sounds like a strawman. Not everyone wrangles multi-GB files, let alone JSON documents. Those who do are already readily aware of the implications. I mean,some popular text editors even struggle with multi-GB of plain text files. You don't need a server to handle JSON. There is no excuse.
  - hutzlibu 4y ago
    "You don't need a server to handle JSON. There is no excuse." No technical excuse, but lots of buisness reasons I guess.

by Mogzol 4y ago

mholt 4y ago

Processing multi-GB files in the browser is... fun. Doing that kind of thing on a server is easier. *I'm not justifying doing it on the server, especially for an application like this where yes: it can be done in the client.* But I do sympathize because I know from experience why it's easier to do it server-side, without any conspiracies. I wrote Papa Parse[0] about 10 years ago, and back then at least, it was extremely difficult to stream large files in an efficient, reliable way. Web Workers make things slightly better, but there's so many issues with large-scale local compute in a browser tab. A few examples: - https://stackoverflow.com/questions/24708649/why-does-web-worker-performance-sharply-decline-after-30-seconds https://stackoverflow.com/questions/24708649/why-does-web-wo... (the answer actually came from Google+ which is still linked to, but no longer available; fortunately I summarized it in my post) - https://stackoverflow.com/questions/27081858/how-can-i-make-a-really-long-string-using-indexeddb-without-crashing-the-browser https://stackoverflow.com/questions/27081858/how-can-i-make-... You get deep enough into the weeds and eventually you realize you can make it work cross-browser if you know which browser you're using (YES, User-Agent does matter for things like this) and call you crazy for trying to find out: - https://stackoverflow.com/questions/27084036/how-can-i-reliably-detect-the-browser-without-using-window-navigator https://stackoverflow.com/questions/27084036/how-can-i-relia... Despite all this, I *100%* agree and local-only processing is also a hard-rule for me as well. (That's why JSON-to-Go[1] does it all client-side. `go fmt` event compiles to WASM and runs in the browser!) [0]: https://www.papaparse.com/ https://www.papaparse.com/ [1]: https://mholt.github.io/json-to-go/ https://mholt.github.io/json-to-go/

rob74 4y ago
Ok, I think I get what you're saying - this is a VS Code extension, but because VS Code is an Electron application, it's still "running in a browser"?
simplotek 4y ago
> Processing multi-GB files in the browser is... fun. Doing that kind of thing on a server is easier. This sounds like a strawman. Not everyone wrangles multi-GB files, let alone JSON documents. Those who do are already readily aware of the implications. I mean,some popular text editors even struggle with multi-GB of plain text files. You don't need a server to handle JSON. There is no excuse.
- hutzlibu 4y ago
  "You don't need a server to handle JSON. There is no excuse." No technical excuse, but lots of buisness reasons I guess.