3 ms·

Yes, and you can also download a million projects like this, the sandbox exists. But my suggestion would be that you build a video codec out of it. Preferably

Yes, and you can also download a million projects like this, the sandbox exists. But my suggestion would be that you build a video codec out of it. Preferably one that has the properties the market demands: performance and energy efficiency.

Dylan16807 3y ago
Not a codec, the only thing customized for this would be the container format. You'd use existing codecs, and the way you get good performance and energy efficiency on a video codec is by having a hardware implementation. Software decoding doesn't even come into that picture. As far as practical software decoding outside of battery-powered video, can I just point at webassembly? Especially the upcoming version with vector instructions. You could use normal webassembly, or even an extra-restricted version. It gets pretty good performance, and when you remove its ability to talk to the outside world it goes from pretty good to extremely good security.
- tinus_hn 3y ago
  The container format would be decoded by a sandboxed codec that can be found by decoding the container? WebAssembly codecs indeed exist, and they are impractical due to a lack in performance.
  - Dylan16807 3y ago
    > The container format would be decoded by a sandboxed codec that can be found by decoding the container? The container parser would not be dynamically downloaded, and may or may not be sandboxed. We don't need a new container with almost every codec. We just need the new codec itself. > WebAssembly codecs indeed exist, and they are impractical due to a lack in performance. Mostly because they don't have vector instructions yet, I bet. But plenty of webassembly is within 50% of native, which is good enough for lots of things, which includes image decoding for sure.
    - tinus_hn 3y ago
      So now container decoders have been magically vetted and secured so they don’t need the sandbox. Which is quite surprising considering most vulnerabilities in streams are in the container decoders and their multitude of hardly used features, but okay. The challenge remains for you to actually provide the codec you describe. Which a few comments ago was trivial because it was a hardware codec anyway, now it’s just a bit of WebAssembly away. Well that should be trivial because cross compilers to WebAssembly exist. So why don’t you just provide a few real world examples? Your probably not the first to think of these ideas, there has to be a reason why it hasn’t been done yet.
      - Dylan16807 3y ago
        > So now container decoders have been magically vetted and secured so they don’t need the sandbox. Not "magically". But you only need one or two, and they don't need to be very fast, so you can put a lot of effort into making them secure. But more importantly, browsers already have many container decoders. This is not an expansion in attack surface. The goal here is allowing a lot more codecs compared to current browsers without a significant increase in attack surface compared to current browsers. Pointing out flaws that already exist doesn't disqualify the idea. > So why don’t you just provide a few real world examples? Your probably not the first to think of these ideas, there has to be a reason why it hasn’t been done yet. Image decoders in webassembly already exist. Did you even look? Including JXL! Video decoding needs more support structure in the browser, and I already said some decoders need things that are being added to webassembly but aren't done yet. Even then, the first google result for "av1 webassembly" is a working decoder from five years ago.

Dylan16807 3y ago

Not a codec, the only thing customized for this would be the container format. You'd use existing codecs, and the way you get good performance and energy efficiency on a video codec is by having a hardware implementation. Software decoding doesn't even come into that picture. As far as practical software decoding outside of battery-powered video, can I just point at webassembly? Especially the upcoming version with vector instructions. You could use normal webassembly, or even an extra-restricted version. It gets pretty good performance, and when you remove its ability to talk to the outside world it goes from pretty good to extremely good security.

tinus_hn 3y ago
The container format would be decoded by a sandboxed codec that can be found by decoding the container? WebAssembly codecs indeed exist, and they are impractical due to a lack in performance.
- Dylan16807 3y ago
  > The container format would be decoded by a sandboxed codec that can be found by decoding the container? The container parser would not be dynamically downloaded, and may or may not be sandboxed. We don't need a new container with almost every codec. We just need the new codec itself. > WebAssembly codecs indeed exist, and they are impractical due to a lack in performance. Mostly because they don't have vector instructions yet, I bet. But plenty of webassembly is within 50% of native, which is good enough for lots of things, which includes image decoding for sure.
  - tinus_hn 3y ago
    So now container decoders have been magically vetted and secured so they don’t need the sandbox. Which is quite surprising considering most vulnerabilities in streams are in the container decoders and their multitude of hardly used features, but okay. The challenge remains for you to actually provide the codec you describe. Which a few comments ago was trivial because it was a hardware codec anyway, now it’s just a bit of WebAssembly away. Well that should be trivial because cross compilers to WebAssembly exist. So why don’t you just provide a few real world examples? Your probably not the first to think of these ideas, there has to be a reason why it hasn’t been done yet.
    - Dylan16807 3y ago
      > So now container decoders have been magically vetted and secured so they don’t need the sandbox. Not "magically". But you only need one or two, and they don't need to be very fast, so you can put a lot of effort into making them secure. But more importantly, browsers already have many container decoders. This is not an expansion in attack surface. The goal here is allowing a lot more codecs compared to current browsers without a significant increase in attack surface compared to current browsers. Pointing out flaws that already exist doesn't disqualify the idea. > So why don’t you just provide a few real world examples? Your probably not the first to think of these ideas, there has to be a reason why it hasn’t been done yet. Image decoders in webassembly already exist. Did you even look? Including JXL! Video decoding needs more support structure in the browser, and I already said some decoders need things that are being added to webassembly but aren't done yet. Even then, the first google result for "av1 webassembly" is a working decoder from five years ago.