10 ms·

Blink-dev – Intent to Ship: SharedArrayBuffer

113 points by okket 10y ago

franciscop 10y ago
I am not sure I like some of the new changes. It seems to me like JS is expanding both towards an upper level language and a lower level language, which frankly makes it look more like a Frankenlanguage. I guess it's the limitations of having a single real language to be used in the browsers. Edit: just check this tutorial. I'd venture to guess to 99% of JS developers it sounds totally bizarre: https://github.com/tc39/ecmascript_sharedmem/blob/master/TUTORIAL.md https://github.com/tc39/ecmascript_sharedmem/blob/master/TUT...
- pmalynin 10y ago
  You mean like Python? I agree.
  - tdb7893 10y ago
    Has python gotten much more low level recently?
    - brianwawok 10y ago
      I would say async is a little bit lower level, but not at the atomic array level.
    - icebraining 10y ago
      import ctypes from multiprocessing import Array mysharedarray = Array(ctypes.<type>, 100) That's essentially the same as this proposal.
  - franciscop 10y ago
    Added a tutorial on SharedArrayBuffer with a lot of other low level JS stuff that looks like a total different language.
- zensavona 10y ago
  But that is exactly what is needed. JavaScript is essentially a becoming a ~native~ language (like C, C++, Java) which is used to implement all kinds of abstractions. Think of C++ for example. It's used to implement frameworks like JUCE, used for high-level application logic and UIs, but underneath that there are a lot of optimised abstractions. Some of these abstractions can only be so good or so deep until the language they are implemented in can get closer to the "metal" (or in this instance, VM). I for one welcome these new additions to the ECMAscript spec.
  - franciscop 10y ago
    But not really, Javascript has to be interpreted/JITed, so it will never be in the same level as C/C++. IMO it'd be way better to: - Introduce real native types, not complex-named functions that are way more bothersome to work with than types. - Use WASM or something similar to provide for a low level variant.
    - foota 10y ago
      I believe the issue is in building high performance types that are both secure and that fit in with the rest of js.
    - Dylan16807 10y ago
      JIT can be faster than static compilation.
  - vvanders 10y ago
    > ... ~native~ language (like C, C++, Java) One of those is not like the others.
    - fulafel 10y ago
      C++ was OS-native on some older now extinct systems, like Symbian and BeOS.
      - 10y ago
- rtpg 10y ago
  I think the idea is for these things (much like TypedArray and the like) to be used by "compile-to-JS" languages, rather than to be used directly too often.
  - johncolanduoni 10y ago
    Making them accessible from WASM is also likely to happen in the future, as they're perfect for running truly multithreaded C/C++, Rust, etc. code.
- tonyedgecombe 10y ago
  We certainly don't seem to know when to stop extending software.
bhouston 10y ago
This is great. It opens up a lot of possibilities for our online 3D editor - background worker threads that each work on a segment of an array. Now if only Node.JS could support worker threads at all.
- rogerwang 10y ago
  In NW.js you can use Node.js in Web Worker: https://nwjs.io/blog/v0.18.4/ https://nwjs.io/blog/v0.18.4/ . HTH.
  - tuananh 10y ago
    why is it available in NW.js but not Node?
    - rogerwang 10y ago
      I think Node community is working on it. In NW the implementation of webworker from browser can be leveraged.
ic4l 10y ago
This is GREAT! Transferable's didn't make sense in a few situations.
caleblloyd 10y ago
The main benefit here will be realized in Shared Workers. Unfortunately Edge support is "not planned" and WebKit support is "removed" http://caniuse.com/#feat=sharedworkers http://caniuse.com/#feat=sharedworkers I guess it may come in handy for Electron apps.
- chrismorgan 10y ago
  You don’t need Shared Workers to benefit from this. I haven’t ever actually used Shared Workers, but as I understand it the normal sort of case for them could be polyfilled without too much difficulty. Using Web Workers instead of Shared Workers just means that any actual communication beyond the shared memory must be posted via the instantiator of the worker.
  - streptomycin 10y ago
    You're right about Shared Workers not being needed for this, but just to continue going off on a tangent... I think a polyfill would be pretty difficult. First you have to make sure only one tab starts a Web Worker. Maybe that's not impossible, but it's tricky. And then the real difficulty... when the tab that spawned the Web Worker closes while another tab is open, you'd need to spawn another Web Worker in that tab and copy the state of the first Web Worker to the second Web Worker before the first one is destroyed by the browser. Not sure if that's possible.
    - the8472 10y ago
      You can use broadcastchannel to coordinate multiple tabs and workers. And I think once you receive a message from such a channel you get a proxy for the sender, which means you can establish a 1-1 channel, which in turn means you can can transfer objects like sharedarraybuffer.
- deleted 10y ago
  [deleted]
Sephr 10y ago
If Chrome is going to ship SharedArrayBuffer, then why will the resolution of performance.now() and other timer APIs continue to be neutered? I don't have any preference one way or the other for implementing SharedArrayBuffer, but I am fully against neutering timers with jitter if a vendor does choose to implement SharedArrayBuffer. Random additional timer jitter becomes a false layer of security in this scenario, and https://bugs.chromium.org/p/chromium/issues/detail?id=506723 https://bugs.chromium.org/p/chromium/issues/detail?id=506723 should be reverted. With SharedArrayBuffer you can easily implement extremely high resolution sub-nanosecond timers and open the door to new timing attacks. For example, here is a high resolution SharedArrayBuffer timer: https://codepen.io/yossioren/pen/LVBNMR https://codepen.io/yossioren/pen/LVBNMR For more discussion regarding the security implications of this change, see https://github.com/tc39/ecmascript_sharedmem/issues/1 https://github.com/tc39/ecmascript_sharedmem/issues/1 . Some of the security problems may be mitigable with Intel CAT, although I am not aware of Chrome making use of that yet.
- skrebbel 10y ago
  Is there any layman's introduction to this subject? I can't comprehend how having an accurate timer lets me steal information from other processes.
  - Sephr 10y ago
    The papers "The Spy in the Sandbox -- Practical Cache Attacks in Javascript" at https://arxiv.org/pdf/1502.07373.pdf https://arxiv.org/pdf/1502.07373.pdf and "ASLR on the Line: Practical Cache Attacks on the MMU" at http://www.cs.vu.nl//~herbertb/download/papers/anc_ndss17.pdf http://www.cs.vu.nl//~herbertb/download/papers/anc_ndss17.pd... are good starting points, though they are not very accessible to a layman. There have even been timing attacks that let you recover AES keys from analyzing a device's power usage. In layman's terms: computers are extremely complicated beasts of coordinated timing of software and hardware, and timing data leaks many things about what a piece of software is doing. This is the reason that many modern encryption libraries pad operations as a defense against timing attacks.
    - 10y ago
riquito 10y ago
I've just stumpled once again on atob/btoa not working with characters like or 𝒞 and here I read about SharedArrayBuffers, which is way a greater goal. Of course they're right to work on such projects, go for it, but I can't avoid to be baffled by the fact that (apparently) more mundane operations don't receive love. Other examples would be MD5 removed from subtlecrypto, killing any way to speak with services that use it for non-cryptographic operations, or the missing functions to encode/decode hex strings, which maybe are not everyday use cases but are provided by most languages and would need an afternoon to write with tests. I want a better standard library. Just my discouraged 5 cents.
- ygra 10y ago
  atob/btoa is atrocious from an API design standpoint. Here we are, designing an API to convert things to Base64 and back and we use a string as a byte array substitute, nevermind the fact that strings are not limited to Latin-1. There's probably a bunch of libraries that solve this, but stumbling over this in a library that has no outside dependencies is a bit annoying since you have to solve it yourself ... again.
- IIIIIIIIIIII 10y ago
  I'm just curious since I never used this feature, are you talking about this issue? > The "Unicode Problem" > Since DOMStrings are 16-bit-encoded strings, in most browsers calling window.btoa on a Unicode string will cause a Character Out Of Range exception if a character exceeds the range of a 8-bit ASCII-encoded character. There are two possible methods to solve this problem: https://developer.mozilla.org/en/docs/Web/API/WindowBase64/Base64_encoding_and_decoding#The_.22Unicode_Problem.22 https://developer.mozilla.org/en/docs/Web/API/WindowBase64/B...
  - riquito 10y ago
    Yes, that one
dom0 10y ago
> Gives JavaScript access to shared memory, atomics, and futex. Holy fuck, this is going to be bad, introducing a whole continent of new bugs into JavaScript land. And also giving JS even more and even better options to spy and attack other applications on the same computer.
- Matthias247 10y ago
  Yes, it introduces a whole new range of multithreading issues also to JS land. However I think threading issues won't be as common as in C++/Java/C# land, since the only thing that will be shared between threads/workers is byte arrays -> There's no shared access to other Javascript objects. As most normal JS developers won't be dealing with plain byte arrays the potential for misuse is not too high.
bricss 10y ago
SharedArrayBuffer and Atomics is already in Chrome, unluckily behind the flag.

113 points by okket 10y ago

I am not sure I like some of the new changes. It seems to me like JS is expanding both towards an upper level language and a lower level language, which frankly makes it look more like a Frankenlanguage. I guess it's the limitations of having a single real language to be used in the browsers. Edit: just check this tutorial. I'd venture to guess to 99% of JS developers it sounds totally bizarre: https://github.com/tc39/ecmascript_sharedmem/blob/master/TUTORIAL.md https://github.com/tc39/ecmascript_sharedmem/blob/master/TUT...

pmalynin 10y ago
You mean like Python? I agree.
- tdb7893 10y ago
  Has python gotten much more low level recently?
  - brianwawok 10y ago
    I would say async is a little bit lower level, but not at the atomic array level.
  - icebraining 10y ago
    import ctypes from multiprocessing import Array mysharedarray = Array(ctypes.<type>, 100) That's essentially the same as this proposal.
- franciscop 10y ago
  Added a tutorial on SharedArrayBuffer with a lot of other low level JS stuff that looks like a total different language.
zensavona 10y ago
But that is exactly what is needed. JavaScript is essentially a becoming a ~native~ language (like C, C++, Java) which is used to implement all kinds of abstractions. Think of C++ for example. It's used to implement frameworks like JUCE, used for high-level application logic and UIs, but underneath that there are a lot of optimised abstractions. Some of these abstractions can only be so good or so deep until the language they are implemented in can get closer to the "metal" (or in this instance, VM). I for one welcome these new additions to the ECMAscript spec.
- franciscop 10y ago
  But not really, Javascript has to be interpreted/JITed, so it will never be in the same level as C/C++. IMO it'd be way better to: - Introduce real native types, not complex-named functions that are way more bothersome to work with than types. - Use WASM or something similar to provide for a low level variant.
  - foota 10y ago
    I believe the issue is in building high performance types that are both secure and that fit in with the rest of js.
  - Dylan16807 10y ago
    JIT can be faster than static compilation.
- vvanders 10y ago
  > ... ~native~ language (like C, C++, Java) One of those is not like the others.
  - fulafel 10y ago
    C++ was OS-native on some older now extinct systems, like Symbian and BeOS.
    - 10y ago
rtpg 10y ago
I think the idea is for these things (much like TypedArray and the like) to be used by "compile-to-JS" languages, rather than to be used directly too often.
- johncolanduoni 10y ago
  Making them accessible from WASM is also likely to happen in the future, as they're perfect for running truly multithreaded C/C++, Rust, etc. code.
tonyedgecombe 10y ago
We certainly don't seem to know when to stop extending software.

bhouston 10y ago

This is great. It opens up a lot of possibilities for our online 3D editor - background worker threads that each work on a segment of an array. Now if only Node.JS could support worker threads at all.

rogerwang 10y ago
In NW.js you can use Node.js in Web Worker: https://nwjs.io/blog/v0.18.4/ https://nwjs.io/blog/v0.18.4/ . HTH.
- tuananh 10y ago
  why is it available in NW.js but not Node?
  - rogerwang 10y ago
    I think Node community is working on it. In NW the implementation of webworker from browser can be leveraged.

ic4l 10y ago

This is GREAT! Transferable's didn't make sense in a few situations.

caleblloyd 10y ago

The main benefit here will be realized in Shared Workers. Unfortunately Edge support is "not planned" and WebKit support is "removed" http://caniuse.com/#feat=sharedworkers http://caniuse.com/#feat=sharedworkers I guess it may come in handy for Electron apps.

chrismorgan 10y ago
You don’t need Shared Workers to benefit from this. I haven’t ever actually used Shared Workers, but as I understand it the normal sort of case for them could be polyfilled without too much difficulty. Using Web Workers instead of Shared Workers just means that any actual communication beyond the shared memory must be posted via the instantiator of the worker.
- streptomycin 10y ago
  You're right about Shared Workers not being needed for this, but just to continue going off on a tangent... I think a polyfill would be pretty difficult. First you have to make sure only one tab starts a Web Worker. Maybe that's not impossible, but it's tricky. And then the real difficulty... when the tab that spawned the Web Worker closes while another tab is open, you'd need to spawn another Web Worker in that tab and copy the state of the first Web Worker to the second Web Worker before the first one is destroyed by the browser. Not sure if that's possible.
  - the8472 10y ago
    You can use broadcastchannel to coordinate multiple tabs and workers. And I think once you receive a message from such a channel you get a proxy for the sender, which means you can establish a 1-1 channel, which in turn means you can can transfer objects like sharedarraybuffer.
deleted 10y ago
[deleted]

Sephr 10y ago

If Chrome is going to ship SharedArrayBuffer, then why will the resolution of performance.now() and other timer APIs continue to be neutered? I don't have any preference one way or the other for implementing SharedArrayBuffer, but I am fully against neutering timers with jitter if a vendor does choose to implement SharedArrayBuffer. Random additional timer jitter becomes a false layer of security in this scenario, and https://bugs.chromium.org/p/chromium/issues/detail?id=506723 https://bugs.chromium.org/p/chromium/issues/detail?id=506723 should be reverted. With SharedArrayBuffer you can easily implement extremely high resolution sub-nanosecond timers and open the door to new timing attacks. For example, here is a high resolution SharedArrayBuffer timer: https://codepen.io/yossioren/pen/LVBNMR https://codepen.io/yossioren/pen/LVBNMR For more discussion regarding the security implications of this change, see https://github.com/tc39/ecmascript_sharedmem/issues/1 https://github.com/tc39/ecmascript_sharedmem/issues/1 . Some of the security problems may be mitigable with Intel CAT, although I am not aware of Chrome making use of that yet.

skrebbel 10y ago
Is there any layman's introduction to this subject? I can't comprehend how having an accurate timer lets me steal information from other processes.
- Sephr 10y ago
  The papers "The Spy in the Sandbox -- Practical Cache Attacks in Javascript" at https://arxiv.org/pdf/1502.07373.pdf https://arxiv.org/pdf/1502.07373.pdf and "ASLR on the Line: Practical Cache Attacks on the MMU" at http://www.cs.vu.nl//~herbertb/download/papers/anc_ndss17.pdf http://www.cs.vu.nl//~herbertb/download/papers/anc_ndss17.pd... are good starting points, though they are not very accessible to a layman. There have even been timing attacks that let you recover AES keys from analyzing a device's power usage. In layman's terms: computers are extremely complicated beasts of coordinated timing of software and hardware, and timing data leaks many things about what a piece of software is doing. This is the reason that many modern encryption libraries pad operations as a defense against timing attacks.
  - 10y ago

riquito 10y ago

I've just stumpled once again on atob/btoa not working with characters like or 𝒞 and here I read about SharedArrayBuffers, which is way a greater goal. Of course they're right to work on such projects, go for it, but I can't avoid to be baffled by the fact that (apparently) more mundane operations don't receive love. Other examples would be MD5 removed from subtlecrypto, killing any way to speak with services that use it for non-cryptographic operations, or the missing functions to encode/decode hex strings, which maybe are not everyday use cases but are provided by most languages and would need an afternoon to write with tests. I want a better standard library. Just my discouraged 5 cents.

ygra 10y ago
atob/btoa is atrocious from an API design standpoint. Here we are, designing an API to convert things to Base64 and back and we use a string as a byte array substitute, nevermind the fact that strings are not limited to Latin-1. There's probably a bunch of libraries that solve this, but stumbling over this in a library that has no outside dependencies is a bit annoying since you have to solve it yourself ... again.
IIIIIIIIIIII 10y ago
I'm just curious since I never used this feature, are you talking about this issue? > The "Unicode Problem" > Since DOMStrings are 16-bit-encoded strings, in most browsers calling window.btoa on a Unicode string will cause a Character Out Of Range exception if a character exceeds the range of a 8-bit ASCII-encoded character. There are two possible methods to solve this problem: https://developer.mozilla.org/en/docs/Web/API/WindowBase64/Base64_encoding_and_decoding#The_.22Unicode_Problem.22 https://developer.mozilla.org/en/docs/Web/API/WindowBase64/B...
- riquito 10y ago
  Yes, that one

dom0 10y ago

> Gives JavaScript access to shared memory, atomics, and futex. Holy fuck, this is going to be bad, introducing a whole continent of new bugs into JavaScript land. And also giving JS even more and even better options to spy and attack other applications on the same computer.

Matthias247 10y ago
Yes, it introduces a whole new range of multithreading issues also to JS land. However I think threading issues won't be as common as in C++/Java/C# land, since the only thing that will be shared between threads/workers is byte arrays -> There's no shared access to other Javascript objects. As most normal JS developers won't be dealing with plain byte arrays the potential for misuse is not too high.

bricss 10y ago

SharedArrayBuffer and Atomics is already in Chrome, unluckily behind the flag.