6 ms·

Note though, that many of the problems that come up with threads can still come up with single-threaded async, especially when building a server. As soon as you

Note though, that many of the problems that come up with threads can still come up with single-threaded async, especially when building a server. As soon as you await, any object visible to another promise chain can potentially change out from under you, and this can cause analogous problems to multithreading. One example where this can get hairy is if you have a database connection shared between two promise chains running "in parallel". You'll get undefined query ordering, which can cause very bizarre behavior. Don't get me wrong: it's still easier. But there's an interesting property I've noticed of statements of the form "by definition, problem X is impossible when you have Y", which is that X-analogous problems absolutely can happen, and they are often more complicated to solve. For another example: "by definition, memory leaks are impossible when you have a garbage collector".

foota 9y ago
I once had to write a lock for JavaScript, it was beautifully simple.
- vanderZwan 9y ago
  You're making me curious enough to want to see it. Would it fit in a post here or on a short gist?
  - HeyImAlex 9y ago
    It can just be a boolean variable called locked?
    - deleted 9y ago
      [deleted]
    - deleted 9y ago
      [deleted]
    - nitely 9y ago
      Almost. In threaded programming a thread will wait/block until the lock is released and then make use of the protected resource. In async world you would probably just check some flag and if it's "locked" then register a callback to some "unlocked" event to mimic this.
      - amelius 9y ago
        "Promises" are a lot like locks.
      - anonacct37 9y ago
        Lookup twisted deferred locks. https://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.DeferredLock.html https://twistedmatrix.com/documents/9.0.0/api/twisted.intern...
        nitely 9y ago
        Looks nice. How is this implemented?
        rdtsc 9y ago
        Not gp, but I used Twisted Matrix for a few years. It is usually implemented with a boolean flag and a list of client waiting to be called back when the lock is released. So users of the lock would call acquire and get back a deferred (a promise in other language), but say in the most basic way, a callback is passed to it. When lock can be acquired, that callback is called (or deferred is fired). When done with the resource, must call the release. Release then will trigger the lock to inspect the list of available waiting clients, pick one and fire their callback letting them know they can access the resource, and so on. https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/defer.py#L1581 https://github.com/twisted/twisted/blob/trunk/src/twisted/in...
    - ChristianBundy 9y ago
      Yep. let lock; const log = (msg) => { lock = new Promise(async (resolve) => { await lock; console.log(msg); setTimeout(resolve, 1000); }); }; 'hello world'.split('').forEach(log);
  - foota 9y ago
    I don't have access to the code anymore, unfortunately. But it kept track of a list of people waiting on the lock and then notified the first caller off the list when the lock was released. It was only beautifully simple with respect to what it would have been in a non single-threaded evnironment. I needed it in order to use a library that had some annoying global state in it.
  - johnbender 9y ago
    Not sure if this helps but I hacked this together the other day to avoid adding a real queue to a very simple application. It uses event queue ordering semantics to get FIFO behavior. The read/write combinations to the semaphore variable need not be atomic since we know that if this thread is running there are no other threads that can be reading and then modifying it. MyObject.queueSemaphore = N; MyObject.emitter = new EventEmitter(); MyObject.queue = function( ... ){ // if this run can take place go // otherwise wait for a run-complete to try again if(this.queueSemaphore > 0) { this.queueSemaphore--; } else { // must return a promise that resolves and possibly requeues return new promise((resolve, reject) => { this.emitter.once("run-complete", () => { resolve(true); }); }).then(() => this.queue( ... )); } this.promiseBasedProcess() // ... work .finally(() => { this.queueSemaphore++; this.emitter.emit("run-complete"); }); }
  - johnsonjo 9y ago
    Here's a lock implementation I made just barely as a proof of concept: https://bl.ocks.org/johnsonjo4531/99256568deaf8c0f1685793a4a8205bb https://bl.ocks.org/johnsonjo4531/99256568deaf8c0f1685793a4a... Basically it is used like this: ``` var lock = new AsyncLock(); (async () => { for await(var _ of lock) { console.log("here 1a"); await sleep(1000); console.log("here 1b") } })(); (async () => { for await(var _ of lock) { console.log("here 2a"); await sleep(1000); console.log("here 2b") } })(); /* When Lock is working should print: here 1a here 1b here 2a here 2b Without the lock it would print: here 1a here 2a here 1b here 2b */ ``` If you want to check out the implementation go to the blocks link. (which is basically just a gist that you can run). Here's the actual gist link: https://gist.github.com/johnsonjo4531/99256568deaf8c0f1685793a4a8205bb https://gist.github.com/johnsonjo4531/99256568deaf8c0f168579.... It requires Chrome 63 (at this point Chrome Canary) because it uses async iterators. I used this because I can close the lock automatically in case an error is thrown, the loop is broken. It only iterates one value and only after the lock is unlocked.
andrewstuart2 9y ago
Concurrency !== parallelism. You can have concurrency, as in async javascript, and all the issues that come with concurrency, even when no two execution paths are executing simultaneously. The main benefit is that your lock structures don't have to worry about being accessed in parallel.
- dahart 9y ago
  Valid point that async is easier than parallel. I've done a lot of both, and parallel is harder. But, I realize that some people say that concurrency in software is not parallel, but I think that's a poor choice of terminology that is prone to confusion and argument. The word concurrent is literally synonymous with "simultaneous". We have a good word for asynchronous. Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning.
  - throwaway613834 9y ago
    > Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning. No, let's not. They mean different things. The execution of Python generators is concurrent with your code, but not parallel or asynchronous. And it's also completely possible to have concurrent synchronous parallel execution [1]. [1] https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_model https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_...
    - dahart 9y ago
      I think you agree with me- the comment I replied to appears to conflate concurrency and asynchrony. The article is about async. I’m saying use async when talking about async. And I avoid using concurrent if I can. It does not have a strict CS definition, and it is prone to miscommunication. Many people use “concurrent” when they are talking about parallelism. Some people’s CS definition of concurrency is not very well matched with the common non-CS definition of the word. The dictionary definition of “concurrent”, according to Google, is: “existing, happening, or done at the same time.” synonyms: simultaneous, coincident, contemporaneous, parallel Using “concurrent” to mean not parallel, or even out of order or not necessarily parallel, just seems like a bad idea.
frutiger 9y ago
To deal with this exact issue, I wrote a 'Promise Queue' for use in our codebase. The implementation of this queue is simple. It is initialized with a resolved promise. Each operation (i.e. a function that returns a promise) is enqueued by calling `.then(theFunction)` on the queue, while replacing it with the returned value from the `.then(...)` invocation. This ensures that two different operations do not interleave. You lose some potential speed this way, but you are able to reason about the states your program will take far more easily.
- mistercow 9y ago
  That sounds sort of similar to what Selenium webdriver does with promises, although it does it so that the promises can be automatically managed without chaining .then calls. In webdriver's case, this causes async/await to deadlock. Does your queue system have that problem, or is it something you had to work around? I ask because at work, we have a lot of browser tests that we'd like to start transitioning off of that deprecated queuing mechanism. If I can rip out Selenium's version and provide my own mechanism that doesn't deadlock on async/await, it would make the transition a lot easier.

foota 9y ago

I once had to write a lock for JavaScript, it was beautifully simple.

vanderZwan 9y ago
You're making me curious enough to want to see it. Would it fit in a post here or on a short gist?
- HeyImAlex 9y ago
  It can just be a boolean variable called locked?
  - deleted 9y ago
    [deleted]
  - deleted 9y ago
    [deleted]
  - nitely 9y ago
    Almost. In threaded programming a thread will wait/block until the lock is released and then make use of the protected resource. In async world you would probably just check some flag and if it's "locked" then register a callback to some "unlocked" event to mimic this.
    - amelius 9y ago
      "Promises" are a lot like locks.
    - anonacct37 9y ago
      Lookup twisted deferred locks. https://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.DeferredLock.html https://twistedmatrix.com/documents/9.0.0/api/twisted.intern...
      - nitely 9y ago
        Looks nice. How is this implemented?
        rdtsc 9y ago
        Not gp, but I used Twisted Matrix for a few years. It is usually implemented with a boolean flag and a list of client waiting to be called back when the lock is released. So users of the lock would call acquire and get back a deferred (a promise in other language), but say in the most basic way, a callback is passed to it. When lock can be acquired, that callback is called (or deferred is fired). When done with the resource, must call the release. Release then will trigger the lock to inspect the list of available waiting clients, pick one and fire their callback letting them know they can access the resource, and so on. https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/defer.py#L1581 https://github.com/twisted/twisted/blob/trunk/src/twisted/in...
  - ChristianBundy 9y ago
    Yep. let lock; const log = (msg) => { lock = new Promise(async (resolve) => { await lock; console.log(msg); setTimeout(resolve, 1000); }); }; 'hello world'.split('').forEach(log);
- foota 9y ago
  I don't have access to the code anymore, unfortunately. But it kept track of a list of people waiting on the lock and then notified the first caller off the list when the lock was released. It was only beautifully simple with respect to what it would have been in a non single-threaded evnironment. I needed it in order to use a library that had some annoying global state in it.
- johnbender 9y ago
  Not sure if this helps but I hacked this together the other day to avoid adding a real queue to a very simple application. It uses event queue ordering semantics to get FIFO behavior. The read/write combinations to the semaphore variable need not be atomic since we know that if this thread is running there are no other threads that can be reading and then modifying it. MyObject.queueSemaphore = N; MyObject.emitter = new EventEmitter(); MyObject.queue = function( ... ){ // if this run can take place go // otherwise wait for a run-complete to try again if(this.queueSemaphore > 0) { this.queueSemaphore--; } else { // must return a promise that resolves and possibly requeues return new promise((resolve, reject) => { this.emitter.once("run-complete", () => { resolve(true); }); }).then(() => this.queue( ... )); } this.promiseBasedProcess() // ... work .finally(() => { this.queueSemaphore++; this.emitter.emit("run-complete"); }); }
- johnsonjo 9y ago
  Here's a lock implementation I made just barely as a proof of concept: https://bl.ocks.org/johnsonjo4531/99256568deaf8c0f1685793a4a8205bb https://bl.ocks.org/johnsonjo4531/99256568deaf8c0f1685793a4a... Basically it is used like this: ``` var lock = new AsyncLock(); (async () => { for await(var _ of lock) { console.log("here 1a"); await sleep(1000); console.log("here 1b") } })(); (async () => { for await(var _ of lock) { console.log("here 2a"); await sleep(1000); console.log("here 2b") } })(); /* When Lock is working should print: here 1a here 1b here 2a here 2b Without the lock it would print: here 1a here 2a here 1b here 2b */ ``` If you want to check out the implementation go to the blocks link. (which is basically just a gist that you can run). Here's the actual gist link: https://gist.github.com/johnsonjo4531/99256568deaf8c0f1685793a4a8205bb https://gist.github.com/johnsonjo4531/99256568deaf8c0f168579.... It requires Chrome 63 (at this point Chrome Canary) because it uses async iterators. I used this because I can close the lock automatically in case an error is thrown, the loop is broken. It only iterates one value and only after the lock is unlocked.

andrewstuart2 9y ago

Concurrency !== parallelism. You can have concurrency, as in async javascript, and all the issues that come with concurrency, even when no two execution paths are executing simultaneously. The main benefit is that your lock structures don't have to worry about being accessed in parallel.

dahart 9y ago
Valid point that async is easier than parallel. I've done a lot of both, and parallel is harder. But, I realize that some people say that concurrency in software is not parallel, but I think that's a poor choice of terminology that is prone to confusion and argument. The word concurrent is literally synonymous with "simultaneous". We have a good word for asynchronous. Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning.
- throwaway613834 9y ago
  > Let's use async instead of saying that concurrent means async but not parallel in spite of it's common meaning. No, let's not. They mean different things. The execution of Python generators is concurrent with your code, but not parallel or asynchronous. And it's also completely possible to have concurrent synchronous parallel execution [1]. [1] https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_model https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_...
  - dahart 9y ago
    I think you agree with me- the comment I replied to appears to conflate concurrency and asynchrony. The article is about async. I’m saying use async when talking about async. And I avoid using concurrent if I can. It does not have a strict CS definition, and it is prone to miscommunication. Many people use “concurrent” when they are talking about parallelism. Some people’s CS definition of concurrency is not very well matched with the common non-CS definition of the word. The dictionary definition of “concurrent”, according to Google, is: “existing, happening, or done at the same time.” synonyms: simultaneous, coincident, contemporaneous, parallel Using “concurrent” to mean not parallel, or even out of order or not necessarily parallel, just seems like a bad idea.

frutiger 9y ago

To deal with this exact issue, I wrote a 'Promise Queue' for use in our codebase. The implementation of this queue is simple. It is initialized with a resolved promise. Each operation (i.e. a function that returns a promise) is enqueued by calling `.then(theFunction)` on the queue, while replacing it with the returned value from the `.then(...)` invocation. This ensures that two different operations do not interleave. You lose some potential speed this way, but you are able to reason about the states your program will take far more easily.

mistercow 9y ago
That sounds sort of similar to what Selenium webdriver does with promises, although it does it so that the promises can be automatically managed without chaining .then calls. In webdriver's case, this causes async/await to deadlock. Does your queue system have that problem, or is it something you had to work around? I ask because at work, we have a lot of browser tests that we'd like to start transitioning off of that deprecated queuing mechanism. If I can rip out Selenium's version and provide my own mechanism that doesn't deadlock on async/await, it would make the transition a lot easier.