4 ms·
At home-assistant.io we just migrated our core from being based on threads + locks to use asyncio. We managed to keep a full backwards compatible API available
by balloob 10y ago
At home-assistant.io we just migrated our core from being based on threads + locks to use asyncio. We managed to keep a full backwards compatible API available so we can slowly migrate other parts of the system over to async.
Asyncio has a steep initial learning curve (especially in our hybrid setup) but it's well worth it. We target low resource computers like Raspberry Pi and using async over threads has speed up things a lot.
The biggest catch is that while writing code you have to think about every function that you use. Is it doing I/O, is it a coroutine or is it callback/async friendly.
- ddorian43 10y ago... why do you need locks in threads and not in asyncio ?
- paulddraper 10y agoBecause with asyncio you can run on one thread. Before, you needed multiple threads, or else incur high I/O costs. But then you have to manage your shared state across threads, hence locks. Their code is now probably async single threaded code, like JS, where locks are irrelevant.
- ddorian43 10y agoBut don't threads run cpu-stuff in 1 thread, by not switching? (in python) The same as async but more overhead/slower.
- paulddraper 10y agoOnly one thread runs at a time because of the GIL, but the threads can preempt each other wherever between opcodes (http://effbot.org/zone/thread-synchronization.htm#atomic-operations http://effbot.org/zone/thread-synchronization.htm#atomic-ope...) Imagine two threads running a += 1 That's four opcodes. Only one operation will run at a time, but one thread might preempt the other midway through the four operations, and `a` will have been incremented by 1, not two. In some sense, mulithreading with Python's GIL is the worst of both worlds: you can use only one CPU, but you still need to account for concurrent state mutation. (Of course, it doesn't have the function color problem of asyncio http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...)
- the_mitsuhiko 10y agoYou need locks in asyncio as well.
- ddorian43 10y agoI know you do, but I asked for his implementation (which needed locks on threads but not on async)
- xapata 10y agoOnly if you have multiple threads/processes/coroutines sharing resources.
- DasIch 10y ago"Only"? You need locks in exactly the same scenarios as you do when you use multiple threads.
- xapata 10y agoGood point. I should have said that asyncio allows patterns that use fewer shared resources. Actually... I take that back. Asyncio is solving a different problem than the locking of shared resources.
- hamiltonkibbe 10y agoProbably because you don't have to worry about context switches in the middle of operations, since they happen at 'await' or 'yield's in coroutines, rather than whenever the OS task timer interrupts
- balloob 10y agoWe no longer need locks in the core because there is now only 1 thread that interacts with the core. We still have other threads but all they do is schedule tasks to be run on the event loop. https://github.com/home-assistant/home-assistant/blob/dev/homeassistant/core.py https://github.com/home-assistant/home-assistant/blob/dev/ho...
- takeda 10y agoThere's still a possibility that a lock might be needed if you use await/yield from inside critical section (another co-routine could make changes at that point). If you don't yield then indeed locks are not needed.