5 ms·

Tinyio: A tiny (~200 line) event loop for Python

117 points by tehnub 1y ago

codethief 1y ago
> Somewhat unusually, our syntax uses yield rather than await, but the behaviour is the same. Await another coroutine with yield coro. Await on multiple with yield [coro1, coro2, ...] (a 'gather' in asyncio terminology; a 'nursery' in trio terminology). Why, though?
- dmerrick 1y ago
  > The reason is that await does not offer a suspension point to an event loop (it just calls `__await__` and maybe that offers a suspension point), so if we wanted to use that syntax then we'd need to replace `yield coro` with something like await `tinyio.Task(coro)`. The traditional syntax is not worth the extra class.
- deleted 1y ago
  [deleted]
- halfcat 1y ago
  From the readme: > ”Ever used asyncio and wished you hadn't?” Yes, that’s me. I always found the yield-based approach to coroutines much easier to reason about. It’s just a generator function.
patrickkidger 1y ago
Oh neat! This is my library. Happy to answer any questions. (Though it's really a pretty tiny library that just does what it says on the tin, not sure how many questions there can be. :D )
- b33j0r 1y ago
  I have a question. Why do you prefix your package files with an underscore? In fact, you write all of your python like you really have something to hide ;) Like `_Todo`. Where did you get this pattern? (I’m way more curious than accusatory. Are people embracing private modules these days as a convention, and I just missed it?)
  - deleted 1y ago
    [deleted]
  - deathanatos 1y ago
    I think _private has always been a convention in Python, though I'd say most Python is not so rigorous about it. I don't see why it couldn't be applied to modules. I honestly love when I see a package do stuff like this: it's very clear then what is public interface, and I should consider usable (without sin) and what is supposed to be an internal detail. Same with the modules: then it is very clear that the re-export of those names in __init__.py is where they're meant to be consumed, and the other modules are just for organizational purposes, not API purposes. _Todo is then a private type. Very clean.
  - Galanwe 1y ago
    I tend to do the same, some colleagues as well, so I guess this is some common pattern. The way I see it there are two schools: - The whitelist school: You write everything without _ prefix, then you whitelist what you want accessible with __all__. - The explicit school: You forget about all and just use _ for symbols, modules, etc. I find the latter more readable and consistent (can be applied to attributes, free functions, modules...
  - patrickkidger 1y ago
    Yup, you(/sibling comments) have it correct, it's to mark it as private. Not sure where I got it from, it just seems clean. I don't think I see this super frequently in the ecosystem at large, although anything I've had a hand in will tend to use this style!
- nurettin 1y ago
  I just want to say this is brilliant. I've had my share of problems with asyncio and went back to using sync python and deque instead.
moreati 1y ago
In https://github.com/patrick-kidger/tinyio/blob/main/tinyio/__init__.py https://github.com/patrick-kidger/tinyio/blob/main/tinyio/__... there's from ._core import ( Loop as Loop, sleep as sleep, ... ) Does using `<name> as <name>` change the runtime behaviour at all? Or is it a stylistic choice?
- patrickkidger 1y ago
  This is how static type checkers are told that an imported object is part of the public API for that file. (In addition to anything else present in that file.) C.f. "the intention here is that only names imported using the form X as X will be exported" from PEP484. [1] I'm generally a fan of the style of putting all the implementation in private modules (whose names start with an underscore) and then using __init__.py files solely to declare the public API. [1] https://peps.python.org/pep-0484/ https://peps.python.org/pep-0484/
  - tayo42 1y ago
    That looks like its only for stub files not __init__.py
    - patrickkidger 1y ago
      It also applies to any .py file. (At least in practice with e.g. pyright) That said, the documentation on this matter is close to nonexistent.
kstrauser 1y ago
This is a fun path to explore. Prior art was an earlier Python asynchronous experiment called Tulip, from GVR himself: https://www.dropbox.com/scl/fi/r3w0b50p3m26je12v93ww/SFMeetup2013.pdf?rlkey=l6wzx5y5qw3i5jrhvuyofjt25&e=1 https://www.dropbox.com/scl/fi/r3w0b50p3m26je12v93ww/SFMeetu...
unsnap_biceps 1y ago
I have a function I want to be a coroutine, but it has zero yield statements, so it is just a normal function? You can distinguish it from a normal Python function by putting if False: yield somewhere inside its body. Another common trick is to put a yield statement after the final return statement. Bit ugly but oh well. I'm fairly unfamiliar with python and I don't quite understand what this is actually doing. Does it change anything in the execution or is it just to mark it in a way for IDEs to do something with?
- kstrauser 1y ago
  Yes. If a function contains the yield statement, calling that function returns a generator instead of executing its body. For example, here's defining and calling a regular function: >>> def foo_function(): ... print('In a function!') ... return ... >>> foo_function() In a function! And here's defining and calling a generator: >>> def foo_generator(): ... print('In a generator!') ... return ... yield ... >>> foo_generator() <generator object foo_generator at 0x10321aa40> >>> next(foo_generator()) In a generator! Traceback (most recent call last): File "<python-input-6>", line 1, in <module> next(foo_generator()) ~~~~^^^^^^^^^^^^^^^^^ StopIteration Notice that the generator's body isn't evaluated until you consume the generator. StopIteration isn't actually an error in usual cases. It just says that the generator doesn't have any more values to return and has exited. For example, that's how Python's for-loop works: >>> for _ in foo_generator(): ... continue ... In bar! Here it executes the generator's body (including the print() call) until it gets to the return statement. Because it's a generator, it returns as normal and then raises a StopIteration exception, which tells the loop to stop looping.
gjvc 1y ago
see also https://github.com/dabeaz/curio https://github.com/dabeaz/curio
jojohohanon 1y ago
Twisted, revisited?

117 points by tehnub 1y ago

> Somewhat unusually, our syntax uses yield rather than await, but the behaviour is the same. Await another coroutine with yield coro. Await on multiple with yield [coro1, coro2, ...] (a 'gather' in asyncio terminology; a 'nursery' in trio terminology). Why, though?

dmerrick 1y ago
> The reason is that await does not offer a suspension point to an event loop (it just calls `__await__` and maybe that offers a suspension point), so if we wanted to use that syntax then we'd need to replace `yield coro` with something like await `tinyio.Task(coro)`. The traditional syntax is not worth the extra class.
deleted 1y ago
[deleted]
halfcat 1y ago
From the readme: > ”Ever used asyncio and wished you hadn't?” Yes, that’s me. I always found the yield-based approach to coroutines much easier to reason about. It’s just a generator function.

patrickkidger 1y ago

Oh neat! This is my library. Happy to answer any questions. (Though it's really a pretty tiny library that just does what it says on the tin, not sure how many questions there can be. :D )

b33j0r 1y ago
I have a question. Why do you prefix your package files with an underscore? In fact, you write all of your python like you really have something to hide ;) Like `_Todo`. Where did you get this pattern? (I’m way more curious than accusatory. Are people embracing private modules these days as a convention, and I just missed it?)
- deleted 1y ago
  [deleted]
- deathanatos 1y ago
  I think _private has always been a convention in Python, though I'd say most Python is not so rigorous about it. I don't see why it couldn't be applied to modules. I honestly love when I see a package do stuff like this: it's very clear then what is public interface, and I should consider usable (without sin) and what is supposed to be an internal detail. Same with the modules: then it is very clear that the re-export of those names in __init__.py is where they're meant to be consumed, and the other modules are just for organizational purposes, not API purposes. _Todo is then a private type. Very clean.
- Galanwe 1y ago
  I tend to do the same, some colleagues as well, so I guess this is some common pattern. The way I see it there are two schools: - The whitelist school: You write everything without _ prefix, then you whitelist what you want accessible with __all__. - The explicit school: You forget about all and just use _ for symbols, modules, etc. I find the latter more readable and consistent (can be applied to attributes, free functions, modules...
- patrickkidger 1y ago
  Yup, you(/sibling comments) have it correct, it's to mark it as private. Not sure where I got it from, it just seems clean. I don't think I see this super frequently in the ecosystem at large, although anything I've had a hand in will tend to use this style!
nurettin 1y ago
I just want to say this is brilliant. I've had my share of problems with asyncio and went back to using sync python and deque instead.

moreati 1y ago

In https://github.com/patrick-kidger/tinyio/blob/main/tinyio/__init__.py https://github.com/patrick-kidger/tinyio/blob/main/tinyio/__... there's from ._core import ( Loop as Loop, sleep as sleep, ... ) Does using `<name> as <name>` change the runtime behaviour at all? Or is it a stylistic choice?

patrickkidger 1y ago
This is how static type checkers are told that an imported object is part of the public API for that file. (In addition to anything else present in that file.) C.f. "the intention here is that only names imported using the form X as X will be exported" from PEP484. [1] I'm generally a fan of the style of putting all the implementation in private modules (whose names start with an underscore) and then using __init__.py files solely to declare the public API. [1] https://peps.python.org/pep-0484/ https://peps.python.org/pep-0484/
- tayo42 1y ago
  That looks like its only for stub files not __init__.py
  - patrickkidger 1y ago
    It also applies to any .py file. (At least in practice with e.g. pyright) That said, the documentation on this matter is close to nonexistent.

kstrauser 1y ago

This is a fun path to explore. Prior art was an earlier Python asynchronous experiment called Tulip, from GVR himself: https://www.dropbox.com/scl/fi/r3w0b50p3m26je12v93ww/SFMeetup2013.pdf?rlkey=l6wzx5y5qw3i5jrhvuyofjt25&e=1 https://www.dropbox.com/scl/fi/r3w0b50p3m26je12v93ww/SFMeetu...

unsnap_biceps 1y ago

I have a function I want to be a coroutine, but it has zero yield statements, so it is just a normal function? You can distinguish it from a normal Python function by putting if False: yield somewhere inside its body. Another common trick is to put a yield statement after the final return statement. Bit ugly but oh well. I'm fairly unfamiliar with python and I don't quite understand what this is actually doing. Does it change anything in the execution or is it just to mark it in a way for IDEs to do something with?

kstrauser 1y ago
Yes. If a function contains the yield statement, calling that function returns a generator instead of executing its body. For example, here's defining and calling a regular function: >>> def foo_function(): ... print('In a function!') ... return ... >>> foo_function() In a function! And here's defining and calling a generator: >>> def foo_generator(): ... print('In a generator!') ... return ... yield ... >>> foo_generator() <generator object foo_generator at 0x10321aa40> >>> next(foo_generator()) In a generator! Traceback (most recent call last): File "<python-input-6>", line 1, in <module> next(foo_generator()) ~~~~^^^^^^^^^^^^^^^^^ StopIteration Notice that the generator's body isn't evaluated until you consume the generator. StopIteration isn't actually an error in usual cases. It just says that the generator doesn't have any more values to return and has exited. For example, that's how Python's for-loop works: >>> for _ in foo_generator(): ... continue ... In bar! Here it executes the generator's body (including the print() call) until it gets to the return statement. Because it's a generator, it returns as normal and then raises a StopIteration exception, which tells the loop to stop looping.

gjvc 1y ago

see also https://github.com/dabeaz/curio https://github.com/dabeaz/curio

jojohohanon 1y ago

Twisted, revisited?