3 ms·

Show HN: Writing a C++20M:N Scheduler from Scratch (EBR, Work-Stealing)

• srean 8mo ago
  Any Fortran folks around ? Since old Fortran does not have function pointers I used to wonder how does one write gradient descent routine for a user defined function in Fortran. In other languages, one typically passes the user defined function and it's gradient to the routine as callbacks, so that the routine can call them at will. It was a moment of great amusement and joy (grinning like a silly kid) when I learned how Fortran did it. It was through a pared down form of a coroutine. Inversion of control. The gradient descent routine would effectively suspend and 'return' with a flag indicating that it needs the user defined function computed, or the gradient function computed. These are computed outside and then the routine is called again with these recently computed values. At this point it resumes from where it had yielded. Pretty neat. The routine's return value indicated whether it is done, or that it is suspended and expects to be called again to resume. Now I don't remember if the suspended state was stored transparently by the run time, or explicitly in the arguments of the routine (pass by reference, I think), or whether the local, routine specific 'program counter' was managed explicitly or transparently by the compiler and run time.
  • throwaway81523 8mo ago
    Fortran has always had function pointers though they were called something different, and could only be used in limited contexts. But look at any numerical integration or root finding library. You just pass the function name in as a parameter to another function.
    • srean 8mo ago
      I see. I stand corrected.
    • pklausler 8mo ago
      Fortran has had dummy procedures (subprogram arguments that can be associated with subroutines or functions) since Fortran II.
  • lixiasky 8mo ago
    That is a fascinating comparison! History really does rhyme. What you described—suspending with a flag and resuming with new values—is surprisingly similar to how the C++20 compiler transforms coroutines into a state machine under the hood. In tiny_coro, the promise_type essentially manages that "inversion of control" you mentioned, but hides the manual state management behind the co_await syntax. It's really cool to see that these fundamental patterns for "stopping and resuming" computation have existed for so long, just in different forms.
    • throwaway81523 8mo ago
      Also look at Protothreads :)
      • lixiasky 8mo ago
        Oh absolutely! Adam Dunkels' work is legendary. It's kind of poetic that Protothreads achieved stackless concurrency via C preprocessor macros and Duff's device tricks, and now years later, C++20 finally baked that exact same "state machine via switch-case" logic directly into the compiler. Same spirit, just less macro magic! :)
        • srean 8mo ago
          Simon Tatham and Tom Duff too. https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html https://www.chiark.greenend.org.uk/~sgtatham/coroutines.html
          • lixiasky 8mo ago
            The legendary "Coroutines in C" article! Thanks for the reference—I learned a lot from his work. His use of macros to hide the switch statement is basically the spiritual ancestor of co_await. It is wild to think that C++20 essentially standardized that exact pattern, just moving the "state machine generation" from the preprocessor hacks to the compiler itself.
    • srean 8mo ago
• throwaway81523 8mo ago
  This looks cool, but says it's simpler than Seastar. Seastar last time I looked at it was fairly simple, but didn't use coroutines at all. Instead it used a futures/promise scheme. It was in C++14, back when C++20 coroutines didn't exist yet. C++20 coroutines have seemed very complex to me and I haven't sat down to try to understand them. I guess tiny_coro could help for that. I do remember that they were stackless. I'm not that keen on coroutine and async approaches to concurrency anyway these days. I'd rather use something with actual multitasking, either with Posix threads or with lightweight processes like Erlang's or Go's. Also, C++ itself is apparently in the process of sinking into a bog. Rust is taking over, which is not entirely good news. I had wanted to spend more time on Ada.
  • lixiasky 8mo ago
    You hit the nail on the head. C++20 coroutines are indeed complex and the barrier to entry is high. However, that complexity actually forced me to start from first principles. It drove me to tackle the essential problems from the ground up, which gave me a much deeper understanding of how coroutines truly work. That is exactly why I built this project—I wanted to create a minimal "laboratory" to dissect stackless coroutines without the overhead of a massive framework like Seastar. Regarding your point on Erlang/Go: That's actually the goal of this scheduler! It implements the M:N threading model (Work-Stealing) to simulate that kind of "lightweight process" concurrency, but giving you manual control over the mechanics. Hope this helps you finally wrap your head around co_await!
    • throwaway81523 8mo ago
      I think there is some hope of a sane wrapper around C++20 coroutines that will make them easier to use. I saw a tutorial a while back mentioning that might eventually become part of the C++ standard. I once tried to use Boost coroutines but it was too much headache and I switched to a different approach. Erlang's processes and Goroutines are stackful unlike C++ coroutines. Erlang also forbids observable data sharing between processes which avoids a lot of pitfalls. I don't think that can be enforced in C++ or Go. GHC lightweight threads and its STM library (software transactional memory) could be another thing to look at. I wonder if a useful STM feature is feasible for tiny_coro.
• srean 8mo ago
  Does anyone know where to find the mailing list archives of Stackless Python ? I want to track how its design evolved over time. Sincere apologies for the hijack. Hopefully it too will be of interest to those who are interested in coroutines.
• stefantalpalaru 8mo ago
  [dead]