24 ms·
C++ Asynchronous Framework
- voz_ 4y agoThis doesn't mean much without benchmarks
- deleted 4y ago[deleted]
- mikewarot 4y agoSomewhere in that library is code to yield()... and all I see is a forest of .hpp files (which I assume are the equivalent of .h files for c++) Where does it actually get implemented?
- usrnm 4y agoIt internally uses ucontext via Boost
- mikewarot 4y agoThank you for explaining that to me. I suspected it was something else and was trying to find the code that saved state and forked in C++ userland. I once did cooperative multi-tasking, which I thought this library did as well, in the 1980s in Turbo Pascal under MS-DOS, and wondered how things have changed.
- brosciencecode 4y ago
- love2read 4y agowhy does it have to be a “russian clone”, yandex works in us too…
- weswil07 4y agoBecause it's a Russian company.
- Kukumber 4y agowhat the country has to do with any of that? is rust an american clone of pony?
- redwood 4y ago
- Kukumber 4y agoIt's better to be direct and share concerns about the fact that it is a Russian company, just like you did, instead of the passive-racism, wich only encourages naive questions like mine, wich i don't think was his goal
- Arnavion 4y agoI don't see any reason to assume anyone in this thread was being racist.
- bardworx 4y agoTo a degree, sorta? I mean, we have examples in history like Porsche making the worlds worst tank for the Nazis (transmission fell out). Lamborghini “hiring” People to avoid going to war. Manhattan project. There is precedence for morality in engineering. There are counter examples, as well.
- gerdesj 4y ago
- Arnavion 4y ago
- r-w 4y agoWhat makes you think it’s a clone rather than a case of not-invented-here syndrome?
- deleted 4y ago[deleted]
- Waterluvian 4y agoThis is an interesting question. A comparison blog post might be an interesting answer.
- deleted 4y ago[deleted]
- Arnavion 4y agoBased solely on the one example function on the homepage, this seems to be pre-empting asynchronous functions, presumably a fiber yielding and resuming internally, so that the code looks synchronous. libuv's API is based on callbacks. So both the implementation and the usage are very different.
- throwaway81523 4y agoThis uses coroutines so there is no preemption. It is much higher level than libuv. I would say it is higher level than Seastar, which uses callbacks. After about 10 minutes of skimming the docs, I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines. There are a bunch of IO primitives that operate with their coroutine system but I didn't check to see how they work. I'd like to see versions of all the usual Linux system calls exposed through io_uring instead of having to use different async options or threadpools for everything. I don't know how close we are to that now. I don't know whether userver uses io_uring at all, but it would be nice if it does. There is a comparison chart with several other such frameworks in various languages, Seastar is not in the chart, but it would be nice if it was. DPDK is not mentioned unless I missed it (which is possible). I don't know if it is still supposed to be faster than Linux kernel networking. Seastar has good support for it. The coroutines are stackful and it looks like you have to allocate space for them on instantiation. I don't know if there is any attempt to catch stack overflows. The obvious approaches using mmap might help though they aren't really sound. There is a hardware extension (CHERI) for pointer ownership on ARM and RISC-V that might be cool for this, so I hope it gets traction. I've never gotten into the deep weeds of cooperative multitasking, but people who have say it gets ugly as the program gets more complicated, and that it tends to give way to OS's with preemption and processes for that reason. C++ itself is also famously fraught with hazards. Overall userver looks useful and worth looking into so I'm glad to hear about this release.
- Arnavion 4y ago>This uses coroutines so there is no preemption. Right, I meant cooperative yielding (which fibers are), but accidentally called it the exact opposite. Whoops. >I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines It's boost coroutines. The /uboost_coro directory is a vendored copy. It's used in /core/src/engine/coro/pool.hpp
- quantum_state 4y ago
- cyanydeez 4y ago
- desindol 4y agoIt would be very sad for the hacker community to forget about how they started...
- trhway 4y ago>It would be very sad for the tech community to become political, labeling tech based on where it happened to be developed initially… people, we are all human being … political differences can naturally be dismissed until it crosses the border when some humans stop behaving like humans... And this is the border that Russia has crossed.
- Kstarek 4y agoNah good code is good regardless of who wrote it
- speed_spread 4y agoAs an old dev, I'll tell you that all code is shit code. But some of it is my shit, and other bits are your shit.
- FridgeSeal 4y agoYandex (and ClickHouse) does some really, really cool technical stuff. I like that a lot of the stuff they write about and open-source has a fairly strong focus on efficiency + performance. I also find the supporting docs (those that detail the problem space, how it solves the problem, etc) quite well written and often a lot more comprehensible than docs from other mainstream tech companies (MS docs are the worst, followed by Google). I’m not sure why this is the case though, is it a different development culture? Different project management culture (provisions time for documentation?) something else?
- alfalfasprout 4y agoUnfortunately, larger tech companies in the US have become increasingly (in the last 5-10 years) political inside and a desire to ship things fast has ultimately meant building extremely high performance scalable infra that's foundational enough to be OSS-able is just not a priority since compute is cheap enough and it's not worth the cost of generalizing it to make it OSS. It's just very hard to pitch these kinds of investments. Documentation follows this pattern-- engineers just don't get promoted or rewarded for writing documentation and with IC's constantly being pushed to deliver faster and faster it just won't take priority. And that's a big problem. In short, it's my impression it's an engineering culture problem in large part caused by bad leadership at the types of companies where these projects could be possible.
- yodsanklai 4y agoPlaying devil's advocate, but why is that a big problem? large tech companies aren't altruistic and leadership isn't stupid. If shipping fast gives them a better competitive advantage than writing OSS, why should they invest in OSS?
- ThalesX 4y agoNot GP but attempting an answer. It really depends on the perspective you're looking at the situation from. If you're concerned with better competitive advantages and the final bottom line for company X or Y, in this current market, in the short term, sure, that sounds amazing, sign me up on that moon rocket! If your concern is a bit more hard to quantify, for example maximizing human potential, allowing individuals access to high quality technological infrastructure so that we may develop as a species and reach new heights, then you might feel like this approach is a bit short sighted.
- coliveira 4y agoExcellent job, I love Yandex products, and this framework seems to be very useful.
- twobitshifter 4y ago“Speed of C++, simplicity of Python, coroutine model of Go.” Is there a speed comparison to Go?
- deleted 4y ago[deleted]
- godmode2019 4y agoYandex has better long tail results than google or bing. If you are trying to find something obscure, yandex is always my final option, before I give up. Google is basically a Amazon pipeline at this point, I save every useful URL because I know I won't be able to find it again. If you are trying to buy something google is easily the best, or anything local, or news related. Bing is a better general search engine, for quick searches.
- shaklee3 4y agoI'm not sure if you noticed, but the Yandex comment was not about the search, but the software engineering and open source projects coming from them.
- jdougan 4y agoWe may have found the GPT-3
- godmode2019 4y agoPeople have low opinions of yandex, why would they trust their software development efforts if they have low opinions of the company. As my comment notes yandex has its place in the western search world, and is very useful for long tail searches. Hardly off topic, while your snide response is.
- shaklee3 4y agoI wasn't trying to be snide; I didn't understand what the search had to do with the company. everyone I've met (including on this thread) have high opinions of Yandex, so to each their own.
- godmode2019 4y agoI may have conflated your response to the child response that suggested I was a bot. I apologise
- hugez9 4y agoDo you know about cython ?
- maxdo 4y agoYandex top managers are running away from Russia as their government forcing them to spread hate. They alter their news, search results as government asks. Their tech including DB's is used for military use and is one of backbone companies that keep this bloody regime to run. No matter how good this product is , Yandex as company is extremely toxic. Contributing to this product you are indirectly contribute in killing people. They are advertising ( online ) recruitment process of an extremely poor and depressed regions to go and fight in ukraine. They are one of the major propaganda engine of a regime who combined 18th century imperialist where they are excited to capture new territories and praise it will be good for their isolated economy. 80% of Russians Support war. Hence yandex as company is in charge of something even more frightful actions. They are actually hate ukrainains getting to the level of nazzis that considered some the west european countries a broken version of Germans. To compare. Soviet Union had thousands of Schools in ukrainian, printed good quality books in Ukrainian, they only didn't tolerate politics. Russians don't recognize ukraine as a nation with any rights and opened educational camps. The first thing they do on captured territories they destroy Ukrainain books, schools materials and any relations with culture. They import brainwashed teachers from Russia and forcefully educate kids. School Teachers are backbone of their regime. Guess who provide them software? Yandex . Yandex software is definitely used to run filtration camps, forcefully re-educate kids from their native language into Russian and other things no one could imagine would exist in 21st century. So no, this is not yet another proxy war between two big powers. This a war of reborn 18th century imperialism combined with fascist way of mobilizing russian people. Is up to you to support their project or not, but one day this new world order can come into your house.
- deleted 4y ago[deleted]
- jillesvangurp 4y agoHere's a nice article by the Moscow times about how Putin's regime relies on Russian engineers to build out it's surveillance and censorship apparatus: https://www.themoscowtimes.com/2022/07/29/the-good-engineer-zima-a78431 https://www.themoscowtimes.com/2022/07/29/the-good-engineer-... Yandex is very much a part of this strategy. It started out as a domestic competitor to Google and as Russia has been banning and censoring Google and other foreign services domestically, it became more important. So, this parent comment while obviously very political given current circumstances, is actually very on topic. Yandex is part of the information war on the ground in the Ukraine right now. And not just the Ukraine. Russia's sphere of influence spreads quite wide. A few practical considerations: - Would you use native libraries like this as part of your architecture that are controlled by people with a history of infiltrating e.g. voting machines, infrastructure used by media companies, etc. Believe what you will but that would be solid no for me just from a security point of view. There's just no way to know what piggybacks along with that code base. - Given the above, would you contribute to such a project and help the people behind it and further their goals? I'm sure it's technically great stuff, but no thanks.
- dataflow 4y agoNo Windows support apparently: https://userver.tech/d9/d08/md_en_userver_supported_platforms.html https://userver.tech/d9/d08/md_en_userver_supported_platform...
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- intelVISA 4y agoLooks good, I usually roll my own async stuff with C++20 coroutines but I'll have to try this out; seems promising.
- fractalb 4y ago> Speed of C++, simplicity of Python, coroutine model of Go. That sounds great. Can someone with more experience with C++ tell how good this framework actually is?
- pjmlp 4y agoFrom the docs it looks nice, and it is taking advantage of several modern features. Apparently it isn't using C++ 20 co-routines to avoid limiting it as existing support it is still pretty much WIP.
- cmm 4y agoI just briefly skimmed the docs, so my observations may be inaccurate, but I got the impression that "stackful coroutines" are basically green threads (not C++20 coroutines, as the name might hint). Green threads need a dedicated stack per thread, with all that implies, and also I don't find the cute "magical" implicit yielding helpful readability- and comprehension-wise, but that's more a matter of taste. It is either telling or a glaring oversight (let's assume the latter) that there is no comparison with Seastar, which supports both C++20 coroutines and callbacks, with helpful syntax sugar for latter (coroutines are still nicer though!), has green threads too (which it calls "threads"), and can use io_uring, dpdk, etc. Seastar is free, actively developed, and is used by real software with demanding performance and scaling requirements, so in light of Seastar existing I'd expect any new C++ async framework to approach the task of "selling" itself more seriously than "it's like Go or Python but in C++!".
- MBkkt 4y agoSeastar is absolutely different approach, based on shared nothing architecture. It's not bad, but you cannot easy compare some async multi threaded architecture with it. Seastar it's something like multi process architecture, where every process doesn't have synchronization, except n spsc queue per process (used to communicate between cores) and have only cooperative multitasking So it good scales, if you don't have a lot of data to share, and have a very good work load balancer But commonly you haven't, so go-like approach more and more easy
- sannysanoff 4y agoSeems like same principles used in https://github.com/yyzybb537/libgo https://github.com/yyzybb537/libgo Using libgo for 3 year, still cool. Most important: you add it, add some convenience methods on top of it, and forget about it.
- mgaunard 4y ago"the" C++ asynchronous framework? How presumptuous.
- ot 4y agoI don't think that's intentional. Russian doesn't have articles [1], so misuse (or omission) of the/a is not uncommon when Russian speakers write English. [1] "Definite and indefinite articles (corresponding to 'the', 'a', 'an' in English) do not exist in the Russian language. The sense conveyed by such articles can be determined in Russian by context." https://en.wikipedia.org/wiki/Russian_grammar https://en.wikipedia.org/wiki/Russian_grammar