8 ms·
Improving Interoperability Between Rust and C++
- hintoftime 3y agoThis might finally be the push for me to learn Rust. I'm primarily a Python developer working on Django, but I used to do embedded development in C, and Operating Systems with Remzi at UW Madison was my favorite class. It got me excited about CS. With Google and industry backing, it seems like Rust is here to stay.
- wongarsu 3y agoIt's worth mentioning that Rust-Python interoperability is already great, in both directions, thanks to pyo3 https://github.com/PyO3/pyo3 https://github.com/PyO3/pyo3
- hintoftime 3y agoYeah, PyO3 is great. I've tried to play around with releasing the GIL from rust in Python 3.12. I would enjoy writing a WSGI/ASGI server with a Celery runtime at some point too. Or contribute to Granian. https://github.com/emmett-framework/granian https://github.com/emmett-framework/granian
- abdullahkhalids 3y agoPyO3 is great. But it needs more and better documentation. I recently did use it for a project, and it was quite a lot of time and pain guessing what would work and what wouldn't.
- MSFT_Edging 3y agoSeconding this. I learned a little rust and really liked its philosophy. I was then working on a small project that needed to avoid the big data science python libraries because they're miserable in terms of portability to air-gapped networks. We had re-implemented a few mathematical and curve finding functions and even with the newest python they were uncomfortably slow. PyO3 made it so easy to implement a runtime switchable rust library exposed to python, it was almost unnerving. By writing those handful of functions in rust we got something like a 30x speedup. Note this was a small project with limited funding for r&d so the level of effort for performance speedup was really nice.
- dralley 3y agoWhy not compile those (python scientific) libraries from source? Why would PyO3 be acceptable while e.g. numpy is not?
- MSFT_Edging 3y agoneeding to compile them on the airgapped machine introduces more dependencies. Things also get weird when your airgapped machine is a slight version off from your dev machine. There's a lot of ways around it but we've found for the size of this tool, it was preferable to just write our own handful of math functions over building out a deployment method for extra dependencies.
- jll29 3y agoI think Rust is here to stay, but I wish folks had designed a systematic standard library (as Java has) rather than let a set of crates evolve uncontrolledly. Hopefully, the language will develop further and then reach ISO standardization - which Java cannot as it remains proprietary.
- steveklabnik 3y agoI would not hold your breath for an ISO standard, ever. MAYBE a Ruby-style "standard dump," but I don't see why such a thing would ever be useful.
- jasonpeacock 3y agoHere's a good article about why Rust has a small stdlib: https://blog.nindalf.com/posts/rust-stdlib/ https://blog.nindalf.com/posts/rust-stdlib/
- jasonpeacock 3y agoWhy does Rust need ISO standardization - what is blocked by the lack of ISO standard?
- summerlight 3y agoI guess the GP didn't mean to point to ISO as an organization specifically, but overall standardization/specification efforts.
- jasonpeacock 3y agoStill, what is blocked by lack of standardization or specification? Many (all?) languages were successful before being fully specified, and many still are not (cough Python cough) yet that has not impacted their adoption, popularity, or effectiveness.
- steveklabnik 3y agoMany people misunderstand how software is written in regulated industries, and assume that a standard is necessary. In practice, this is not the case. Note that Ferrocene[1] had to produce a specification[2] in order to qualify the compiler. But there isn't a requirement that it must be a standard in any way, only that it describes how the Ferrocene compiler works. Nor that it be accepted by upstream. 1: https://ferrous-systems.com/blog/officially-qualified-ferrocene/ https://ferrous-systems.com/blog/officially-qualified-ferroc... 2: https://github.com/ferrocene/specification https://github.com/ferrocene/specification
- wegfawefgawefg 3y agoDoing ffi from c to python is easy. From rust to python is annoying. Have to use unsafe, and weird parts of the rust stl. funky refs. Random crashes! oi vey.
- shadowgovt 3y agoThe followup question is "Is it easy in C because it's actually easier or because the result is code that has a half-dozen hidden undefined-behavior mines that C does a poor job of detecting at compile time?"
- wegfawefgawefg 3y agoRust isnt safe across the ffi barrier, so its not a fair criticism of c. go check it out.
- shadowgovt 3y agoIt isn't safe, true. Is it, on average, safer?
- wegfawefgawefg 3y agoIf i remember correctly it can even be worse.
- Shawnecy 3y agoNot my experience at all. At work we rewrote a small bit of hotspot python in Rust with no issues. This was what we primarily followed: https://ohadravid.github.io/posts/2023-03-rusty-python/ https://ohadravid.github.io/posts/2023-03-rusty-python/
- gnatolf 3y agoThank you for that great link!
- digdugdirk 3y agoIt blows my mind that Google is celebrating a 1 million dollar grant to something that serves as a cornerstone of every aspect of the future of their business. $1 mil is less than a rounding error for an organization like that. I'd love to know how much they've burned on the numerous projects that occupy their product graveyard, and imagine a future where that was devoted to the growth and education surrounding core language development in every language - Rust, Go, C++, Python, etc.
- erdaniels 3y agoHow much do you think they should allocate to rust?
- refulgentis 3y ago4.5 million is a good start for a project of this magnitude affecting this much of Google. Back 5-10 years ago Google tended to calculate opportunities in SWE-years. Roughly $300K/engineer. So I'm asking for 5 engineers for 3 years.
- summerlight 3y agoI think there's a dedicated Rust team in Google working on interop between C++/Rust. If you count this as well, it would be close to that number.
- MuffinFlavored 3y ago> $1 mil is less than a rounding error for an organization like that. That's 4 $200k/yr engineers working on this full time and then 1 $200k/yr manager to keep them on track.
- refulgentis 3y agoFor a year only?
- gdcbe 3y ago
- sho_hn 3y agoNice, I guess, but I had hoped for more technical particulars rather than just "we're trying to throw money it and hope it gets better". What will the money actually fund? What are the conditions and KPIs of the grant? Are they putting their weight behind any preferred approach? Edit: The RF announcement includes more information: https://foundation.rust-lang.org/news/google-contributes-1m-to-rust-foundation-to-support-c-rust-interop-initiative/ https://foundation.rust-lang.org/news/google-contributes-1m-... "The Rust Foundation’s first Interop Initiative task will be to draft a scope of work proposal for discussion amongst our team members, the Rust Project Leadership Council, Rust Project stakeholders, relevant Rust Foundation member organizations, and its board. [...] Recommendations will likely include the hiring of one or more Interop Initiative engineers and may include the provision of resources towards expanding on existing interoperability work, build system integration, using AI for C++ to Rust conversion, or some combination of all these. The Foundation will engage the appropriate stakeholders across the Rust Project and its member base to review the proposal and carve the path forward for this important work."
- TillE 3y agoCarbon's approach is to integrate with LLVM internals (particularly the clang AST) and interoperate at that level. I guess Rust could do something similar, but probably with limitations since Rust isn't explicitly designed as a C++ successor language.
- pjmlp 3y agoIn any case Carbon is a research project, even if the Internets give it more spotlight than the authors themselves.
- myth17 3y agoC++ and RUST interop with async is very nascent. Hope this helps to make improvements in this space.
- cod1r 3y agoI'm loving that Rust is getting more and more support
- cmrdporcupine 3y agoInteresting. I think I wouldn't have quit Google 2 years ago if I'd been able to get myself on a team doing Rust dev. Such a team did not exist at Waterloo though.
- Ericson2314 3y agoIf there really serious about this, they'll realize they need to add erased but not specialized types to rust, and existential types. If you can replace void qsort(void base[.size * .nmemb], size_t nmemb, size_t size, int (*compar)(const void [.size], const void [.size])) with extern "C" fn qsort<erase T>(base: *const T, nmemb: usize, compar: fn(*const T, *const T) -> std::ffi::c_int) such that `qsort` is still single a regular C function, not something that corresponds to many separate C functions, many good things are possible. In particular for C++, being able to do struct MyVTable<erase T> { virt_method_0: fn(*const T) -> bool, virt_method_2: fn(*const T, usize) -> (usize, usize), }; struct MyType<erase R> { vtable: *const MyVTable<MyType<R>>, rest_of_fields: R, }; Opens a lot of doors. People while whine because it is exposing the C++ ABI in the Rust API, but that's fine. That's what `private` is there to fix.
- eximius 3y agoWhat does `erase T` mean here? "We don't deref it so don't worry about the type, just erase it?"
- jcranmer 3y agoNormally, if you have generics, Rust makes copies of the function (/struct/enum/etc.) (aka "monomorphization"), one for each type that you use with it. The idea is to have a generic function but only one copy of the function, presumably because none of the code actually relies on the details of T.
- TheCoreh 3y agoIsn't this just `&dyn Trait`? Would it be a stack allocated, owning version of that, that can be moved? A pure `dyn Trait`?
- Ericson2314 3y agoNo. &mut dyn Foo is sugar for something like struct DynFoo<erase T> { vtable: &mut VTableFoo<T> ptr: &mut T } This is the underlying feature. `dyn` is a bit of an ad-hoc hack around not having this feature.
- ogoffart 3y agoI am the current passive maintainer of the cpp crate: https://crates.io/crates/cpp https://crates.io/crates/cpp / https://github.com/mystor/rust-cpp https://github.com/mystor/rust-cpp I'm a bit disappointed that cxx gets all the glory and nobody likes the approach of the cpp crate.
- rapsey 3y ago4M downloads of your crate is quite an achievement.
- operator-name 3y agoAnnouncement from the Rust side: https://news.ycombinator.com/item?id=39263750 https://news.ycombinator.com/item?id=39263750
- senkora 3y agoI imagine that for Google, one path that would make sense would be 1) upgrading their C++ to Carbon and then 2) making sure that the Rust + Carbon interoperability is solid. Of course, in a company of Google’s size, I’m sure different groups are pushing all reasonable options.
- pjmlp 3y agoCarbon is a research project, whose authors are the first to say use Rust if you can.
- gumby 3y agoIf they want googlers to program with one hand metaphorically tied behind the back they already have go for that purpose.
- IshKebab 3y agoThis would be nice, but the biggest pain I had with C++ is you can't use C++ types as values, e.g. as described here: https://cxx.rs/binding/cxxstring.html https://cxx.rs/binding/cxxstring.html It's because Rust doesn't have move constructors. I doubt they're going to add move constructors to Rust so that seems kind of unsolvable which is a shame.