5 ms·

Of course, but then you've hamstrung the typesystem that makes Rust safer than C, and generated a lot of work for developers who have to convert all internal ty

by gue5t 11y ago

Of course, but then you've hamstrung the typesystem that makes Rust safer than C, and generated a lot of work for developers who have to convert all internal types to #[repr(C)] ones at library boundaries. Rust without cross-crate generics is way less fun--you lose "theorems for free", unless you can make guarantees about the pre-ABI translation to/from C types (which would ultimately amount to defining an ABI).

dbaupp 11y ago
You've hamstring the type system at the interface, internally you still get all the benefits. In any case, a typical use-case for a shared library like this is embedded into other applications, which may not be written in Rust at all, and hence things like generics don't work anyway. In fact, this is reflected in a common way to expose an interface in this manner: have a main Rust interface (i.e. with normal generics etc.), with thin C-compatible shims that do the right set-up (e.g. converting arguments) to call that code. Also, I'm not exactly sure how you expect arbitrary cross-crate generics to work for a dynamically reloadable library: they're somewhat inherently incompatible (a downstream type can be inserted deeply into a library's code). I vaguely recall inklings of some approach for C++, beyond just explicitly instantiating specific template parameter combinations in the shared library, so if you know more about that, I'd love to here.
- gue5t 11y ago
  Personally, I've always thought that type safety provides the most benefit at boundaries between codebases: you have people composing pieces that they don't understand, and the types serve to guide how those pieces can be composed to result in a correct program. Within a library, it's not uncommon that the core data structure or processing is only correct because a human looked hard at it--I would be surprised if Rust implementations of, say, font rendering, used types internally to guarantee the correctness of their rendering relative to a formal specification. Instead they're using types to make sure they process all input, or make sure they handle memory safely. Generics are another place where this becomes the case: if a library exposes a "fold" function generic over both the accumulator and element type, accepting void* instead of T and U means you lose the ability to make sure user don't mix up places where you want an element and places where you want an accumulator. Dynamically upgradable cross-crate generics, ultimately, require recompilation if you want the performance benefits of monomorphization. But it would be possible to have patchable sites around inlined generics that the dynamic linker could change to instead jump to a version which did not inline generics and which simply called into a non-type-specialized routine in the other library. You'd end up with two versions of compiled code in the binary around each call site for a generic: an inlined-optimized-and-code-motioned one, as usual, and an assumption-free one compiled considering generics to be opaque blobs and using dynamic dispatch for trait methods so that the same code can be used for all instantiations of the generic function. For example, a function parameterized over an unconstrained type T can't do anything useful with values of that type, except dropping them. Thus code would be generated in which the call to the drop impl would be dynamic. I think this can be made to work w/r/t non-object-safe trait bounds as well, but I'm not sure how to justify that. I think library changes which alter which impls get called on which types (i.e. which affect specialization) would still be ABI-breaking, unless you had some more cleverness (type-id-based dispatch for the deoptimized case? ugh) to deal with that complexity. I also don't think this solution is ideal, but I think it's very important for Rust to be considering these issues, because given the design of the current modern operating system, having separately upgradable libraries is really important. Maybe Rust should be pushing for changes in the OS design, in a direction like NixOS is taking, but it seems like it isn't ready use for building the future's userspace when it can't provide both safety benefit across library boundaries and separate upgradeability.
  - dbaupp 11y ago
    > Personally, I've always thought that type safety provides the most benefit at boundaries between codebases: you have people composing pieces that they don't understand, and the types serve to guide how those pieces can be composed to result in a correct program. Yeah, type safety is definitely a big benefit when joining pieces together, but writing normal Rust code does this too, it's only the cross-language boundary that suffers (this is of course the most critical one, so having tools that avoid mistakes here is important, like autogenerating a C header). Joining random codebases together is especially common with cargo making it so easy to pull in dependencies. Please don't misinterpret me: I definitely don't disagree with you that having in-language type safety and generics are important, but those are fundamentally not things you can guarantee when you're doing things across languages. > Within a library, it's not uncommon that the core data structure or processing is only correct because a human looked hard at it--I would be surprised if Rust implementations of, say, font rendering, used types internally to guarantee the correctness of their rendering relative to a formal specification. Instead they're using types to make sure they process all input, or make sure they handle memory safely. I would hope/expect libraries to use types internally. That's the easiest way to have the computer stop you screwing up. E.g. the standard library does, the Vec type contains a RawVec to manage memory ( https://github.com/rust-lang/rust/blob/a212264011289885bdb8b2306a8fdd50e1f83c99/src/libcollections/vec.rs#L230 https://github.com/rust-lang/rust/blob/a212264011289885bdb8b... ) and similarly with HashMap ( https://github.com/rust-lang/rust/blob/a212264011289885bdb8b2306a8fdd50e1f83c99/src/libstd/collections/hash/map.rs#L339 https://github.com/rust-lang/rust/blob/a212264011289885bdb8b... ). > an assumption-free one compiled considering generics to be opaque blobs and using dynamic dispatch for trait methods so that the same code can be used for all instantiations of the generic function. > For example, a function parameterized over an unconstrained type T can't do anything useful with values of that type, except dropping them. Thus code would be generated in which the call to the drop impl would be dynamic. This sounds nice on the face of it... but it also sounds outlandishly complicated to implement. On the latter quote, a function can also move data around, and moving (unknown-size) data around seems difficult, e.g. what's an always-works impl of the following? fn push<T>(x: &mut Vec<T>, y: T) { let z = y; x.push(z) } Note, no bounds. Calling object safe methods (like a destructor) is the easiest part of implementing this. > I think this can be made to work w/r/t non-object-safe trait bounds as well, but I'm not sure how to justify that. I'd be... surprised; non-object safe traits suffer from all the problems of the `push` example, and more.

by gue5t 11y ago

dbaupp 11y ago

You've hamstring the type system at the interface, internally you still get all the benefits. In any case, a typical use-case for a shared library like this is embedded into other applications, which may not be written in Rust at all, and hence things like generics don't work anyway. In fact, this is reflected in a common way to expose an interface in this manner: have a main Rust interface (i.e. with normal generics etc.), with thin C-compatible shims that do the right set-up (e.g. converting arguments) to call that code. Also, I'm not exactly sure how you expect arbitrary cross-crate generics to work for a dynamically reloadable library: they're somewhat inherently incompatible (a downstream type can be inserted deeply into a library's code). I vaguely recall inklings of some approach for C++, beyond just explicitly instantiating specific template parameter combinations in the shared library, so if you know more about that, I'd love to here.

gue5t 11y ago
Personally, I've always thought that type safety provides the most benefit at boundaries between codebases: you have people composing pieces that they don't understand, and the types serve to guide how those pieces can be composed to result in a correct program. Within a library, it's not uncommon that the core data structure or processing is only correct because a human looked hard at it--I would be surprised if Rust implementations of, say, font rendering, used types internally to guarantee the correctness of their rendering relative to a formal specification. Instead they're using types to make sure they process all input, or make sure they handle memory safely. Generics are another place where this becomes the case: if a library exposes a "fold" function generic over both the accumulator and element type, accepting void* instead of T and U means you lose the ability to make sure user don't mix up places where you want an element and places where you want an accumulator. Dynamically upgradable cross-crate generics, ultimately, require recompilation if you want the performance benefits of monomorphization. But it would be possible to have patchable sites around inlined generics that the dynamic linker could change to instead jump to a version which did not inline generics and which simply called into a non-type-specialized routine in the other library. You'd end up with two versions of compiled code in the binary around each call site for a generic: an inlined-optimized-and-code-motioned one, as usual, and an assumption-free one compiled considering generics to be opaque blobs and using dynamic dispatch for trait methods so that the same code can be used for all instantiations of the generic function. For example, a function parameterized over an unconstrained type T can't do anything useful with values of that type, except dropping them. Thus code would be generated in which the call to the drop impl would be dynamic. I think this can be made to work w/r/t non-object-safe trait bounds as well, but I'm not sure how to justify that. I think library changes which alter which impls get called on which types (i.e. which affect specialization) would still be ABI-breaking, unless you had some more cleverness (type-id-based dispatch for the deoptimized case? ugh) to deal with that complexity. I also don't think this solution is ideal, but I think it's very important for Rust to be considering these issues, because given the design of the current modern operating system, having separately upgradable libraries is really important. Maybe Rust should be pushing for changes in the OS design, in a direction like NixOS is taking, but it seems like it isn't ready use for building the future's userspace when it can't provide both safety benefit across library boundaries and separate upgradeability.
- dbaupp 11y ago
  > Personally, I've always thought that type safety provides the most benefit at boundaries between codebases: you have people composing pieces that they don't understand, and the types serve to guide how those pieces can be composed to result in a correct program. Yeah, type safety is definitely a big benefit when joining pieces together, but writing normal Rust code does this too, it's only the cross-language boundary that suffers (this is of course the most critical one, so having tools that avoid mistakes here is important, like autogenerating a C header). Joining random codebases together is especially common with cargo making it so easy to pull in dependencies. Please don't misinterpret me: I definitely don't disagree with you that having in-language type safety and generics are important, but those are fundamentally not things you can guarantee when you're doing things across languages. > Within a library, it's not uncommon that the core data structure or processing is only correct because a human looked hard at it--I would be surprised if Rust implementations of, say, font rendering, used types internally to guarantee the correctness of their rendering relative to a formal specification. Instead they're using types to make sure they process all input, or make sure they handle memory safely. I would hope/expect libraries to use types internally. That's the easiest way to have the computer stop you screwing up. E.g. the standard library does, the Vec type contains a RawVec to manage memory ( https://github.com/rust-lang/rust/blob/a212264011289885bdb8b2306a8fdd50e1f83c99/src/libcollections/vec.rs#L230 https://github.com/rust-lang/rust/blob/a212264011289885bdb8b... ) and similarly with HashMap ( https://github.com/rust-lang/rust/blob/a212264011289885bdb8b2306a8fdd50e1f83c99/src/libstd/collections/hash/map.rs#L339 https://github.com/rust-lang/rust/blob/a212264011289885bdb8b... ). > an assumption-free one compiled considering generics to be opaque blobs and using dynamic dispatch for trait methods so that the same code can be used for all instantiations of the generic function. > For example, a function parameterized over an unconstrained type T can't do anything useful with values of that type, except dropping them. Thus code would be generated in which the call to the drop impl would be dynamic. This sounds nice on the face of it... but it also sounds outlandishly complicated to implement. On the latter quote, a function can also move data around, and moving (unknown-size) data around seems difficult, e.g. what's an always-works impl of the following? fn push<T>(x: &mut Vec<T>, y: T) { let z = y; x.push(z) } Note, no bounds. Calling object safe methods (like a destructor) is the easiest part of implementing this. > I think this can be made to work w/r/t non-object-safe trait bounds as well, but I'm not sure how to justify that. I'd be... surprised; non-object safe traits suffer from all the problems of the `push` example, and more.