4 ms·

> What I'd really like to see, though, is a language with parameterized typing with hybrid static+runtime enforcement. You should be able to be holding a raw ad

> What I'd really like to see, though, is a language with parameterized typing with hybrid static+runtime enforcement. You should be able to be holding a raw address in memory, and then have that become one of two strong types at the language level depending on the whether that memory address is nonzero—"lifting" the NULL check into the type system, rather than replacing the NULL check with extra type annotation. In Rust, the compiler optimizes pointer types wrapped in Options to nullable pointers. Is this the kind of thing you meant?

derefr 11y ago
Possibly—does the Rust FFI allow you to import a C library, writing its foreign-function declarations such that the nullable pointers it returns are transformed into, from Rust's perspective, Option-typed non-nullable pointers, with no runtime memory overhead? And then, if so, does Rust further guarantee that those "virtual" Option-typed pointers can be passed back to the same API "just the way it sent them"? That's sort of what I mean, here: the nullable ptr as the "canonical, machine-level" representation, with the type serving as a language-level abstraction, effectively a "lie" because it has no real effect on the type system, and only serves as syntax sugar for generating runtime checks.
- johncolanduoni 11y ago
  The Rust FFI takes all pointers as "raw pointers". The reason for this is that Rust's various safe pointer types have a lot of extra conditions on them to enforce memory/data race safety without garbage collection. For example, some pointers are uniquely owned and will be freed when they go out of scope, some can't be stored because they are "borrowed", some can't be mutated because they're shared etc. When Rust gets a pointer from another language like C (that obviously doesn't have such semantics), whoever is writing the FFI has to decide what category these raw pointers fall under before passing them back to "safe" code. Depending on the type of semantics you are attaching to the pointer, you can use something like this[1] function, which does exactly what you're asking. It is a runtime no-op, but it coerces the unsafe pointer type so that memory safe code can use it (based on your word that it can be interpreted in this way). [1]: https://doc.rust-lang.org/std/primitive.pointer.html#method.as_ref https://doc.rust-lang.org/std/primitive.pointer.html#method....
  - derefr 11y ago
    To be clear, I wasn't so much imagining using the FFI declaration to vouchsafe the opaque handle into a completely Rust-nativized memory-safe owned reference; but rather, as taking the opaque nullable pointer from the FFI, and unwrapping it exactly one level into an Option<opaque non-nullable pointer>—that is, turning a nullable FFI "handle" into an Option<non-nullable FFI "handle">. Such an FFI function declaration really can't require that the thing returned inside the Option be a dereference of the pointer: in the happy case, it's still an opaque FFI handle, so dereferencing it just exposes an opaque struct you shouldn't touch; and in the unhappy case, it's still an invalid pointer, just one that's invalid because somebody else (like the library that gave it to you) freed it. The raw pointer itself, even if non-NULL, is still "tainted"; you still want to apply manual runtime checks to it before casting it to a Rust reference. (And, past that, in both cases you still probably want to deal with the pointer mostly just by passing it back to other FFI functions from the same library, treating the pointer itself as an identifier, a key the API associates with something.) I'm emphasizing this distinction because the NULLness of the pointer is, from what I can see, "outside" the lifetime of the reference, and even outside the "identifier-ness" of the handle. Knowing the validity of the pointer requires knowing about ownership/lifetime; the pointer might point to memory that has been free()ed. But you can know that the pointer is intentionally NULL (and therefore not a valid handle/identifier from the FFI's perspective) without attempting to construct a reference. Effectively, a [nullable] raw pointer is the moral equivalent of a tagged data structure. There's a piece of metadata you can get from the tag: when it's zero, the contained raw pointer is intentionally invalid; when it's nonzero, the contained raw pointer is only maybe-invalid. You can destructure and act on the "tag" without doing anything else to the [non-nullable] raw pointer "stored inside". I wouldn't expect most languages to care about a distinction this fine, because if a language has references, it really only wants to think about "pointers: maybe invalid" and "references: always valid". But Rust is in the unique position of trying to work "directly with C" without the impedance mismatch of glue like SWIG. And C provides many libraries that return nominally opaque handles, but expect their consumers to notice if those handles are NULL (which happens often, when e.g. a factory-function from such a library is called with NULL arguments, or unexpectedly receives a NULL from a library it itself consumes.)
    - 11y ago

derefr 11y ago

Possibly—does the Rust FFI allow you to import a C library, writing its foreign-function declarations such that the nullable pointers it returns are transformed into, from Rust's perspective, Option-typed non-nullable pointers, with no runtime memory overhead? And then, if so, does Rust further guarantee that those "virtual" Option-typed pointers can be passed back to the same API "just the way it sent them"? That's sort of what I mean, here: the nullable ptr as the "canonical, machine-level" representation, with the type serving as a language-level abstraction, effectively a "lie" because it has no real effect on the type system, and only serves as syntax sugar for generating runtime checks.

johncolanduoni 11y ago
The Rust FFI takes all pointers as "raw pointers". The reason for this is that Rust's various safe pointer types have a lot of extra conditions on them to enforce memory/data race safety without garbage collection. For example, some pointers are uniquely owned and will be freed when they go out of scope, some can't be stored because they are "borrowed", some can't be mutated because they're shared etc. When Rust gets a pointer from another language like C (that obviously doesn't have such semantics), whoever is writing the FFI has to decide what category these raw pointers fall under before passing them back to "safe" code. Depending on the type of semantics you are attaching to the pointer, you can use something like this[1] function, which does exactly what you're asking. It is a runtime no-op, but it coerces the unsafe pointer type so that memory safe code can use it (based on your word that it can be interpreted in this way). [1]: https://doc.rust-lang.org/std/primitive.pointer.html#method.as_ref https://doc.rust-lang.org/std/primitive.pointer.html#method....
- derefr 11y ago
  To be clear, I wasn't so much imagining using the FFI declaration to vouchsafe the opaque handle into a completely Rust-nativized memory-safe owned reference; but rather, as taking the opaque nullable pointer from the FFI, and unwrapping it exactly one level into an Option<opaque non-nullable pointer>—that is, turning a nullable FFI "handle" into an Option<non-nullable FFI "handle">. Such an FFI function declaration really can't require that the thing returned inside the Option be a dereference of the pointer: in the happy case, it's still an opaque FFI handle, so dereferencing it just exposes an opaque struct you shouldn't touch; and in the unhappy case, it's still an invalid pointer, just one that's invalid because somebody else (like the library that gave it to you) freed it. The raw pointer itself, even if non-NULL, is still "tainted"; you still want to apply manual runtime checks to it before casting it to a Rust reference. (And, past that, in both cases you still probably want to deal with the pointer mostly just by passing it back to other FFI functions from the same library, treating the pointer itself as an identifier, a key the API associates with something.) I'm emphasizing this distinction because the NULLness of the pointer is, from what I can see, "outside" the lifetime of the reference, and even outside the "identifier-ness" of the handle. Knowing the validity of the pointer requires knowing about ownership/lifetime; the pointer might point to memory that has been free()ed. But you can know that the pointer is intentionally NULL (and therefore not a valid handle/identifier from the FFI's perspective) without attempting to construct a reference. Effectively, a [nullable] raw pointer is the moral equivalent of a tagged data structure. There's a piece of metadata you can get from the tag: when it's zero, the contained raw pointer is intentionally invalid; when it's nonzero, the contained raw pointer is only maybe-invalid. You can destructure and act on the "tag" without doing anything else to the [non-nullable] raw pointer "stored inside". I wouldn't expect most languages to care about a distinction this fine, because if a language has references, it really only wants to think about "pointers: maybe invalid" and "references: always valid". But Rust is in the unique position of trying to work "directly with C" without the impedance mismatch of glue like SWIG. And C provides many libraries that return nominally opaque handles, but expect their consumers to notice if those handles are NULL (which happens often, when e.g. a factory-function from such a library is called with NULL arguments, or unexpectedly receives a NULL from a library it itself consumes.)
  - 11y ago