6 ms·

this is an exciting development for Postgres. Since PL/Rust is now a trusted language, it means that cloud providers like RDS and Supabase will be able to provi

this is an exciting development for Postgres. Since PL/Rust is now a trusted language, it means that cloud providers like RDS and Supabase will be able to provide it. This means that you can write your database functions in rust, as an alternative to pgplsql / plv8. (disclosure: i work at supabase)

kiwicopple 4y ago
for those who want to know the difference between "trusted" and "untrusted": https://tcdi.github.io/plrust/trusted-untrusted.html https://tcdi.github.io/plrust/trusted-untrusted.html _Normally, PL/Rust is installed as a "trusted" programming language named plrust. In this setup, certain Rust and pgx operations are disabled to preserve security. In general, the operations that are restricted are those that interact with the environment. This includes file handle operations, require, and use (for external modules). There is no way to access internals of the database server process or to gain OS-level access with the permissions of the server process, as a C function can do. Thus, any unprivileged database user can be permitted to use this language._ Languages like pl_python are "untrusted" and give too much access to the file system, which is why cloud providers never support them on their platforms
- zabzonk 4y ago
  cannot rust functions do anything a c function can?
  - lexicality 4y ago
    only when using `unsafe` which this forbids
    - tyingq 4y ago
      I don't know much about rust, so it seemed odd to me that just forbidding unsafe would make for a reasonable sandbox. At least for the postgres concept of "trusted language", that's supposed to mean things like closing off access to the filesystem. Not just typical file io calls either, but more obscure stuff like sendfile(). I do see that rust access to sendfile() would be via a syscall, which is in the unsafe category...so perhaps that's not the best example. But it does make me curious how comprehensive a sandbox PL/Rust is providing, beyond just forbidding unsafe.
      - iudqnolq 4y ago
        pl/rust is its own target platform, so they provide their own standard library*. The IO stuff all panics, for example. (In Rust a panic should be used to indicate invariant failure. pl/rust catches panics and converts them to postgres errors). In Rust you'd normally be able to link c code, but calling c requires unsafe because you have to manually ensure the c code upholds any relevant Rust invariants. > But it does make me curious how comprehensive a sandbox PL/Rust is providing, beyond just forbidding unsafe. They also hook the compiler and try and detect shenanigans. It's not perfect, but it's pretty thought out. *Technically the Rust standard library builds on top of a lower-level io module, which is all you have to replace.
      - gpm 4y ago
        More than just forbidding unsafe, but not enough to make this secure against competent adversaries by their own admission. They argue postgres itself isn't secure against competent adversaries so this doesn't matter too much. https://tcdi.github.io/plrust/plrust.html#what-about-rust-compiler-bugs https://tcdi.github.io/plrust/plrust.html#what-about-rust-co... Rust keeps a list of soundness bugs via a tag on github - they're pretty common: https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AI-unsound https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais...
      - yarg 4y ago
        Read the article, they specify that they use a compiler target that restricts system access. As I read it, it seemed that the unsafe prohibition was more about the safety of the code and not the security of the system.
    - formalLogicGirl 4y ago
      And there is also an alternative implementation to Rust std that blocks, among others, filesystem access via std::fs https://github.com/tcdi/postgrestd https://github.com/tcdi/postgrestd
    - chrismorgan 4y ago
      This implementation blocks file system access and is thus not vulnerable, but note that in Rust in general you can actually violate safety on some platforms without any `unsafe` by modifying magic files like /proc/self/mem. This is a known issue but considered unfixable (because the technical solution of marking opening a file as unsafe would cause far more trouble than it could ever hope to solve).
      - IshKebab 4y ago
        It's more of an interesting fact than an issue to be fixed.
  - ekimekim 4y ago
    Normally yes, but it looks like the trusted PL/Rust being discussed here is limited to some subset of Rust. They specifically note that `unsafe` code is not allowed, which means you can't (for example) implement your own syscalls or construct a pointer into postgres internals memory. However, they make it clear that this is not intended to be your only defence against an attacker: > Note that this is done on a best-effort basis, and does not provide a strong level of security — it's not a sandbox, and as such, it's likely that a skilled hostile attacker who is sufficiently motivated could find ways around it
    - IshKebab 4y ago
      I don't think it's really any defence at all against an attacker. I don't think Rust's `unsafe` was ever intended to be a security boundary. Surely all a "sufficiently motivated" attacker would need to do is peruse the unsound bugs on GitHub? https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AI-unsound https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais... Those aren't considered to be security issues. Makes me wonder what the point of banning `unsafe` is at all. You're going to need some other system anyway...
      - feike 4y ago
        The trust is not just banning unsafe, it is using a limited std: > The "trusted" version of PL/Rust uses a unique fork of Rust's std entitled postgrestd when compiling LANGUAGE plrust user functions. https://github.com/tcdi/postgrestd https://github.com/tcdi/postgrestd
        IshKebab 4y ago
        That doesn't matter. If you can use `unsafe` you can make syscalls directly.
        ekimekim 4y ago
        A large number of unsoundness bugs only work if you have access to the stdlib, because they're flaws in stdlib types and functions that use `unsafe` internally, and are supposed to present a safe interface around it. If you a) don't have access to unsafe, and b) don't have access to the stdlib that lets you do powerful things without unsafe, then you're very limited in what you can do. https://smallcultfollowing.com/babysteps/blog/2016/10/02/observational-equivalence-and-unsafe-code/ https://smallcultfollowing.com/babysteps/blog/2016/10/02/obs... discusses this further. Conceptually, you can think of "entirely Safe Rust" to be a very limited language, which you then progressively add "capabilites" to by exposing safe interfaces implemented with unsafe code. For example, Vec and Box (which require unsafe) grant safe code the ability to do heap allocations. It's true that this is not designed as a security boundary. As I note in my comment above, the PL/Rust devs also make that clear. That doesn't mean it has no value as part of a defence in depth strategy.
  - deleted 4y ago
    [deleted]
techdragon 4y ago
Can… but how long until they actually do provide it? I’m a big fan of PostgreSQL and it’s constantly an annoyance to find cool new capabilities provided by extensions I can never use since I’m not going to manage my own database in a critical environment for a lot of reasons… I’ve done it before, I know how hard it is to do well, and I don’t want this to be my job anymore… so when I find cool stuff like vector search or graph traversals but can never use them it’s just a constant disappointment. Does this “trusted” state actually translate into greater adoption by cloud providers or is it just something the developers behind this effort hope will happen?
- kiwicopple 4y ago
  We are launching something with the RDS team this week that will make it very easy to install your own extensions that are written in Trusted Languages. We will share more on Friday. In terms of pl/rust becoming available on RDS and supabase: months. It’s going through security audits now
  - ttfkam 4y ago
    It will be very hard for other database engines to keep up if this gets implemented by cloud vendors. MySQL would get marginalized further and further not just in the relative lack of SQL features, but all the easily available add-ons the community will inevitably write for Postgres with no viable answer from its competition.
  - techdragon 4y ago
    Wonderful! I’ll be checking for news on Friday. Thanks for replying, otherwise I’d probably not have noticed till a couple months from now when I next check the AWS service feature release news.
- rywalker 4y ago
  pl/rust will be available on CoreDB when we launch, along many other popular and interesting Postgres extensions. We’re a new company, closed beta - but join our waitlist @ https://coredb.io https://coredb.io if you’d like a shot at early access
Cyphase 4y ago
Thank you for saying "disclosure" instead of "disclaimer".
- robertlagrant 4y ago
  Disclaimer: I know what I'm talking about.
mike_hearn 4y ago
But bear in mind, those providers generally don't provide all trusted languages out there already. PL/Java has existed for a long time, has a trusted mode, and isn't supported by Supabase for example.

kiwicopple 4y ago

for those who want to know the difference between "trusted" and "untrusted": https://tcdi.github.io/plrust/trusted-untrusted.html https://tcdi.github.io/plrust/trusted-untrusted.html _Normally, PL/Rust is installed as a "trusted" programming language named plrust. In this setup, certain Rust and pgx operations are disabled to preserve security. In general, the operations that are restricted are those that interact with the environment. This includes file handle operations, require, and use (for external modules). There is no way to access internals of the database server process or to gain OS-level access with the permissions of the server process, as a C function can do. Thus, any unprivileged database user can be permitted to use this language._ Languages like pl_python are "untrusted" and give too much access to the file system, which is why cloud providers never support them on their platforms

zabzonk 4y ago
cannot rust functions do anything a c function can?
- lexicality 4y ago
  only when using `unsafe` which this forbids
  - tyingq 4y ago
    I don't know much about rust, so it seemed odd to me that just forbidding unsafe would make for a reasonable sandbox. At least for the postgres concept of "trusted language", that's supposed to mean things like closing off access to the filesystem. Not just typical file io calls either, but more obscure stuff like sendfile(). I do see that rust access to sendfile() would be via a syscall, which is in the unsafe category...so perhaps that's not the best example. But it does make me curious how comprehensive a sandbox PL/Rust is providing, beyond just forbidding unsafe.
    - iudqnolq 4y ago
      pl/rust is its own target platform, so they provide their own standard library*. The IO stuff all panics, for example. (In Rust a panic should be used to indicate invariant failure. pl/rust catches panics and converts them to postgres errors). In Rust you'd normally be able to link c code, but calling c requires unsafe because you have to manually ensure the c code upholds any relevant Rust invariants. > But it does make me curious how comprehensive a sandbox PL/Rust is providing, beyond just forbidding unsafe. They also hook the compiler and try and detect shenanigans. It's not perfect, but it's pretty thought out. *Technically the Rust standard library builds on top of a lower-level io module, which is all you have to replace.
    - gpm 4y ago
      More than just forbidding unsafe, but not enough to make this secure against competent adversaries by their own admission. They argue postgres itself isn't secure against competent adversaries so this doesn't matter too much. https://tcdi.github.io/plrust/plrust.html#what-about-rust-compiler-bugs https://tcdi.github.io/plrust/plrust.html#what-about-rust-co... Rust keeps a list of soundness bugs via a tag on github - they're pretty common: https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AI-unsound https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais...
    - yarg 4y ago
      Read the article, they specify that they use a compiler target that restricts system access. As I read it, it seemed that the unsafe prohibition was more about the safety of the code and not the security of the system.
  - formalLogicGirl 4y ago
    And there is also an alternative implementation to Rust std that blocks, among others, filesystem access via std::fs https://github.com/tcdi/postgrestd https://github.com/tcdi/postgrestd
  - chrismorgan 4y ago
    This implementation blocks file system access and is thus not vulnerable, but note that in Rust in general you can actually violate safety on some platforms without any `unsafe` by modifying magic files like /proc/self/mem. This is a known issue but considered unfixable (because the technical solution of marking opening a file as unsafe would cause far more trouble than it could ever hope to solve).
    - IshKebab 4y ago
      It's more of an interesting fact than an issue to be fixed.
- ekimekim 4y ago
  Normally yes, but it looks like the trusted PL/Rust being discussed here is limited to some subset of Rust. They specifically note that `unsafe` code is not allowed, which means you can't (for example) implement your own syscalls or construct a pointer into postgres internals memory. However, they make it clear that this is not intended to be your only defence against an attacker: > Note that this is done on a best-effort basis, and does not provide a strong level of security — it's not a sandbox, and as such, it's likely that a skilled hostile attacker who is sufficiently motivated could find ways around it
  - IshKebab 4y ago
    I don't think it's really any defence at all against an attacker. I don't think Rust's `unsafe` was ever intended to be a security boundary. Surely all a "sufficiently motivated" attacker would need to do is peruse the unsound bugs on GitHub? https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3AI-unsound https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais... Those aren't considered to be security issues. Makes me wonder what the point of banning `unsafe` is at all. You're going to need some other system anyway...
    - feike 4y ago
      The trust is not just banning unsafe, it is using a limited std: > The "trusted" version of PL/Rust uses a unique fork of Rust's std entitled postgrestd when compiling LANGUAGE plrust user functions. https://github.com/tcdi/postgrestd https://github.com/tcdi/postgrestd
      - IshKebab 4y ago
        That doesn't matter. If you can use `unsafe` you can make syscalls directly.
        ekimekim 4y ago
        A large number of unsoundness bugs only work if you have access to the stdlib, because they're flaws in stdlib types and functions that use `unsafe` internally, and are supposed to present a safe interface around it. If you a) don't have access to unsafe, and b) don't have access to the stdlib that lets you do powerful things without unsafe, then you're very limited in what you can do. https://smallcultfollowing.com/babysteps/blog/2016/10/02/observational-equivalence-and-unsafe-code/ https://smallcultfollowing.com/babysteps/blog/2016/10/02/obs... discusses this further. Conceptually, you can think of "entirely Safe Rust" to be a very limited language, which you then progressively add "capabilites" to by exposing safe interfaces implemented with unsafe code. For example, Vec and Box (which require unsafe) grant safe code the ability to do heap allocations. It's true that this is not designed as a security boundary. As I note in my comment above, the PL/Rust devs also make that clear. That doesn't mean it has no value as part of a defence in depth strategy.
- deleted 4y ago
  [deleted]

techdragon 4y ago

Can… but how long until they actually do provide it? I’m a big fan of PostgreSQL and it’s constantly an annoyance to find cool new capabilities provided by extensions I can never use since I’m not going to manage my own database in a critical environment for a lot of reasons… I’ve done it before, I know how hard it is to do well, and I don’t want this to be my job anymore… so when I find cool stuff like vector search or graph traversals but can never use them it’s just a constant disappointment. Does this “trusted” state actually translate into greater adoption by cloud providers or is it just something the developers behind this effort hope will happen?

kiwicopple 4y ago
We are launching something with the RDS team this week that will make it very easy to install your own extensions that are written in Trusted Languages. We will share more on Friday. In terms of pl/rust becoming available on RDS and supabase: months. It’s going through security audits now
- ttfkam 4y ago
  It will be very hard for other database engines to keep up if this gets implemented by cloud vendors. MySQL would get marginalized further and further not just in the relative lack of SQL features, but all the easily available add-ons the community will inevitably write for Postgres with no viable answer from its competition.
- techdragon 4y ago
  Wonderful! I’ll be checking for news on Friday. Thanks for replying, otherwise I’d probably not have noticed till a couple months from now when I next check the AWS service feature release news.
rywalker 4y ago
pl/rust will be available on CoreDB when we launch, along many other popular and interesting Postgres extensions. We’re a new company, closed beta - but join our waitlist @ https://coredb.io https://coredb.io if you’d like a shot at early access

Cyphase 4y ago

Thank you for saying "disclosure" instead of "disclaimer".

robertlagrant 4y ago
Disclaimer: I know what I'm talking about.

mike_hearn 4y ago

But bear in mind, those providers generally don't provide all trusted languages out there already. PL/Java has existed for a long time, has a trusted mode, and isn't supported by Supabase for example.