13 ms·

A New AWS SDK for Rust

219 points by carllerche 5y ago

RcouF1uZ4gsC 5y ago
This is huge for Rust. With Microsoft Windows SDK and now with this, Rust is starting to get first party support on platforms. More so than language syntax, the driver of adoption of a language is ecosystem. Rust is rapidly building up a very impressive ecosystem around it.
ibraheemdev 5y ago
As good as rusoto is, an official SDK is a huge step forward. Many were wondering how AWS could be "commited to Rust" but not provide an official SDK, this looks great! > We are exploring ways to support multiple Rust async runtimes. Looking to be runtime agnostic from the start is a good sign, although it can be challenging in async rust's current state. It looks like they aren't tied into much async IO, so it should be relatively easy for this library.
- uulu 5y ago
  > As good as rusoto is, an official SDK is a huge step forward. Yes. It might not be "a huge step forward" in tech terms, but it would be definitely very positive for wider adoption, especially from "enterprises" where all is based on "standards".
  - staticassertion 5y ago
    TBH it will be a huge step forward in tech terms - Rusoto is maintenance mode, and it's missing functionality. I had to jump into it just the other day to add S3 Select support.
    - nicoburns 5y ago
      It will be (hopefully), but it isn't yet. This crate doesn't even suppory S3 at all so far.
      - staticassertion 5y ago
        For sure :) I just have expectations.
- antihero 5y ago
  Shit man, as long as they do a better job than they did with Amplify.
  - skohan 5y ago
    Not holding my breath there...
    - antihero 5y ago
      Ugh. To anyone contemplating using Amplify I would say: Rolling your own UI components and straight up using the amazon-cognito-identity-js library and wrapping the bits you need in promises, is far far far less of a headache than using Amplify, unless being tied to Webpack and dealing with disgracefully lazy types is your idea of fun.
      - skohan 5y ago
        That's what we're doing actually. I get the impression that Amplify was designed more for vendor-lock-in than actually solving the problem in the best way
theopsguy 5y ago
Still waiting on GCP to announce theirs
- ibraheemdev 5y ago
  The Rust community opened an issue on their tracker [0] - it seems that they are not working on one right now: > Greetings folks! I work on the team that builds the Cloud Client Libraries. Per the last comment - I can absolutely share that we're not actively working on a Rust SDK right now, and we have no immediate plans to start. We completely understand this would be useful, and we'd love to go build it. However, we're focusing all of our energies on Java, Go, Node.js, Python, C#, Ruby, PHP, and C++ at the moment. If you're looking to build something, I'd be happy to bounce ideas with you! For now, we have https://github.com/Byron/google-apis-rs https://github.com/Byron/google-apis-rs. [0]: https://issuetracker.google.com/issues/124538614 https://issuetracker.google.com/issues/124538614
scriptsmith 5y ago
Rusoto is awesome, but looked to have an uncertain future. I hope the new SDK can reach the same level of quality and breadth fairly quickly. Also, rusoto is a great name and I hope we get to keep it.
oblio 5y ago
Now they need an official Jetbrains IDE and we'll know Rust has made it.
- agrinman 5y ago
  It's basically already here (in the form of a plugin): https://www.jetbrains.com/rust/ https://www.jetbrains.com/rust/ I've been enjoying it quite a bit.
- scriptsmith 5y ago
  I believe Jetbrains have stated that there would need to be more value to create a whole new product. Using the plugin with CLion feels very well integrated, and I’m not sure what else we’d get out of a seperate IDE other than more distinct branding.
- warmwaffles 5y ago
  Honestly though, VSCode has been pretty good in this respect. Although I wouldn't use it to write Java or Kotlin, but for C/C++/Zig/D/Rust it's been good.
  - dboreham 5y ago
    <raises hand> writing Scala here in VScode.
    - valzam 5y ago
      Some things are better, some worse. Highlights for unused code or imports is imho worse in vscode/metals and refactoring isn't as easy. But running tests for me has always been much faster. Not sure if you can use bloop with intelij but incremental compilation is much better in vscode.
    - warmwaffles 5y ago
      I've had mixed success with VSCode and JVM related stuff. Although the projects I was working on were using some really obscure Maven stuff and god forbid gradle.
zokier 5y ago
> Just like our newer SDKs, AWS used the Smithy toolchain and service models to build AWS SDK for Rust This is pretty impressive. I haven't looked at the SDK in detail, but managing to produce ergonomic APIs (semi-)automatically based on some service models for language as complex as Rust seems like a major accomplishment. I have most experience with the Python library (boto3) which is bit weird at places, but the Pythons dynamic nature is pretty good at hiding lot of that weirdness. Interesting to see how that works out for Rust where so much is static and explicit instead. This also means that the possibility for multiple runtimes might not be that far fetched idea, if they can just make a code generator that spits out different flavors of the SDK.
- bradleyjg 5y ago
  The Java SDK is not especially ergonomic and the docs are exactly what you’d expect from auto generation.
  - nkozyra 5y ago
    This universal for all of their SDKs. They seem to get community examples, too, which often contain some of the worst code you'll find. But the consistency is worth it. If you know how to work with boto you can use the java, JS, go, etc sdks easily.
- OJFord 5y ago
  > managing to produce ergonomic APIs (semi-)automatically [...] I have most experience with the Python library (boto3) This almost sounds tongue-in-cheek: the slightest use of boto3 makes it seem obviously auto-generated. It's fine, it even has advantages (whole classes of bugs removed, or at least fixed once fixed everywhere), I just wouldn't call it ergonomic or idiomatic. 's3 = boto3.client("s3")'.
- RideAndWave 5y ago
  I think there is a trend for a set of tools which auto-generate stuff. It started back with web tools generating websites from WYSIWYG, now it is getting everywhere. We build services and models using https://www.protoforce.io https://www.protoforce.io, which also auto-generates client and server side. Generally, it is impossible to generate good code using templates only, so it has to be transpiled. I think from amazon's side it was coming sooner or later anyway, with the amount of APIs they have - it is inevitable. I just can't see how many teams you need to have to manage all of the client side code for many languages.
matthewkmayer 5y ago
This is wonderful to see and a great outcome from the effort put into Rust and Rusoto.
justin_oaks 5y ago
That's an odd set of initial services. I was thinking that it might be just some newer services, but then I saw DynamoDB which has been around for quite a while.
- k__ 5y ago
  I think, they started with the services that share a protocol.
technological 5y ago
Does SDK for rust provide any advantages or this is more for people who develop in RUST and can use single language across their environment ?
cbb330 5y ago
How often do the domains of systems programming and cloud administration intersect? I'm assuming that rust programs are primarily used in systems programming and that this SDK would be used for cloud administration but I could be wrong. I see an obvious match with python and boto3 but not this.
- nevi-me 5y ago
  There's a lot or growing number of people who don't use Rust for "systems programming" per se. People are writing web services or data products that would need to interact with cloud services. Object stores, SQL interaction, lambda/serverless, etc. I'd think that people using this SDK for cloud administration would be in the minority, but it's just a hunch.
  - myrryr 5y ago
    I'll give it a go, as long as the documentation is good. I worry about that.
- _greim_ 5y ago
  > How often do the domains of systems programming and cloud administration intersect? I can see Rust being attractive for high-traffic network apps, based on the idea of requests-per-second not being amortized over GC pauses.
  - secondcoming 5y ago
    Anyone in this space would already have been using apache or nginx, which are in C or C++. How mature are rust's HTTP implementations?
    - staticassertion 5y ago
      https://linkerd.io/ https://linkerd.io/ Seems relevant.
    - steveklabnik 5y ago
19h 5y ago
Rusoto was an amazing effort. I was pretty sad when it was semi-sunset, but then I realized how all of this was entirely voluntary work. There's been a massive amount of new AWS features recently and this made the inofficial SDKs lag behind. Looking forward to having an officially maintained SDK. Hope the documentation will be better than the generated ones in Rusoto.
AzzieElbab 5y ago
Does this still run against the ts/js runtime like the other sdks?
- phjordon 5y ago
  I believe you are thinking of the CDKs. The SDKs are all fully language native.
  - AzzieElbab 5y ago
    you are right, my brain must have developed fixation on cdk due to work
k__ 5y ago
In turn with this release I learned that S3 isn't supported from the beginning because it's the only AWS service using an XML protocol. Made me chuckle a bit.
- phjordon 5y ago
  There's a handful of services that use XML protocols, but certainly not a lot.
closeparen 5y ago
When is it actually desirable to imperatively program your cloud provider rather than using something like Terraform? Obviously there are services meant to be consumed from application code, but the infrastructure provisioning stuff?
- KptMarchewa 5y ago
  Even if I wanted to use real programming language for that, Pulumi seems to be better choice.
  - tonyhb 5y ago
    It's not, trust me. Use the terraform CDK instead.
- bigyikes 5y ago
  Maybe this makes it easier to develop things like Terraform using Rust?
- dominotw 5y ago
  > When is it actually desirable to imperatively program your cloud provider rather than using something like Terraform? One place I've done is when your infra needs to dynamically scale up/down based on load.
  - NikolaeVarius 5y ago
    TF supports this just fine
- catlifeonmars 5y ago
  I have built several services that do dynamic infrastructure provisioning in response to API calls as part of the core functionality. This isn’t unusual.
- jen20 5y ago
  When implementing a Terraform provider, for one. At one point I had the scaffolding up to build providers in Rust (contributing to Tonic and Rusoto along the way) - many of the bugs in TF providers would be resolved by generics and type safety.
scrubs 5y ago
Off hand question for Rust: if memory is borrowed, I gather Rust does not need --- perhaps resists --- C++ style allocators? Here I give emphasis to owner of the (heap) data rather than efficiency. Both languages might use an allocator for fast re-use, less fragmentation, MT support etc.. On the the hand STL strings and containers are replete with allocators in their constructors leading to such questions like if two non-empty std::vector<T> A,B vectors are equal (same size, specialized both on T, resp. elements equal) is A==B true if A has a different allocator from B? Now granted allocator aware containers is a STL thing not per se a C++ thing. But the STL should leverage C++ design to the good ... so it's not too wrong or off base to think about the language in terms of a primary exemplar library.

219 points by carllerche 5y ago

This is huge for Rust. With Microsoft Windows SDK and now with this, Rust is starting to get first party support on platforms. More so than language syntax, the driver of adoption of a language is ecosystem. Rust is rapidly building up a very impressive ecosystem around it.

ibraheemdev 5y ago

As good as rusoto is, an official SDK is a huge step forward. Many were wondering how AWS could be "commited to Rust" but not provide an official SDK, this looks great! > We are exploring ways to support multiple Rust async runtimes. Looking to be runtime agnostic from the start is a good sign, although it can be challenging in async rust's current state. It looks like they aren't tied into much async IO, so it should be relatively easy for this library.

uulu 5y ago
> As good as rusoto is, an official SDK is a huge step forward. Yes. It might not be "a huge step forward" in tech terms, but it would be definitely very positive for wider adoption, especially from "enterprises" where all is based on "standards".
- staticassertion 5y ago
  TBH it will be a huge step forward in tech terms - Rusoto is maintenance mode, and it's missing functionality. I had to jump into it just the other day to add S3 Select support.
  - nicoburns 5y ago
    It will be (hopefully), but it isn't yet. This crate doesn't even suppory S3 at all so far.
    - staticassertion 5y ago
      For sure :) I just have expectations.
antihero 5y ago
Shit man, as long as they do a better job than they did with Amplify.
- skohan 5y ago
  Not holding my breath there...
  - antihero 5y ago
    Ugh. To anyone contemplating using Amplify I would say: Rolling your own UI components and straight up using the amazon-cognito-identity-js library and wrapping the bits you need in promises, is far far far less of a headache than using Amplify, unless being tied to Webpack and dealing with disgracefully lazy types is your idea of fun.
    - skohan 5y ago
      That's what we're doing actually. I get the impression that Amplify was designed more for vendor-lock-in than actually solving the problem in the best way

theopsguy 5y ago

Still waiting on GCP to announce theirs

ibraheemdev 5y ago
The Rust community opened an issue on their tracker [0] - it seems that they are not working on one right now: > Greetings folks! I work on the team that builds the Cloud Client Libraries. Per the last comment - I can absolutely share that we're not actively working on a Rust SDK right now, and we have no immediate plans to start. We completely understand this would be useful, and we'd love to go build it. However, we're focusing all of our energies on Java, Go, Node.js, Python, C#, Ruby, PHP, and C++ at the moment. If you're looking to build something, I'd be happy to bounce ideas with you! For now, we have https://github.com/Byron/google-apis-rs https://github.com/Byron/google-apis-rs. [0]: https://issuetracker.google.com/issues/124538614 https://issuetracker.google.com/issues/124538614

scriptsmith 5y ago

Rusoto is awesome, but looked to have an uncertain future. I hope the new SDK can reach the same level of quality and breadth fairly quickly. Also, rusoto is a great name and I hope we get to keep it.

oblio 5y ago

Now they need an official Jetbrains IDE and we'll know Rust has made it.

agrinman 5y ago
It's basically already here (in the form of a plugin): https://www.jetbrains.com/rust/ https://www.jetbrains.com/rust/ I've been enjoying it quite a bit.
scriptsmith 5y ago
I believe Jetbrains have stated that there would need to be more value to create a whole new product. Using the plugin with CLion feels very well integrated, and I’m not sure what else we’d get out of a seperate IDE other than more distinct branding.
warmwaffles 5y ago
Honestly though, VSCode has been pretty good in this respect. Although I wouldn't use it to write Java or Kotlin, but for C/C++/Zig/D/Rust it's been good.
- dboreham 5y ago
  <raises hand> writing Scala here in VScode.
  - valzam 5y ago
    Some things are better, some worse. Highlights for unused code or imports is imho worse in vscode/metals and refactoring isn't as easy. But running tests for me has always been much faster. Not sure if you can use bloop with intelij but incremental compilation is much better in vscode.
  - warmwaffles 5y ago
    I've had mixed success with VSCode and JVM related stuff. Although the projects I was working on were using some really obscure Maven stuff and god forbid gradle.

zokier 5y ago

> Just like our newer SDKs, AWS used the Smithy toolchain and service models to build AWS SDK for Rust This is pretty impressive. I haven't looked at the SDK in detail, but managing to produce ergonomic APIs (semi-)automatically based on some service models for language as complex as Rust seems like a major accomplishment. I have most experience with the Python library (boto3) which is bit weird at places, but the Pythons dynamic nature is pretty good at hiding lot of that weirdness. Interesting to see how that works out for Rust where so much is static and explicit instead. This also means that the possibility for multiple runtimes might not be that far fetched idea, if they can just make a code generator that spits out different flavors of the SDK.

bradleyjg 5y ago
The Java SDK is not especially ergonomic and the docs are exactly what you’d expect from auto generation.
- nkozyra 5y ago
  This universal for all of their SDKs. They seem to get community examples, too, which often contain some of the worst code you'll find. But the consistency is worth it. If you know how to work with boto you can use the java, JS, go, etc sdks easily.
OJFord 5y ago
> managing to produce ergonomic APIs (semi-)automatically [...] I have most experience with the Python library (boto3) This almost sounds tongue-in-cheek: the slightest use of boto3 makes it seem obviously auto-generated. It's fine, it even has advantages (whole classes of bugs removed, or at least fixed once fixed everywhere), I just wouldn't call it ergonomic or idiomatic. 's3 = boto3.client("s3")'.
RideAndWave 5y ago
I think there is a trend for a set of tools which auto-generate stuff. It started back with web tools generating websites from WYSIWYG, now it is getting everywhere. We build services and models using https://www.protoforce.io https://www.protoforce.io, which also auto-generates client and server side. Generally, it is impossible to generate good code using templates only, so it has to be transpiled. I think from amazon's side it was coming sooner or later anyway, with the amount of APIs they have - it is inevitable. I just can't see how many teams you need to have to manage all of the client side code for many languages.

matthewkmayer 5y ago

This is wonderful to see and a great outcome from the effort put into Rust and Rusoto.

justin_oaks 5y ago

That's an odd set of initial services. I was thinking that it might be just some newer services, but then I saw DynamoDB which has been around for quite a while.

k__ 5y ago
I think, they started with the services that share a protocol.

technological 5y ago

Does SDK for rust provide any advantages or this is more for people who develop in RUST and can use single language across their environment ?

cbb330 5y ago

How often do the domains of systems programming and cloud administration intersect? I'm assuming that rust programs are primarily used in systems programming and that this SDK would be used for cloud administration but I could be wrong. I see an obvious match with python and boto3 but not this.

nevi-me 5y ago
There's a lot or growing number of people who don't use Rust for "systems programming" per se. People are writing web services or data products that would need to interact with cloud services. Object stores, SQL interaction, lambda/serverless, etc. I'd think that people using this SDK for cloud administration would be in the minority, but it's just a hunch.
- myrryr 5y ago
  I'll give it a go, as long as the documentation is good. I worry about that.
_greim_ 5y ago
> How often do the domains of systems programming and cloud administration intersect? I can see Rust being attractive for high-traffic network apps, based on the idea of requests-per-second not being amortized over GC pauses.
- secondcoming 5y ago
  Anyone in this space would already have been using apache or nginx, which are in C or C++. How mature are rust's HTTP implementations?
  - staticassertion 5y ago
    https://linkerd.io/ https://linkerd.io/ Seems relevant.
  - steveklabnik 5y ago

19h 5y ago

Rusoto was an amazing effort. I was pretty sad when it was semi-sunset, but then I realized how all of this was entirely voluntary work. There's been a massive amount of new AWS features recently and this made the inofficial SDKs lag behind. Looking forward to having an officially maintained SDK. Hope the documentation will be better than the generated ones in Rusoto.

AzzieElbab 5y ago

Does this still run against the ts/js runtime like the other sdks?

phjordon 5y ago
I believe you are thinking of the CDKs. The SDKs are all fully language native.
- AzzieElbab 5y ago
  you are right, my brain must have developed fixation on cdk due to work

k__ 5y ago

In turn with this release I learned that S3 isn't supported from the beginning because it's the only AWS service using an XML protocol. Made me chuckle a bit.

phjordon 5y ago
There's a handful of services that use XML protocols, but certainly not a lot.

closeparen 5y ago

When is it actually desirable to imperatively program your cloud provider rather than using something like Terraform? Obviously there are services meant to be consumed from application code, but the infrastructure provisioning stuff?

KptMarchewa 5y ago
Even if I wanted to use real programming language for that, Pulumi seems to be better choice.
- tonyhb 5y ago
  It's not, trust me. Use the terraform CDK instead.
bigyikes 5y ago
Maybe this makes it easier to develop things like Terraform using Rust?
dominotw 5y ago
> When is it actually desirable to imperatively program your cloud provider rather than using something like Terraform? One place I've done is when your infra needs to dynamically scale up/down based on load.
- NikolaeVarius 5y ago
  TF supports this just fine
catlifeonmars 5y ago
I have built several services that do dynamic infrastructure provisioning in response to API calls as part of the core functionality. This isn’t unusual.
jen20 5y ago
When implementing a Terraform provider, for one. At one point I had the scaffolding up to build providers in Rust (contributing to Tonic and Rusoto along the way) - many of the bugs in TF providers would be resolved by generics and type safety.

scrubs 5y ago

Off hand question for Rust: if memory is borrowed, I gather Rust does not need --- perhaps resists --- C++ style allocators? Here I give emphasis to owner of the (heap) data rather than efficiency. Both languages might use an allocator for fast re-use, less fragmentation, MT support etc.. On the the hand STL strings and containers are replete with allocators in their constructors leading to such questions like if two non-empty std::vector<T> A,B vectors are equal (same size, specialized both on T, resp. elements equal) is A==B true if A has a different allocator from B? Now granted allocator aware containers is a STL thing not per se a C++ thing. But the STL should leverage C++ design to the good ... so it's not too wrong or off base to think about the language in terms of a primary exemplar library.