7 ms·

re: Why is this a language, not just a framework for JavaScript or whatever? One product that's taking this approach is shuttle -> https://www.shuttle.rs/ http

by dohman 4y ago

re: Why is this a language, not just a framework for JavaScript or whatever? One product that's taking this approach is shuttle -> https://www.shuttle.rs/ https://www.shuttle.rs/ (disclaimer: I work there). It's built around Rust and not TS but we have some updates coming soon which is going to expand on this. We've been thinking about this DSL question for a while and it's pretty much a trade-off. By having your own language and parser you can build language primitives which get your close to your domain, as for example in the example you gave with HCL. It will be interesting to see where Wing goes with this concept, it could offer a great developer experience. For us, we've chosen to stick with a robust and performant programming language which is increasing in popularity and has enough meta-programming capabilities to make this 'infrastructure from code' paradigm feel natural - the framework blends in well with the language. The advantages of this for us is that users of the language experience a very small learning curve and they can also use incredible Rust libraries off-the-shelf. We can also piggy-back off the Rust compiler and ecosystem to provide an A-class developer experience (those compiler errors are amazing). We're excited to see more companies in this space - optimising for a beautiful developer experience.

mikercampbell 4y ago
I hecking love the concept of shuttle. The only thing keeping me from using it is unfamiliarity with rust, but it's enough to make me rethink things. I think the future isn't as much Lo-code as much as it will be (and goshdarn should be) Lo-DevOps. There are so many advances in programming, like WASM for example, that will continue to cause this Lo-DevOps movement, but honestly, digging into the language, using the stuff that already is baked in for other existing business purposes. But things like encore.dev and shuttle.rs (I'm associated with neither) are just mind-blowingly straightforward with no language, infrastructure, or other cognitively burdening information.
- zozbot234 4y ago
  > I hecking love the concept of shuttle. I'm not sure about it. If you're going to have Rust-specific deployment workflows, you should reuse the solutions that have been developed for the embedded ecosystem. There's no reason why "cargo flash" could not be extended to upload your binaries to the cloud, and support the same "probe run" workflow for remote logging and debugging.
ZeroCool2u 4y ago
I'm very excited about Shuttle, I've been watching it for a while and it looks fantastic. The only thing I'm looking for is support for GCP, but I don't see that happening until Google makes official Rust client packages.
- openquery 4y ago
  > The only thing I'm looking for is support for GCP Glad you like Shuttle! What do you mean exactly by support for GCP? Shuttle right now is built on AWS but that is mostly abstracted away from you. Do you want to use GCP products (say BigQuery) or do you want to host Shuttle on your own GCP project?
  - ZeroCool2u 4y ago
    Really both. I'd prefer to be a bit more familiar with the underlying infrastructure and also I'd like to use GCP products, BigQuery is definitely a big one though!
verdverm 4y ago
The main disadvantage of having a custom language, for a user is, having to learn a bespoke language for a single tool. Very few are willing to do this. As a company, it is generally a distraction from delivering value. Engineers like it because it is interesting work to write a language. Most should not use HCL as an argument for forcing bespoke languages on users. (1) HCL is not a programming language, more of a config language. (2) It was the right time and place, but has shown rough edges as it has aged, it feels more hacky now. Just write libraries
- OkayPhysicist 4y ago
  The good arguments for "This should be a new language" are either: 1. You need to enforce a relatively unique set of invariants, which in turn come with new constructs to interact with them. For example, if you want a system for managing a distributed system, enforcing the lack of global state can be a lot more natural by using a language that cannot express it. 2. You want to provide some sort of novel language construct that cannot be expressed well in existing languages (function argument pattern matching, for example, cannot be retrofitted onto an existing language by a library, maybe by pre-proccessor or sufficiently advanced macro system).
  - verdverm 4y ago
    1. Could a distributed language language really know what I'm using data for, i.e. if I put an object in an object store, is it global or not? Seems like a really complicated analysis. What if this happens across services? (Separate code bases)
    - vineyardmike 4y ago
      Well there exists a language today meant for distributed systems (erlang, elixir). It doesn’t support global state. Instead you’d write a “process” that’d respond to requests with the state at time of request. Another request could mutate the state, but each request is processed one at a time. So a “bucket” type data store (object store etc) would likely be implemented as “server” that returns the state. If the objects were large you could pass a URL/handle to the file which you’d retrieve later. These processes/servers/whatever are running inside the language VM and are SUPER lightweight. Multiple language VMs can be tied together so the actual processes can be distributed across a fleet of physical hosts in a data center - this would be transparent to how you write the code. If you wanted to implement that in native AWS primitives you’d probably run a lambda function to transform an http request into a S3 bucket address, then you’d go and make an s3 request to download/modify the file. Or the lambda would make some modifications to the file. So, that’s the prior art on a hypothetical distributed language handling global state, which fwiw is basically how “cloud” systems work today.
Thaxll 4y ago
But Shuttle only deploy Rust code on your platform, it's a completely different use case. What if I want to deploy Python code on AWS? Also, linking the infra code inside of your app code is a terrible idea imo.
- openquery 4y ago
  > Also, linking the infra code inside of your app code is a terrible idea imo. Why do you think it's a terrible idea?
  - ukd1 4y ago
    Well, I'd be curious if that means it has more permissions that necessary to run in production because of that - e.g. to create an arbitrary bucket, or queue, or IAM policy.
    - openquery 4y ago
      Deployed services don't currently have permissions to provision or modify infrastructure. This is done by our build system while your services are being compiled. It is an orthogonal control plane with its own permission system.
  - Thaxll 4y ago
    Mostly because the two should be separated, you platform is closed, so now you have very specific code just for shuttle.rs inside "business logic", what if I want to run my code in a docker image somewhere else, now I have to strip the service from some integration with a provider.
  - rollcat 4y ago
    Because lifecycle management, access controls, auditing, monitoring, delegation/federation, budgeting, capacity planning - all of these other things together, which these tools may or may not help you with - are still far less important than troubleshooting. Seriously, when shit's on fire, the last thing you want is to be deciphering the magic that brought it to its current state. How many layers to dig thru are there, between "thingctl deploy" and a critical S3 bucket having somehow been deleted? Not trying to shoot down the idea - I'm sure there are people 10x smarter than me working on this, who are more than able to make a lot of this work well in practice. But that's the problem with using stuff made by people 10x smarter than you - when everything goes wrong, it tends to require being 10x-smarter-than-average to fix it. The true power and success of Terraform is in the blunt simplicity of its interface. Every time you run it, it will spell out in very large writing, what exactly is it going to do: add this, change that, DESTROY this - it even uses the word "destroy" to signify the danger. I really appreciate any tool that makes it harder to do the wrong thing.
thurn 4y ago
Neat, hadn't heard of Shuttle. Any plans to support Tonic out of the box? We're pretty locked-in to gRPC instead of HTTP unfortunately.
mikercampbell 4y ago
I tried shuttle just because of this. Super easy!!! Like, too easy. Like, easy enough to justify making the leap to rust with this.

by dohman 4y ago

mikercampbell 4y ago

I hecking love the concept of shuttle. The only thing keeping me from using it is unfamiliarity with rust, but it's enough to make me rethink things. I think the future isn't as much Lo-code as much as it will be (and goshdarn should be) Lo-DevOps. There are so many advances in programming, like WASM for example, that will continue to cause this Lo-DevOps movement, but honestly, digging into the language, using the stuff that already is baked in for other existing business purposes. But things like encore.dev and shuttle.rs (I'm associated with neither) are just mind-blowingly straightforward with no language, infrastructure, or other cognitively burdening information.

zozbot234 4y ago
> I hecking love the concept of shuttle. I'm not sure about it. If you're going to have Rust-specific deployment workflows, you should reuse the solutions that have been developed for the embedded ecosystem. There's no reason why "cargo flash" could not be extended to upload your binaries to the cloud, and support the same "probe run" workflow for remote logging and debugging.

ZeroCool2u 4y ago

I'm very excited about Shuttle, I've been watching it for a while and it looks fantastic. The only thing I'm looking for is support for GCP, but I don't see that happening until Google makes official Rust client packages.

openquery 4y ago
> The only thing I'm looking for is support for GCP Glad you like Shuttle! What do you mean exactly by support for GCP? Shuttle right now is built on AWS but that is mostly abstracted away from you. Do you want to use GCP products (say BigQuery) or do you want to host Shuttle on your own GCP project?
- ZeroCool2u 4y ago
  Really both. I'd prefer to be a bit more familiar with the underlying infrastructure and also I'd like to use GCP products, BigQuery is definitely a big one though!

verdverm 4y ago

The main disadvantage of having a custom language, for a user is, having to learn a bespoke language for a single tool. Very few are willing to do this. As a company, it is generally a distraction from delivering value. Engineers like it because it is interesting work to write a language. Most should not use HCL as an argument for forcing bespoke languages on users. (1) HCL is not a programming language, more of a config language. (2) It was the right time and place, but has shown rough edges as it has aged, it feels more hacky now. Just write libraries

OkayPhysicist 4y ago
The good arguments for "This should be a new language" are either: 1. You need to enforce a relatively unique set of invariants, which in turn come with new constructs to interact with them. For example, if you want a system for managing a distributed system, enforcing the lack of global state can be a lot more natural by using a language that cannot express it. 2. You want to provide some sort of novel language construct that cannot be expressed well in existing languages (function argument pattern matching, for example, cannot be retrofitted onto an existing language by a library, maybe by pre-proccessor or sufficiently advanced macro system).
- verdverm 4y ago
  1. Could a distributed language language really know what I'm using data for, i.e. if I put an object in an object store, is it global or not? Seems like a really complicated analysis. What if this happens across services? (Separate code bases)
  - vineyardmike 4y ago
    Well there exists a language today meant for distributed systems (erlang, elixir). It doesn’t support global state. Instead you’d write a “process” that’d respond to requests with the state at time of request. Another request could mutate the state, but each request is processed one at a time. So a “bucket” type data store (object store etc) would likely be implemented as “server” that returns the state. If the objects were large you could pass a URL/handle to the file which you’d retrieve later. These processes/servers/whatever are running inside the language VM and are SUPER lightweight. Multiple language VMs can be tied together so the actual processes can be distributed across a fleet of physical hosts in a data center - this would be transparent to how you write the code. If you wanted to implement that in native AWS primitives you’d probably run a lambda function to transform an http request into a S3 bucket address, then you’d go and make an s3 request to download/modify the file. Or the lambda would make some modifications to the file. So, that’s the prior art on a hypothetical distributed language handling global state, which fwiw is basically how “cloud” systems work today.

Thaxll 4y ago

But Shuttle only deploy Rust code on your platform, it's a completely different use case. What if I want to deploy Python code on AWS? Also, linking the infra code inside of your app code is a terrible idea imo.

openquery 4y ago
> Also, linking the infra code inside of your app code is a terrible idea imo. Why do you think it's a terrible idea?
- ukd1 4y ago
  Well, I'd be curious if that means it has more permissions that necessary to run in production because of that - e.g. to create an arbitrary bucket, or queue, or IAM policy.
  - openquery 4y ago
    Deployed services don't currently have permissions to provision or modify infrastructure. This is done by our build system while your services are being compiled. It is an orthogonal control plane with its own permission system.
- Thaxll 4y ago
  Mostly because the two should be separated, you platform is closed, so now you have very specific code just for shuttle.rs inside "business logic", what if I want to run my code in a docker image somewhere else, now I have to strip the service from some integration with a provider.
- rollcat 4y ago
  Because lifecycle management, access controls, auditing, monitoring, delegation/federation, budgeting, capacity planning - all of these other things together, which these tools may or may not help you with - are still far less important than troubleshooting. Seriously, when shit's on fire, the last thing you want is to be deciphering the magic that brought it to its current state. How many layers to dig thru are there, between "thingctl deploy" and a critical S3 bucket having somehow been deleted? Not trying to shoot down the idea - I'm sure there are people 10x smarter than me working on this, who are more than able to make a lot of this work well in practice. But that's the problem with using stuff made by people 10x smarter than you - when everything goes wrong, it tends to require being 10x-smarter-than-average to fix it. The true power and success of Terraform is in the blunt simplicity of its interface. Every time you run it, it will spell out in very large writing, what exactly is it going to do: add this, change that, DESTROY this - it even uses the word "destroy" to signify the danger. I really appreciate any tool that makes it harder to do the wrong thing.

thurn 4y ago

Neat, hadn't heard of Shuttle. Any plans to support Tonic out of the box? We're pretty locked-in to gRPC instead of HTTP unfortunately.

mikercampbell 4y ago

I tried shuttle just because of this. Super easy!!! Like, too easy. Like, easy enough to justify making the leap to rust with this.