5 ms·

> Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing

> Leveraging public cloud infrastructure has traditionally demanded sifting through layers of boilerplate code and intricate configurations, often monopolizing a significant chunk of the developer’s time and energy. I don't buy this line of reasoning. At the end of the day we are building infrastructure that needs to be reliable. Spending 30 minutes to set up an SQS queue (proven technology) doesn't sound as bad as putting all my trust in a toy queue that Deno built "on top of SQLite / FoundationDB". Is the 30 minute setup cost and added "developer experience" really worth the risk? How often are you setting up queues anyway.

falcolas 3y ago
> How often are you setting up queues anyway. Very infrequently. Even more infrequently as we move to IAC and use terraform or even cloudformation.
asim 3y ago
Many people don't have experience with AWS or SQS and even those that do will still need to think about it. Deno is taking a batteries included approach which means the Dev can just import something and go. If someone provided equivalent libraries for the cloud then I'm sure people would adopt them. In the case of Deno I'm sure they've identified a common problem many Devs in their ecosystem have and are tackling it by bundling in a good solution.
- leptons 3y ago
  If Deno is only going after amateur devs by implementing simplistic features like this, then it will remain an amateur product. AWS/SQS may seem complex to some, but they are very powerful. Maybe Deno is doing what Apple did in the 80's by putting free Apple computers in classrooms - try to get new users hooked into their system and then they'll use it for life. Anyone doing anything serious with cloud computing is already using AWS, or should be. I would rather put in the time to learn AWS (which is entirely free in many small use cases), so I won't build stuff in a toy system and then later realize I need something better.
  - lopatin 3y ago
    Devs who know they need an event queue but are not technical enough to set one up in SQS sounds like a very niche market to me.
  - endisneigh 3y ago
    > Anyone doing anything serious with cloud computing is already using AWS, or should be. I would rather put in the time to learn AWS (which is entirely free in many small use cases), so I won't build stuff in a toy system and then later realize I need something better. So ridiculous.
  - asim 3y ago
    AWS was a toy in 2006. No one in their right mind was switching to the cloud back then. We laughed at it. It took years to mature. I'd say things that look like toys today will be the dominant platforms of the future because generationally a 20 something person out of college is more likely to adopt it than use the overwhelming and complex AWS. Barriers to entry are something you have to consider. Meaning, many of us grew up in an era of transition from bare metal to cloud and we became early adopters of it. These new tools are the same for a younger generation.
    - leptons 3y ago
      > overwhelming and complex AWS Any other systems a "20 something person" would cobble together for an app of any complexity from "toys" of today would be a complex web of half solutions. They'd be trying to do the same stuff that can be done within AWS, and often what they cobble together would be worse off having it made of disparate components from maybe dozens of different vendors. I can't see how it's any easier to connect all those dots than it is to do it within AWS. If you're new and doing simple stuff on simple systems that's all well and good, but don't expect it to scale easily or at all, and if you try you're in for a whole lot of dev-ops and networking and other bullshit just to get things to talk to each other. There's a lot less of that in AWS. Usually you just copy an ARN and paste it into another box, and the things are connected. >Barriers to entry are something you have to consider. There's no barrier to entry for AWS, unless you can't afford $0 per month. Anyone can sign up and use free services, and they are very well documented. There's tutorials galore, probably more than any other current toy platform(s). There's tools and tooling and all kinds of support out there for it. But sure, some toy platform might be more fun to use for your hello-world task tracking app if you aren't building anything serious.
locallost 3y ago
How do you know it's a toy queue?
- ricardobeat 3y ago
  Presumably everything AWS offers is way more battle-hardened and with stricter SLOs just due to the sheer volume of their customers. But people are way too afraid of simple tech these days. If there aren’t dedicated QA and Ops teams, at least 2k GitHub stars, it’s not web-scale™ and meant to run on a fleet of 726 servers at minimum, it’s not to be trusted!
  - paulddraper 3y ago
    If it doesn't have 1k GitHub stars it's either trivial or it has high risk of bugs.
skybrian 3y ago
I think this depends on who you are. As a hobbyist programmer, I don't use the big providers like AWS and Google because they seem rather complex? Maybe it's not so bad if you're used to them. I also like to make each project independent, as its own repo on GitHub. Ideally, a web app would be easy for anyone else to launch, as their their own independent web app, using a separate domain name, because I don't want to be responsible for their data. (This is sort of the Sandstorm use case.) "Try it out locally and get a Deno Deploy account if you want to use it for real" seem like reasonable install instructions?
- willsmith72 3y ago
  If Deno only suits hobbyist programmers, it's failed. To pick up steam it's going to have to pull people away from AWS and Google who already know how to use them. As for making project's independent, so do I, but that's simple in other clouds too with infra as code or one of the various deployment frameworks (fly lets you do this, AWS copilot does this, terraform, serverless)
  - skybrian 3y ago
    I think you might overestimate how much other people know about cloud infrastructure. I am very out-of-date and only vaguely know what these things do. I looked at Fly and they have nice docs and interesting ideas, but they’re also clear that they don’t do “fully managed” databases and I don’t want to be a DBA. I know of AWS as a huge pile of complexity that I’m not sure I want to get into? Using it directly seems too low-level for me (I’ve used Digital Ocean and it seems more my kind of thing, but it doesn’t scale down to zero for a website that gets no traffic.) Looking at Terraform, it’s not obviously about solving any problem I care about, and wasn’t there a huge controversy about them? Serverless is a buzzword. Isn’t Deno sort of serverless too? I think of Deno Deploy as a step up from Netlify, which is fine for static websites. Or maybe a reincarnation of App Engine, which I quite liked back when it launched, but it seems to have lost its way.
    - willsmith72 3y ago
      this is the serverless framework im talking about https://www.serverless.com/ https://www.serverless.com/ but fair enough. I do think the simplicity of deno will be a trap though. The benefit of a mature framework is you don't have to get lost in the weeds on obscure bugs or functionality with slightly niche use cases.
paxys 3y ago
By that logic no one should ever launch anything new, because it is a "toy project" by default and only AWS is proven?
marmaduke 3y ago
I don't recall SQS taking anytime to set up queues, at least when using Celery (python task queue). IIRC you just name it in your code and use it.
- antod 3y ago
  That's just at the codebase end of it. But at the infra end of it, it's a different story. AWS is suited towards a very different scale of development effort than Deno currently is. The effort required from scratch to securely and reliably get to a point where a dev can just name a queue is substantial in AWS. You really need account hierarchies, guard rails, roles, permissions, delegated IaC etc etc in place to be able to do that - otherwise it eventually gets out of hand. AWS is geared towards larger dev teams or groups of dev teams, while (to me at least without using it) Deno seems far more oriented at the smaller scale set and forget PaaS oriented teams - eg Heroku refugees etc. Those teams could very well outgrow Deno and need AWS at some stage though.
  - willsmith72 3y ago
    It used to be like that. These days I often see 1-3 person teams just copying around some terraform config they've had forever, and all of their AWS setup is done.
  - BoorishBears 3y ago
    You can use libraries like SST and make it a few lines of code.
tonyhb 3y ago
Hot take: I don't think that people should even use queues any more. It's like using raw HTTP2, or managing threads. Use a workflow engine which abstracts queues for you so you can "just write code" without managing jobs/schedules/state/retries. https://www.inngest.com/blog/how-durable-workflow-engines-work https://www.inngest.com/blog/how-durable-workflow-engines-wo... (disclaimer: I'm biased as I'm an author of a workflow engine)
- xwowsersx 3y ago
  In the example in your post, the queue is abstracted away in some sense, but you aren't just "writing code". There's step.run(), step.sleep(), step.sleepUntil() sprinkled throughout. I'd say in something like Celery, which is explicitly about running jobs asynchronously backed by a message broker/queue, you really do just write code, but then call the function with .s().apply_async() and so on. Now, I'm not saying you're wrong that we ought to abstract queues away, just pointing out that the abstraction leaks in your workflow example. Happy to hear where/how I'm wrong on this.
  - tonyhb 3y ago
    It leaks in that the entire function becomes declarative, and state is colocated to a single function. You don't have multiple jobs for each attempt, passing indexes into each job queue. You don't have to worry about enqueueing each "step" as a separate job. The code becomes "wait until this time", vs "enqueue this function with this state to run at this time via this message broker, which may enqueue other jobs in ways you can't see". There's no real silver bullet that will let you run code in the future without specifying "when" that code should run — but workflow engines are by far the more productive of the two.
  - mfateev 3y ago
    Check out temporal.io that fully abstract this. Disclaimer, I'm one of the founders.
    - xwowsersx 3y ago
59nadir 3y ago
To extend this into more productive territory, there is no reason I should have to use either the Deno version or AWS; it should be an interface that can be implemented so that I can choose whatever makes sense, including my own implementation.

falcolas 3y ago

> How often are you setting up queues anyway. Very infrequently. Even more infrequently as we move to IAC and use terraform or even cloudformation.

asim 3y ago

Many people don't have experience with AWS or SQS and even those that do will still need to think about it. Deno is taking a batteries included approach which means the Dev can just import something and go. If someone provided equivalent libraries for the cloud then I'm sure people would adopt them. In the case of Deno I'm sure they've identified a common problem many Devs in their ecosystem have and are tackling it by bundling in a good solution.

leptons 3y ago
If Deno is only going after amateur devs by implementing simplistic features like this, then it will remain an amateur product. AWS/SQS may seem complex to some, but they are very powerful. Maybe Deno is doing what Apple did in the 80's by putting free Apple computers in classrooms - try to get new users hooked into their system and then they'll use it for life. Anyone doing anything serious with cloud computing is already using AWS, or should be. I would rather put in the time to learn AWS (which is entirely free in many small use cases), so I won't build stuff in a toy system and then later realize I need something better.
- lopatin 3y ago
  Devs who know they need an event queue but are not technical enough to set one up in SQS sounds like a very niche market to me.
- endisneigh 3y ago
  > Anyone doing anything serious with cloud computing is already using AWS, or should be. I would rather put in the time to learn AWS (which is entirely free in many small use cases), so I won't build stuff in a toy system and then later realize I need something better. So ridiculous.
- asim 3y ago
  AWS was a toy in 2006. No one in their right mind was switching to the cloud back then. We laughed at it. It took years to mature. I'd say things that look like toys today will be the dominant platforms of the future because generationally a 20 something person out of college is more likely to adopt it than use the overwhelming and complex AWS. Barriers to entry are something you have to consider. Meaning, many of us grew up in an era of transition from bare metal to cloud and we became early adopters of it. These new tools are the same for a younger generation.
  - leptons 3y ago
    > overwhelming and complex AWS Any other systems a "20 something person" would cobble together for an app of any complexity from "toys" of today would be a complex web of half solutions. They'd be trying to do the same stuff that can be done within AWS, and often what they cobble together would be worse off having it made of disparate components from maybe dozens of different vendors. I can't see how it's any easier to connect all those dots than it is to do it within AWS. If you're new and doing simple stuff on simple systems that's all well and good, but don't expect it to scale easily or at all, and if you try you're in for a whole lot of dev-ops and networking and other bullshit just to get things to talk to each other. There's a lot less of that in AWS. Usually you just copy an ARN and paste it into another box, and the things are connected. >Barriers to entry are something you have to consider. There's no barrier to entry for AWS, unless you can't afford $0 per month. Anyone can sign up and use free services, and they are very well documented. There's tutorials galore, probably more than any other current toy platform(s). There's tools and tooling and all kinds of support out there for it. But sure, some toy platform might be more fun to use for your hello-world task tracking app if you aren't building anything serious.

locallost 3y ago

How do you know it's a toy queue?

ricardobeat 3y ago
Presumably everything AWS offers is way more battle-hardened and with stricter SLOs just due to the sheer volume of their customers. But people are way too afraid of simple tech these days. If there aren’t dedicated QA and Ops teams, at least 2k GitHub stars, it’s not web-scale™ and meant to run on a fleet of 726 servers at minimum, it’s not to be trusted!
- paulddraper 3y ago
  If it doesn't have 1k GitHub stars it's either trivial or it has high risk of bugs.

skybrian 3y ago

I think this depends on who you are. As a hobbyist programmer, I don't use the big providers like AWS and Google because they seem rather complex? Maybe it's not so bad if you're used to them. I also like to make each project independent, as its own repo on GitHub. Ideally, a web app would be easy for anyone else to launch, as their their own independent web app, using a separate domain name, because I don't want to be responsible for their data. (This is sort of the Sandstorm use case.) "Try it out locally and get a Deno Deploy account if you want to use it for real" seem like reasonable install instructions?

willsmith72 3y ago
If Deno only suits hobbyist programmers, it's failed. To pick up steam it's going to have to pull people away from AWS and Google who already know how to use them. As for making project's independent, so do I, but that's simple in other clouds too with infra as code or one of the various deployment frameworks (fly lets you do this, AWS copilot does this, terraform, serverless)
- skybrian 3y ago
  I think you might overestimate how much other people know about cloud infrastructure. I am very out-of-date and only vaguely know what these things do. I looked at Fly and they have nice docs and interesting ideas, but they’re also clear that they don’t do “fully managed” databases and I don’t want to be a DBA. I know of AWS as a huge pile of complexity that I’m not sure I want to get into? Using it directly seems too low-level for me (I’ve used Digital Ocean and it seems more my kind of thing, but it doesn’t scale down to zero for a website that gets no traffic.) Looking at Terraform, it’s not obviously about solving any problem I care about, and wasn’t there a huge controversy about them? Serverless is a buzzword. Isn’t Deno sort of serverless too? I think of Deno Deploy as a step up from Netlify, which is fine for static websites. Or maybe a reincarnation of App Engine, which I quite liked back when it launched, but it seems to have lost its way.
  - willsmith72 3y ago
    this is the serverless framework im talking about https://www.serverless.com/ https://www.serverless.com/ but fair enough. I do think the simplicity of deno will be a trap though. The benefit of a mature framework is you don't have to get lost in the weeds on obscure bugs or functionality with slightly niche use cases.

paxys 3y ago

By that logic no one should ever launch anything new, because it is a "toy project" by default and only AWS is proven?

marmaduke 3y ago

I don't recall SQS taking anytime to set up queues, at least when using Celery (python task queue). IIRC you just name it in your code and use it.

antod 3y ago
That's just at the codebase end of it. But at the infra end of it, it's a different story. AWS is suited towards a very different scale of development effort than Deno currently is. The effort required from scratch to securely and reliably get to a point where a dev can just name a queue is substantial in AWS. You really need account hierarchies, guard rails, roles, permissions, delegated IaC etc etc in place to be able to do that - otherwise it eventually gets out of hand. AWS is geared towards larger dev teams or groups of dev teams, while (to me at least without using it) Deno seems far more oriented at the smaller scale set and forget PaaS oriented teams - eg Heroku refugees etc. Those teams could very well outgrow Deno and need AWS at some stage though.
- willsmith72 3y ago
  It used to be like that. These days I often see 1-3 person teams just copying around some terraform config they've had forever, and all of their AWS setup is done.
- BoorishBears 3y ago
  You can use libraries like SST and make it a few lines of code.

tonyhb 3y ago

Hot take: I don't think that people should even use queues any more. It's like using raw HTTP2, or managing threads. Use a workflow engine which abstracts queues for you so you can "just write code" without managing jobs/schedules/state/retries. https://www.inngest.com/blog/how-durable-workflow-engines-work https://www.inngest.com/blog/how-durable-workflow-engines-wo... (disclaimer: I'm biased as I'm an author of a workflow engine)

xwowsersx 3y ago
In the example in your post, the queue is abstracted away in some sense, but you aren't just "writing code". There's step.run(), step.sleep(), step.sleepUntil() sprinkled throughout. I'd say in something like Celery, which is explicitly about running jobs asynchronously backed by a message broker/queue, you really do just write code, but then call the function with .s().apply_async() and so on. Now, I'm not saying you're wrong that we ought to abstract queues away, just pointing out that the abstraction leaks in your workflow example. Happy to hear where/how I'm wrong on this.
- tonyhb 3y ago
  It leaks in that the entire function becomes declarative, and state is colocated to a single function. You don't have multiple jobs for each attempt, passing indexes into each job queue. You don't have to worry about enqueueing each "step" as a separate job. The code becomes "wait until this time", vs "enqueue this function with this state to run at this time via this message broker, which may enqueue other jobs in ways you can't see". There's no real silver bullet that will let you run code in the future without specifying "when" that code should run — but workflow engines are by far the more productive of the two.
- mfateev 3y ago
  Check out temporal.io that fully abstract this. Disclaimer, I'm one of the founders.
  - xwowsersx 3y ago

59nadir 3y ago

To extend this into more productive territory, there is no reason I should have to use either the Deno version or AWS; it should be an interface that can be implemented so that I can choose whatever makes sense, including my own implementation.