4 ms·

Honest question how is it easier for a startup vs DO with few VPSes and say manual DB fail-over.

by qaq 8y ago

adzicg 8y ago
Two things, based on my experience (we migrated from Heroku to Lambda in 2016, so been there for a while) 1. you don’t need to worry about reserving capacity, so you don’t need to pay for growth you expect to happen, or worry about not meeting a spike in demand if it happens 2. Most of the operations stuff (apart from packaging) is included in the price, so things like monitoring, alerts, dead-letter queues, traffic shifting beween canary versions, failovers, balancing... and it’s priced per request, so this comes to effectively free if you don’t have a lot of traffic. The big catch is that you don’t control the containers, so there’s no session stickiness. Getting the benefits from Lambda requires re-thinking how you do sessions and storage.
- qaq 8y ago
  That the thing though the dev overhead and limitations for startup with avg compute needs might be more $ than tiny savings (e.g who cares if it is $200/month vs $70/month) if it will cost 50k extra in dev time to architect for lambda
  - scarface74 8y ago
    How is “architecting for lambda” harder than traditional architecture?
    - qaq 8y ago
      having persistent processes ? Not needing API gateway? Not having to learn a whole bunch of AWS services? Not having to work around latency? Having a simple system that easy to debug?
      - scarface74 8y ago
        If by “persistent processes” you mean storing state locally. You usually shouldn’t be counting on that anyway. My VMs are just as disposable as lambda. API Gateway - we use API Gateway even for our VM hosted APIs for the other features it provides. We also integrate with swagger so creating and maintaining our API Gateway set up is basically automatic. Not having to learn a bunch of AWS services - if you are using AWS for more than just hosting VMs, you would have to learn AWS anyway. Even if you are hosting a bunch of VMs, you still need to know about security groups, VPCs, subnets, ELBs, route 53, autoscsling etc. Creating a lambda is adding one function to your code. Easy to debug? Your code should not know nor care that you’re running serverlessly outside of your lambda handler. You test and debug your lambda code just like people have been testing APIs forever. You dry up a test harness that calls the entry point of your domain logic. I don’t mean real AAA automated test, just an entry point for manual debugging.
        qaq 8y ago
        there is startup cost you pay in latency. You have to debug systems in prod not just in dev and debugging lambda used to be pretty crappy experience. "if you are using AWS for more than just hosting VMs" thats the thing the original question was how is better for avg starup than simple DO setup?
        scarface74 8y ago
        You debug systems in prod with lambda just like you do with anything else - via a good logging infrastructure. As far as how it’s better for a startup? With AWS I get hosted Load balancers, multi AZ redundancies, autoscaling, managed queues, databases with failover, a mobile push infrastructure, unlimited storage, etc. All stuff I don’t have to manage myself. There is absolutely no way that I could both develop and babysit infrastructure if it was all just a bunch of VMs. The money we save by not having a dedicated ops department and just one guy who does the grunt work more than makes up for the cost of AWS. Not to mention, the speed at which I can stand up infrastructure.
    - falcolas 8y ago
      Because you're immediately having to go to a MSA, which immediately adds a whole new layer of complexity, monitoring, network latency, API coordination, and configuration to otherwise simple webapps.
      - scarface74 8y ago
        In the case of .Net at least, you write your API like you always do and the SDK provides a wrapper. You can have multiple actions/controllers just like you would with a traditional Web API project. https://aws.amazon.com/blogs/developer/serverless-asp-net-core-2-0-applications/ https://aws.amazon.com/blogs/developer/serverless-asp-net-co... You test locally just like you would any other WebAPI project.
        time0ut 8y ago
        Yup. You can do the same in the other languages as well if you want.
  - tnolet 8y ago
    I would say architecting for Lambda is easier and quicker than “traditional” by now. Was a bit flaky in the beginning, admittedly.
scarface74 8y ago
I would never work for a company that wants to save a few bucks by having as part of their process “manual failovers” that might cause me to wake up in the middle of the night when they could have just allowed me to properly architect the thing in the first place.
- qaq 8y ago
  Ive seen people having to wake up way more often when running overarchitected systems that is beyond their ops capability vs having simple solution that requires manual intervention a few times a year. So the proper solution would be what say for PG: HAProxy in HA configuration + Patroni (so ETCD). So from running 2 postgres instances you went to running 3 postgres instances a distributed database Patroni HAproxy in HA configuration so you need floating IP some heartbeat service etc.
  - scarface74 8y ago
    Or I can just let AWS do the work and not worry about it running HAProxy, Patroni, etc.
    - qaq 8y ago
      Sure but at a very different price point.
      - scarface74 8y ago
        And too often, developers (smaller companies) and ops (larger companies) time is considered free because they are salaried and are expected to work more than 40-45 hours per week. Yes, earlier in my career I would have accepted that, but I’ve successfully pushed back a few times and said I won’t babysit a process/server etc but I will design a method where it is auto healing, autoscaling, etc.
        matwood 8y ago
        As someone who has been both the server baby sitter and implementor of auto-healing/scaling, it still puts a smile on my face when a server fails a health check, AWS kills it, starts another one, and moves the work over. All with zero downtime. I just see the emails in the morning that it happened. The other huge benefit of building this process is we can use dirt cheap spot instances with near abandon. For me, any infrastructure that I can't treat as cattle is on my hit list.

Honest question how is it easier for a startup vs DO with few VPSes and say manual DB fail-over.

by qaq 8y ago

Honest question how is it easier for a startup vs DO with few VPSes and say manual DB fail-over.

Two things, based on my experience (we migrated from Heroku to Lambda in 2016, so been there for a while) 1. you don’t need to worry about reserving capacity, so you don’t need to pay for growth you expect to happen, or worry about not meeting a spike in demand if it happens 2. Most of the operations stuff (apart from packaging) is included in the price, so things like monitoring, alerts, dead-letter queues, traffic shifting beween canary versions, failovers, balancing... and it’s priced per request, so this comes to effectively free if you don’t have a lot of traffic. The big catch is that you don’t control the containers, so there’s no session stickiness. Getting the benefits from Lambda requires re-thinking how you do sessions and storage.

qaq 8y ago
That the thing though the dev overhead and limitations for startup with avg compute needs might be more $ than tiny savings (e.g who cares if it is $200/month vs $70/month) if it will cost 50k extra in dev time to architect for lambda
- scarface74 8y ago
  How is “architecting for lambda” harder than traditional architecture?
  - qaq 8y ago
    having persistent processes ? Not needing API gateway? Not having to learn a whole bunch of AWS services? Not having to work around latency? Having a simple system that easy to debug?
    - scarface74 8y ago
      If by “persistent processes” you mean storing state locally. You usually shouldn’t be counting on that anyway. My VMs are just as disposable as lambda. API Gateway - we use API Gateway even for our VM hosted APIs for the other features it provides. We also integrate with swagger so creating and maintaining our API Gateway set up is basically automatic. Not having to learn a bunch of AWS services - if you are using AWS for more than just hosting VMs, you would have to learn AWS anyway. Even if you are hosting a bunch of VMs, you still need to know about security groups, VPCs, subnets, ELBs, route 53, autoscsling etc. Creating a lambda is adding one function to your code. Easy to debug? Your code should not know nor care that you’re running serverlessly outside of your lambda handler. You test and debug your lambda code just like people have been testing APIs forever. You dry up a test harness that calls the entry point of your domain logic. I don’t mean real AAA automated test, just an entry point for manual debugging.
      - qaq 8y ago
        there is startup cost you pay in latency. You have to debug systems in prod not just in dev and debugging lambda used to be pretty crappy experience. "if you are using AWS for more than just hosting VMs" thats the thing the original question was how is better for avg starup than simple DO setup?
        scarface74 8y ago
        You debug systems in prod with lambda just like you do with anything else - via a good logging infrastructure. As far as how it’s better for a startup? With AWS I get hosted Load balancers, multi AZ redundancies, autoscaling, managed queues, databases with failover, a mobile push infrastructure, unlimited storage, etc. All stuff I don’t have to manage myself. There is absolutely no way that I could both develop and babysit infrastructure if it was all just a bunch of VMs. The money we save by not having a dedicated ops department and just one guy who does the grunt work more than makes up for the cost of AWS. Not to mention, the speed at which I can stand up infrastructure.
  - falcolas 8y ago
    Because you're immediately having to go to a MSA, which immediately adds a whole new layer of complexity, monitoring, network latency, API coordination, and configuration to otherwise simple webapps.
    - scarface74 8y ago
      In the case of .Net at least, you write your API like you always do and the SDK provides a wrapper. You can have multiple actions/controllers just like you would with a traditional Web API project. https://aws.amazon.com/blogs/developer/serverless-asp-net-core-2-0-applications/ https://aws.amazon.com/blogs/developer/serverless-asp-net-co... You test locally just like you would any other WebAPI project.
      - time0ut 8y ago
        Yup. You can do the same in the other languages as well if you want.
- tnolet 8y ago
  I would say architecting for Lambda is easier and quicker than “traditional” by now. Was a bit flaky in the beginning, admittedly.

scarface74 8y ago

I would never work for a company that wants to save a few bucks by having as part of their process “manual failovers” that might cause me to wake up in the middle of the night when they could have just allowed me to properly architect the thing in the first place.

qaq 8y ago
Ive seen people having to wake up way more often when running overarchitected systems that is beyond their ops capability vs having simple solution that requires manual intervention a few times a year. So the proper solution would be what say for PG: HAProxy in HA configuration + Patroni (so ETCD). So from running 2 postgres instances you went to running 3 postgres instances a distributed database Patroni HAproxy in HA configuration so you need floating IP some heartbeat service etc.
- scarface74 8y ago
  Or I can just let AWS do the work and not worry about it running HAProxy, Patroni, etc.
  - qaq 8y ago
    Sure but at a very different price point.
    - scarface74 8y ago
      And too often, developers (smaller companies) and ops (larger companies) time is considered free because they are salaried and are expected to work more than 40-45 hours per week. Yes, earlier in my career I would have accepted that, but I’ve successfully pushed back a few times and said I won’t babysit a process/server etc but I will design a method where it is auto healing, autoscaling, etc.
      - matwood 8y ago
        As someone who has been both the server baby sitter and implementor of auto-healing/scaling, it still puts a smile on my face when a server fails a health check, AWS kills it, starts another one, and moves the work over. All with zero downtime. I just see the emails in the morning that it happened. The other huge benefit of building this process is we can use dirt cheap spot instances with near abandon. For me, any infrastructure that I can't treat as cattle is on my hit list.