14 ms·

I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a yea

I’ve been saying for a couple of years whoever builds a tool to one click deploy an AWS alike to run on local bare metal has built the next billion dollar a year company. I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.

IceWreck 5y ago
But AWS is huge. They employ thousands of engineers. Granted most of the complexity is due to its huge huge scale, but still it has too many features to do what youre asking. And then there are AWS exclusives like DynamoDB. Yes there are open source public/private cloud alternatives like OpenStack but they won't solve the problem of lock in and you'd still have to change all your tooling.
- sekh60 5y ago
  OpenStack isn't a redhat project, it is contributed to by a large number of players. Redhat, SUSE and Canonical all provide their own OpenStack service subscriptions, but one can easily deploy OpenStack with a project like OpenStack Kolla. I run it at home and it is great.
  - Sanguinaire 5y ago
    OpenStack is an over-complicated mess. Most enterprise IT orgs already struggle to hire the people with skills to operate a public cloud account, nevermind having to build the underlying infrastructure as well.
    - sekh60 5y ago
      I suppose it is complicated, just a homelabber here, but I don't think it's that bad. I deployed my cluster at home using kolla, and I've had relatively few hiccups that non-sysadmin/non-coder me couldn't get around. I've had the same cluster going for a few openstack releases now. Kolla really makes it easy. That said before I found Kolla I did deploy openstack manually a few times before messing things up and having to restart.
- Spivak 5y ago
  I mean the value is really implementing AWS’s API surface area in a single control-plane overtop of the usual on-prem tooling. K8s is kinda getting there but is too narrow in scope I think for what people actually want.
- unethical_ban 5y ago
  I think it would not be impossible for a talented, well-funded group to pick the top 20% of AWS's services and then implement them with the top 80% of features, with an API-compatible layer. Even "just" to emulate a single region with 3 AZs would be useful for testing and for prod for many people. I know of several large organizations that to this day haven't scaled their CI beyond one region.
  - GordonS 5y ago
    This makes sense; I would guess that 90% of AWS customers are only using the top 10% of available AWS features. Same deal with Azure and GCP, although note that Azure has Azure Stack, which is a self-hosted version of Azure (I guess it's still expensive though, and I don't know how many services it includes).
- belter 5y ago
  You can install DynamoDB locally although you will still miss the rest of the stack. "Setting Up DynamoDB Local (Downloadable Version)" https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
  - jdreaver 5y ago
    https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... > The downloadable version of DynamoDB is intended for development and testing purposes only. By comparison, the DynamoDB web service is a managed service with scalability, availability, and durability features that make it ideal for production use.
    - belter 5y ago
      Correct. I meant to use it to save some money while developing your apps, not as a solution for local deployment.
      - jdreaver 5y ago
        Ah understood. I thought this comment thread was talking about replacing _production_ AWS use cases with a bare metal system that has a compatible API.
        belter 5y ago
        LocalStack says its about testing: "...Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow..."
- kortilla 5y ago
  > they won't solve the problem of lock in Conflating lock in to an open source tool with lock in to a vendor who you literally have to keep paying money to survive is a shitty tactic used by companies like Amazon. Using open stack absolutely solves the problem of lock in from a business perspective. You can’t be cancelled, the price can’t be changed, etc.
  - NathanKP 5y ago
    I legitimately wonder how many people feel free from lock in after they finish paying $75k to Canonical for a 12 node OpenStack "private cloud build" (https://assets.ubuntu.com/v1/d4766546-Private-Cloud-Build_Datasheet.pdf https://assets.ubuntu.com/v1/d4766546-Private-Cloud-Build_Da...), not to mention ongoing support bills and billing for addons "starting at $25k". Just because something is open source does not mean your org will actually have the expertise to be successful at running it yourself. That's how most "open source" projects like Kubernetes and OpenStack become so successful: the supporting orgs behind the project make lots of money charging companies for support and maintenance and issues that pop up with the setup. There are many forms of lock in. Buying a big upfront investment that has to pay itself off for the next X years is a form of lock in. Paying for expensive professional consulting support to build out the system and keep the system up and running is a form of lock in. Or even worse, hiring or training expensive dedicated employees to learn the open source system and become the internal professional consultants for the rest of the org, that is also lock in.
    - kortilla 5y ago
      > I legitimately wonder how many people feel free from lock in after they finish paying $75k to Canonical for a 12 node OpenStack "private cloud build" Here’s the part where it’s not lock-in. We stopped paying for support from Redhat when we got big enough that our own SRE expertise to run our own open stack cloud. You know what the impact was on our applications running on it? Absolutely nothing. How much do you pay Canonical to use grep every month? Inexperienced developers have just been conditioned by cloud providers to think that an IaaS platform must cost something in payments per month to some tech company. It does not. This is no different than Windows vs Linux on servers. I can’t wait until 20 years from now when all of the proprietary shit looks ridiculous in hindsight.
- thayne 5y ago
  For DynamoDB specifically, ScyllaDB had a Dynamodb compatible mode. There are several options (open source and otherwise) with s3 compatible APIs. I think it is more likely we'll end up with replacements for individual services and maybe at some point it will be possible to glue them together as a replacement for a significant subset of AWS
- Aeolun 5y ago
  Not necessarily if your end goal is to be a unicorn. You can get a lot of investment for that.
- dimitrios1 5y ago
  What's so exclusive about a half-baked Dynamo implementation? Dynamo is just a worse Cassandra. DynamoDB itself doesn't even implement the Dynamo Paper.
salex89 5y ago
Ever looked into Nutanix (https://www.nutanix.com/ https://www.nutanix.com/)?
simonh 5y ago
I worked on a system called Quartz at Bank of America. It wasn't an AWS clone, but was targeting the same flexibility for the development and deployment of applications on 'programmable' infrastructure. The team that built it had done similar projects before at J. P. Morgan and Goldmans. It was a set of infrastructure and services all accessed through Python APIs, with YAML config, all stored in a single distributed version control system. It was a lot of fun to work on.
- whummer 5y ago
  Wow that sounds super interesting! If you can shoot us a message to info@localstack.cloud we'd love to chat and learn more about your journey!
- no_wizard 5y ago
  I would not be shocked if they just had some abstractions built around Ansible[0], it definitely fits the description [0]: https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html https://docs.ansible.com/ansible/latest/user_guide/playbooks... Intro to playbooks - Ansible Documentation
  - simonh 5y ago
    It’s very different, much more like AWS or Azure. The underlying infra is there, you just allocate resources using config and APIs. For example there’s a server farm for running server applications such as batch processing or web applications. To stand up a new application you develop the app on a local instance of the Quartz Python runtime, check it into version control, then to deploy it by configuring a YAML file in version control that defines which server farm it runs on, the path to the code in VC, the schedule and various resource limits. That’s about it. Job done.
- magmasystems 5y ago
  Wow, my brother works on Quartz. But I think that half of BofA does. Higgins and Kirat formed their own company to bring the Quartz/Athena paradigm to the masses.
bob229 5y ago
Which will never happen you child
rkeene2 5y ago
I built this a few times. The most current implementation is called CloudSeed, though I am trying to fundraise for an open source version (with a RedHat-like business model of support). CloudSeed has been a key part of several >$1bn contracts for KnightPoint/Perspecta/Peraton, most recently DHS DCCO ($2.7bn).
asim 5y ago
Agreed. Although it's a big ask. I have something that's slowly going in that direction but it'll be a while before it gets the interest I'm looking for. https://github.com/micro https://github.com/micro
ragona 5y ago
I think this misunderstands the hard part of running AWS. You're paying for the software, sure. What you're really paying for is the hundreds of oncall engineers supporting you at all times, and the culture of ensuring that those services keep working.
- kevinsundar 5y ago
  As an Amazon engineer who frequently goes oncall, this is Amazon’s true competitive advantage. The company works because every night there’s 1000s of engineers sleeping with a pager by their head.
  - devit 5y ago
    Why do that for general customer support instead of hiring in different time zones?
    - hansel_der 5y ago
      cultural cohesion? we have often delayed calls because it's often hard to explain it to some "foreigner"
    - kevinsundar 5y ago
      Because minutes after a metric drops that indicates the possibility of an issue an engineer who probably wrote the code or has the code checked out on their laptop and is intimately familiar with it working on the problem. No waiting for a support team to triage, find the owner, and get ahold of them.
kongin 5y ago
It's kind of stunning that starting with fully open technologies we build another closed mainframe on top of it. All that's old is new again.
- whummer 5y ago
  Agreed, that's really stunning - seen the analogy of cloud as the "modern mainframe" come up more and more recently. I guess the (business) reason is that the centralized computing model enables enormous economies of scale. (Makes perfect sense from an operations point of view.) What it seems to miss in the current incarnation/iteration, though, is the "power of distribution" - leveraging the fact that local compute capabilities (especially during development/integration) can help reduce the cognitive load, increase the efficiency, and thereby reduce overall costs. There seems to be a tendency to focus mostly on the operational aspects, rather than the overall end-to-end developer journey. What remains to be seen is whether future iterations of "the cloud" will do a better job of embracing the power of distributed and/or hybrid.
  - kongin 5y ago
    The reason why AWS got popular everywhere I worked was that you didn't need to get buying new hardware past ops. You just spun up your own, then when it was supporting half the business you pointed at it and said "Gee wouldn't it be nice if we had a box in our own data center to run it?" then there'd be a fire lit under the ass of ops and you'd get your computers in a week instead of next financial year. Now it's exactly the same thing in AWS. My next guess is that you're going to be running production code on fleets of devs computers because you don't have to get extra budget for AWS next financial year to afford spinning up another instance.
1vuio0pswjnm7 5y ago
Need more details. Matching AWS service for service would be overkill I imagine, so what services does this need to have. Is hardware virtualisation is must-have. The way I made systems/applications before AWS existed was to create bootable images. I could already use a hypervisor, Xen, because the OS fully supported both guest and host, all virtualisation modes, before Linux did, and before AWS existed. But because I am not a hosting provider I saw little need for virtualisation. The OS I used also had "unikernel" capability, before Docker, etc. existed. As it happened, this inexpensive, self-determination was not to be the future, instead we got "the cloud". Sharing servers with other "tenants". Less expensive for the hosting provider, but more expensive and more limited for the subscriber. No argument, the "limited" part has improved since then, but it is still expensive (for continuous use that is, spot pricing was a neat benefit of "the cloud"). Anyway, I "deploy" images to "local bare metal" which is a laptop, RPi, or some other smaller form factor. I can use unikernel for some kernel drivers in userspace. Basic filesystem is embedded in the kernel, and larger filesystem is on the USB stick filesystem in compressed format. Updates are easy enough. I put two kernels on the USB stick, one is the running kernel and the other is the update kernel. Same for the larger filesystem which may contain servers and configuration files. Can update or go back to last working kernel/configuration by selecting one or the other in the boot menu/renaming the larger compressed filesystem. Here is someone running a search engine out of his living room. AKAIK, his setup survived a sustained HN thundering herd, hug of death without a hiccup. https://news.ycombinator.com/item?id=28552805 https://news.ycombinator.com/item?id=28552805 The expense of AWS is an obvious point of discussion but another one not mentioned here is control. When I create images for bare metal I do not need to jump through any hoops as I would in order to create an image that will run on AWS. Nor do I need to fiddle with all the AWS knobs. There are no silly marketing names for every program I run. I know the system I am creating as well as I know the OS and the software I choose. That is much better than how well I know every aspect of AWS which just gets more and more complex every year. AWS documentation is as cringeworthy as it it is voluminous. The ever-increasing complexity of AWS, including the "tooling", is, IMO, How To Create Lock-In 101
- pdimitar 5y ago
  I would absolutely love reading you blogging in details about this. It's a lost art, and one I've ever been good at. I want to lately, but people like you are always undercover and don't document their knowledge. :(
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
SteveNuts 5y ago
Nutanix is getting pretty good at this. My enterprise style company switched from AWS to nutanix this year, we bought three all nvme/ssd nodes with 1TB memory in each. We were heavy EC2/RDS users, now we run their NX whitebox nodes with their KVM based hypervisor, and their database management platform "Era". Everything is one click for deployment and upgrading of things, down to stuff like upgrading the SSD firmware, NIC firmware, etc. It auto migrates the VMs around hot, and does one node at a time. They also have a kubernetes platform called Karbon that seems to be pretty good.
TonyCoffman 5y ago
Eucalyptus built a clone. It wasn’t one click but it also wasn’t hard to deploy. It really never made much of a dent. I’m pretty sure it ended up where all software goes to die: HP.
- rzzzt 5y ago
  The website is still up and points to a fork on GitHub: - https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/intro/oview/ https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/int... - https://github.com/corymbia/eucalyptus/ https://github.com/corymbia/eucalyptus/ There are quite a few moving parts. I think I got stuck around just comprehending the networking bits.
SkyRocknRoll 5y ago
Nutanix is strong contender for this use case. We see lot of workloads moving from Cloud to onprem on nutanix stack. P.S I work on engineering side in nutanix. https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap-cloud-lifecycle-scale-growth-repatriation-optimization/ https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...
phendrenad2 5y ago
But why? Because they can run an on-prem production "AWS"? Or because it's easier for testing? If the former, well, sure, but why stop there? Why not an easy on-prem Heroku clone? If the latter, I don't think that goes far enough. Simply emulating AWS (as LocalStack does) is barely usable for testing. You really would want a GUI that lets you see everything in the LocalStack config and data stores. Running a fake DynamoDB is okay, but it's painful if you need to use DynamoDB queries to see the data.
randmeerkat 5y ago
So are you saying that the tooling needs to deploy to their datacenters or AWS, or just deploying to their own datacenters needs to be as simple as using AWS? If it’s both, use Kubernetes. If it’s deploying to their own datacenters, then maybe a new product could disrupt that space. You would still need to have a team competent in datacenter operations and hardware procurement though.

IceWreck 5y ago

But AWS is huge. They employ thousands of engineers. Granted most of the complexity is due to its huge huge scale, but still it has too many features to do what youre asking. And then there are AWS exclusives like DynamoDB. Yes there are open source public/private cloud alternatives like OpenStack but they won't solve the problem of lock in and you'd still have to change all your tooling.

sekh60 5y ago
OpenStack isn't a redhat project, it is contributed to by a large number of players. Redhat, SUSE and Canonical all provide their own OpenStack service subscriptions, but one can easily deploy OpenStack with a project like OpenStack Kolla. I run it at home and it is great.
- Sanguinaire 5y ago
  OpenStack is an over-complicated mess. Most enterprise IT orgs already struggle to hire the people with skills to operate a public cloud account, nevermind having to build the underlying infrastructure as well.
  - sekh60 5y ago
    I suppose it is complicated, just a homelabber here, but I don't think it's that bad. I deployed my cluster at home using kolla, and I've had relatively few hiccups that non-sysadmin/non-coder me couldn't get around. I've had the same cluster going for a few openstack releases now. Kolla really makes it easy. That said before I found Kolla I did deploy openstack manually a few times before messing things up and having to restart.
Spivak 5y ago
I mean the value is really implementing AWS’s API surface area in a single control-plane overtop of the usual on-prem tooling. K8s is kinda getting there but is too narrow in scope I think for what people actually want.
unethical_ban 5y ago
I think it would not be impossible for a talented, well-funded group to pick the top 20% of AWS's services and then implement them with the top 80% of features, with an API-compatible layer. Even "just" to emulate a single region with 3 AZs would be useful for testing and for prod for many people. I know of several large organizations that to this day haven't scaled their CI beyond one region.
- GordonS 5y ago
  This makes sense; I would guess that 90% of AWS customers are only using the top 10% of available AWS features. Same deal with Azure and GCP, although note that Azure has Azure Stack, which is a self-hosted version of Azure (I guess it's still expensive though, and I don't know how many services it includes).
belter 5y ago
You can install DynamoDB locally although you will still miss the rest of the stack. "Setting Up DynamoDB Local (Downloadable Version)" https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
- jdreaver 5y ago
  https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html https://docs.aws.amazon.com/amazondynamodb/latest/developerg... > The downloadable version of DynamoDB is intended for development and testing purposes only. By comparison, the DynamoDB web service is a managed service with scalability, availability, and durability features that make it ideal for production use.
  - belter 5y ago
    Correct. I meant to use it to save some money while developing your apps, not as a solution for local deployment.
    - jdreaver 5y ago
      Ah understood. I thought this comment thread was talking about replacing _production_ AWS use cases with a bare metal system that has a compatible API.
      - belter 5y ago
        LocalStack says its about testing: "...Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow..."
kortilla 5y ago
> they won't solve the problem of lock in Conflating lock in to an open source tool with lock in to a vendor who you literally have to keep paying money to survive is a shitty tactic used by companies like Amazon. Using open stack absolutely solves the problem of lock in from a business perspective. You can’t be cancelled, the price can’t be changed, etc.
- NathanKP 5y ago
  I legitimately wonder how many people feel free from lock in after they finish paying $75k to Canonical for a 12 node OpenStack "private cloud build" (https://assets.ubuntu.com/v1/d4766546-Private-Cloud-Build_Datasheet.pdf https://assets.ubuntu.com/v1/d4766546-Private-Cloud-Build_Da...), not to mention ongoing support bills and billing for addons "starting at $25k". Just because something is open source does not mean your org will actually have the expertise to be successful at running it yourself. That's how most "open source" projects like Kubernetes and OpenStack become so successful: the supporting orgs behind the project make lots of money charging companies for support and maintenance and issues that pop up with the setup. There are many forms of lock in. Buying a big upfront investment that has to pay itself off for the next X years is a form of lock in. Paying for expensive professional consulting support to build out the system and keep the system up and running is a form of lock in. Or even worse, hiring or training expensive dedicated employees to learn the open source system and become the internal professional consultants for the rest of the org, that is also lock in.
  - kortilla 5y ago
    > I legitimately wonder how many people feel free from lock in after they finish paying $75k to Canonical for a 12 node OpenStack "private cloud build" Here’s the part where it’s not lock-in. We stopped paying for support from Redhat when we got big enough that our own SRE expertise to run our own open stack cloud. You know what the impact was on our applications running on it? Absolutely nothing. How much do you pay Canonical to use grep every month? Inexperienced developers have just been conditioned by cloud providers to think that an IaaS platform must cost something in payments per month to some tech company. It does not. This is no different than Windows vs Linux on servers. I can’t wait until 20 years from now when all of the proprietary shit looks ridiculous in hindsight.
thayne 5y ago
For DynamoDB specifically, ScyllaDB had a Dynamodb compatible mode. There are several options (open source and otherwise) with s3 compatible APIs. I think it is more likely we'll end up with replacements for individual services and maybe at some point it will be possible to glue them together as a replacement for a significant subset of AWS
Aeolun 5y ago
Not necessarily if your end goal is to be a unicorn. You can get a lot of investment for that.
dimitrios1 5y ago
What's so exclusive about a half-baked Dynamo implementation? Dynamo is just a worse Cassandra. DynamoDB itself doesn't even implement the Dynamo Paper.

salex89 5y ago

Ever looked into Nutanix (https://www.nutanix.com/ https://www.nutanix.com/)?

simonh 5y ago

I worked on a system called Quartz at Bank of America. It wasn't an AWS clone, but was targeting the same flexibility for the development and deployment of applications on 'programmable' infrastructure. The team that built it had done similar projects before at J. P. Morgan and Goldmans. It was a set of infrastructure and services all accessed through Python APIs, with YAML config, all stored in a single distributed version control system. It was a lot of fun to work on.

whummer 5y ago
Wow that sounds super interesting! If you can shoot us a message to info@localstack.cloud we'd love to chat and learn more about your journey!
no_wizard 5y ago
I would not be shocked if they just had some abstractions built around Ansible[0], it definitely fits the description [0]: https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html https://docs.ansible.com/ansible/latest/user_guide/playbooks... Intro to playbooks - Ansible Documentation
- simonh 5y ago
  It’s very different, much more like AWS or Azure. The underlying infra is there, you just allocate resources using config and APIs. For example there’s a server farm for running server applications such as batch processing or web applications. To stand up a new application you develop the app on a local instance of the Quartz Python runtime, check it into version control, then to deploy it by configuring a YAML file in version control that defines which server farm it runs on, the path to the code in VC, the schedule and various resource limits. That’s about it. Job done.
magmasystems 5y ago
Wow, my brother works on Quartz. But I think that half of BofA does. Higgins and Kirat formed their own company to bring the Quartz/Athena paradigm to the masses.

bob229 5y ago

Which will never happen you child

rkeene2 5y ago

I built this a few times. The most current implementation is called CloudSeed, though I am trying to fundraise for an open source version (with a RedHat-like business model of support). CloudSeed has been a key part of several >$1bn contracts for KnightPoint/Perspecta/Peraton, most recently DHS DCCO ($2.7bn).

asim 5y ago

Agreed. Although it's a big ask. I have something that's slowly going in that direction but it'll be a while before it gets the interest I'm looking for. https://github.com/micro https://github.com/micro

ragona 5y ago

I think this misunderstands the hard part of running AWS. You're paying for the software, sure. What you're really paying for is the hundreds of oncall engineers supporting you at all times, and the culture of ensuring that those services keep working.

kevinsundar 5y ago
As an Amazon engineer who frequently goes oncall, this is Amazon’s true competitive advantage. The company works because every night there’s 1000s of engineers sleeping with a pager by their head.
- devit 5y ago
  Why do that for general customer support instead of hiring in different time zones?
  - hansel_der 5y ago
    cultural cohesion? we have often delayed calls because it's often hard to explain it to some "foreigner"
  - kevinsundar 5y ago
    Because minutes after a metric drops that indicates the possibility of an issue an engineer who probably wrote the code or has the code checked out on their laptop and is intimately familiar with it working on the problem. No waiting for a support team to triage, find the owner, and get ahold of them.

kongin 5y ago

It's kind of stunning that starting with fully open technologies we build another closed mainframe on top of it. All that's old is new again.

whummer 5y ago
Agreed, that's really stunning - seen the analogy of cloud as the "modern mainframe" come up more and more recently. I guess the (business) reason is that the centralized computing model enables enormous economies of scale. (Makes perfect sense from an operations point of view.) What it seems to miss in the current incarnation/iteration, though, is the "power of distribution" - leveraging the fact that local compute capabilities (especially during development/integration) can help reduce the cognitive load, increase the efficiency, and thereby reduce overall costs. There seems to be a tendency to focus mostly on the operational aspects, rather than the overall end-to-end developer journey. What remains to be seen is whether future iterations of "the cloud" will do a better job of embracing the power of distributed and/or hybrid.
- kongin 5y ago
  The reason why AWS got popular everywhere I worked was that you didn't need to get buying new hardware past ops. You just spun up your own, then when it was supporting half the business you pointed at it and said "Gee wouldn't it be nice if we had a box in our own data center to run it?" then there'd be a fire lit under the ass of ops and you'd get your computers in a week instead of next financial year. Now it's exactly the same thing in AWS. My next guess is that you're going to be running production code on fleets of devs computers because you don't have to get extra budget for AWS next financial year to afford spinning up another instance.

1vuio0pswjnm7 5y ago

Need more details. Matching AWS service for service would be overkill I imagine, so what services does this need to have. Is hardware virtualisation is must-have. The way I made systems/applications before AWS existed was to create bootable images. I could already use a hypervisor, Xen, because the OS fully supported both guest and host, all virtualisation modes, before Linux did, and before AWS existed. But because I am not a hosting provider I saw little need for virtualisation. The OS I used also had "unikernel" capability, before Docker, etc. existed. As it happened, this inexpensive, self-determination was not to be the future, instead we got "the cloud". Sharing servers with other "tenants". Less expensive for the hosting provider, but more expensive and more limited for the subscriber. No argument, the "limited" part has improved since then, but it is still expensive (for continuous use that is, spot pricing was a neat benefit of "the cloud"). Anyway, I "deploy" images to "local bare metal" which is a laptop, RPi, or some other smaller form factor. I can use unikernel for some kernel drivers in userspace. Basic filesystem is embedded in the kernel, and larger filesystem is on the USB stick filesystem in compressed format. Updates are easy enough. I put two kernels on the USB stick, one is the running kernel and the other is the update kernel. Same for the larger filesystem which may contain servers and configuration files. Can update or go back to last working kernel/configuration by selecting one or the other in the boot menu/renaming the larger compressed filesystem. Here is someone running a search engine out of his living room. AKAIK, his setup survived a sustained HN thundering herd, hug of death without a hiccup. https://news.ycombinator.com/item?id=28552805 https://news.ycombinator.com/item?id=28552805 The expense of AWS is an obvious point of discussion but another one not mentioned here is control. When I create images for bare metal I do not need to jump through any hoops as I would in order to create an image that will run on AWS. Nor do I need to fiddle with all the AWS knobs. There are no silly marketing names for every program I run. I know the system I am creating as well as I know the OS and the software I choose. That is much better than how well I know every aspect of AWS which just gets more and more complex every year. AWS documentation is as cringeworthy as it it is voluminous. The ever-increasing complexity of AWS, including the "tooling", is, IMO, How To Create Lock-In 101

pdimitar 5y ago
I would absolutely love reading you blogging in details about this. It's a lost art, and one I've ever been good at. I want to lately, but people like you are always undercover and don't document their knowledge. :(

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

SteveNuts 5y ago

Nutanix is getting pretty good at this. My enterprise style company switched from AWS to nutanix this year, we bought three all nvme/ssd nodes with 1TB memory in each. We were heavy EC2/RDS users, now we run their NX whitebox nodes with their KVM based hypervisor, and their database management platform "Era". Everything is one click for deployment and upgrading of things, down to stuff like upgrading the SSD firmware, NIC firmware, etc. It auto migrates the VMs around hot, and does one node at a time. They also have a kubernetes platform called Karbon that seems to be pretty good.

TonyCoffman 5y ago

Eucalyptus built a clone. It wasn’t one click but it also wasn’t hard to deploy. It really never made much of a dent. I’m pretty sure it ended up where all software goes to die: HP.

rzzzt 5y ago
The website is still up and points to a fork on GitHub: - https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/intro/oview/ https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/int... - https://github.com/corymbia/eucalyptus/ https://github.com/corymbia/eucalyptus/ There are quite a few moving parts. I think I got stuck around just comprehending the networking bits.

SkyRocknRoll 5y ago

Nutanix is strong contender for this use case. We see lot of workloads moving from Cloud to onprem on nutanix stack. P.S I work on engineering side in nutanix. https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap-cloud-lifecycle-scale-growth-repatriation-optimization/ https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...

phendrenad2 5y ago

But why? Because they can run an on-prem production "AWS"? Or because it's easier for testing? If the former, well, sure, but why stop there? Why not an easy on-prem Heroku clone? If the latter, I don't think that goes far enough. Simply emulating AWS (as LocalStack does) is barely usable for testing. You really would want a GUI that lets you see everything in the LocalStack config and data stores. Running a fake DynamoDB is okay, but it's painful if you need to use DynamoDB queries to see the data.

randmeerkat 5y ago

So are you saying that the tooling needs to deploy to their datacenters or AWS, or just deploying to their own datacenters needs to be as simple as using AWS? If it’s both, use Kubernetes. If it’s deploying to their own datacenters, then maybe a new product could disrupt that space. You would still need to have a team competent in datacenter operations and hardware procurement though.