5 ms·

Automating cloud architecture creation. I'm building a "catalog" of architectures that you could use to create a complete cloud architecture on your AWS, GCP o

Automating cloud architecture creation. I'm building a "catalog" of architectures that you could use to create a complete cloud architecture on your AWS, GCP or Azure account in less than one minute. So, for example, you could create a docker-based architecture with CI/CD, auto-scaling, zero downtime deployment, SSL, load-balancing, high availability and MongoDB in less than one minute in your own AWS account. It's like Terraform with the user-friendliness of Heroku. It's very hard because every providers have different APIs and concepts so you have to start from scratch for each. I love working on it because cloud-computing may have so much impact in some organizations like biotech startups or NGOs.

ahnberg 6y ago
Do you have something published yet? Code, articles or anything? Sounds very interesting!
- jeremylevy 6y ago
  Thanks! I've released an alpha version three days ago. I'm not very comfortable using a comment for promotional purpose so if you would like to try it out, drop me an email at jje.levy@gmail.com, I will send you the link. I will try to make a Show HN post in the following weeks.
hashamali 6y ago
Have you considered something like Pulumi for the glue? Love this idea, the ease of Heroku + the flexibility of Kubernetes would be amazing.
- jeremylevy 6y ago
  Thanks! Yes! I used Terraform because it was the tool I know best but Pulumi seems very elegant.
jonwalch 6y ago
Any thoughts about maintenance? My personal journey with learning AWS is/was very painful, but now that I understand everything that I need at a high level, it is quite a bit more manageable. I can't imagine having to make minor tweaks to my infra if I had no idea what it was doing.
- jeremylevy 6y ago
  It's something I've been thinking about for a while. I think it will depend on the people that will use it. Some architecture require more maintenance than other. For example, the first architecture that I've created is a serverless architecture based on Docker. Under the hood it only uses managed services like Fargate, CodePipeline, CodeBuild, Aurora... Given that you'll have a catalog of different architectures, people that don't have any experience with AWS may need to choose architectures like that. Also, each architecture contains a detailed Readme that explain the basic operations (updating ELB HealthCheck, increasing the amount of RAM that could be used by a Fargate task and so on...). It's a complex subject, by the way, so I am entirely open to any comments, ideas or suggestions.
tlrobinson 6y ago
> It's like Terraform with the user-friendliness of Heroku. > It's very hard because every providers have different APIs and concepts so you have to start from scratch for each. Why not build it as a layer on top of Terraform?
- threentaway 6y ago
  This was my first thought. Terraform modules can accomplish all of this.
- jeremylevy 6y ago
  I do use Terraform. Terraform is very good to create the building blocks of your architecture but not so much for the user-friendliness part. Let's take an example: You have an architecture with a CI/CD. Great. You add your CodePipeline and CodeBuild ressources to your plan. Perfect. For the CI part, you want your build to start on every commit on every non-master branch. Bad news: CodePipeline doesn't have support for multi-branch. So you will need to find a way to clone the default pipeline for each new branch. Repeat this for all the user-friendly features (GitHub check runs, env vars management, deployment monitoring/rollback...) for the three cloud providers for all the different architectures and you start to feel the difficulty (I can testify ;)).
  - tlrobinson 6y ago
    I’m suggesting your tool could generate Terraform code and run the Terraform command “under the hood” so you don’t have to reimplement everything Terraform already does. Users wouldn’t need to know about Terraform (unless they wanted to. It might be nice to have a way to export the Terraform code)
    - jeremylevy 6y ago
      Sorry! My answer was certainly not very clear. I use Terraform “under the hood” for this project. What I was trying to explain is that the "hard part" is not the creation of the architecture components but the user-friendly features.
mattjoyce 6y ago
That does sound challenging. APIs and service change too. I can't imagine how complex that can get if something stops working. Still when you succeed perhaps the next goal is to simply specify what sort of compute function, the scale and risk tolerances, and let the patterns be assembled accordingly.
- jeremylevy 6y ago
  Yes, you're right. Even if you can lock the API version of the SDKs, you will still need to update all the architectures with the new features and changes. It's very challenging but I love working on it!
busted 6y ago
I've thought about this in the past. There's a modern way to build architecture, and cloud providers are in general only providing the building blocks and requiring you to put them together. It's unscalable to expect every company to hire someone smart enough to construct a good architecture (let alone the time) but at the same time, people who have been working on infrastructure long enough know that there's really only a handful of useful architectures that solve 90% of problems. I thought of it as CTO-as-a-service or CTO-in-a-box. Service meshes like istio.io start to solve a portion of this. Almost certainly there is endless complexity but I bet you can come up with something useful. Good luck!
- jeremylevy 6y ago
  Thanks! > people who have been working on infrastructure long enough know that there's really only a handful of useful architectures that solve 90% of problems I totally agree with you on that point. Also, given that architectures are created on your own cloud provider account you could easily add the missing parts.
derision 6y ago
Check out gruntwork.io
- jeremylevy 6y ago
  Gruntwork is very good to generate your infrastructure as code, but in my opinion it misses the "user-friendly" part. What do you think about it?
anewlanguage 6y ago
I've often found myself wanting something like this, a bootstrap to spin something relatively complex up quickly and then start customizing it. I think it would be nice if there was a way to export to the format native to whatever cloud you're targeting, e.g. CloudFoundry. This is something I've found missing in tools like Serverless Framework.
- jeremylevy 6y ago
  Thanks! For the export feature, It's something I've been thinking about for a while. Given that I use Terraform under the hood, I've been thinking about a feature to merge the base architecture plan with your custom plan. So you could have a catalog of "base" architectures and another catalog of "custom" architectures that you could spin up in seconds. What do you think about it?
lapnitnelav 6y ago
This so much. One of the main reason the big players dominate the Web (in the marketing space) so much is because SaaS is easier to handle (for most people) than having to deploy and maintain your infra. if you streamline the process so that your regular SMB can build their digital infrastructure and maintain their private data clouds, it would go a long way to help us all with privacy. A suite of open-source / on premise license one stop shop.
- jeremylevy 6y ago
  Thanks! I hope I won't disappoint you.
Addono 6y ago
Reminds me a bit of https://juju.is/ https://juju.is/. Haven't looked at it much, but it seems super powerfull and tries to solve at least some of the issues you mentioned.
- jeremylevy 6y ago
  Thanks! I didn't know Juju, I'll try it out!

ahnberg 6y ago

Do you have something published yet? Code, articles or anything? Sounds very interesting!

jeremylevy 6y ago
Thanks! I've released an alpha version three days ago. I'm not very comfortable using a comment for promotional purpose so if you would like to try it out, drop me an email at jje.levy@gmail.com, I will send you the link. I will try to make a Show HN post in the following weeks.

hashamali 6y ago

Have you considered something like Pulumi for the glue? Love this idea, the ease of Heroku + the flexibility of Kubernetes would be amazing.

jeremylevy 6y ago
Thanks! Yes! I used Terraform because it was the tool I know best but Pulumi seems very elegant.

jonwalch 6y ago

Any thoughts about maintenance? My personal journey with learning AWS is/was very painful, but now that I understand everything that I need at a high level, it is quite a bit more manageable. I can't imagine having to make minor tweaks to my infra if I had no idea what it was doing.

jeremylevy 6y ago
It's something I've been thinking about for a while. I think it will depend on the people that will use it. Some architecture require more maintenance than other. For example, the first architecture that I've created is a serverless architecture based on Docker. Under the hood it only uses managed services like Fargate, CodePipeline, CodeBuild, Aurora... Given that you'll have a catalog of different architectures, people that don't have any experience with AWS may need to choose architectures like that. Also, each architecture contains a detailed Readme that explain the basic operations (updating ELB HealthCheck, increasing the amount of RAM that could be used by a Fargate task and so on...). It's a complex subject, by the way, so I am entirely open to any comments, ideas or suggestions.

tlrobinson 6y ago

> It's like Terraform with the user-friendliness of Heroku. > It's very hard because every providers have different APIs and concepts so you have to start from scratch for each. Why not build it as a layer on top of Terraform?

threentaway 6y ago
This was my first thought. Terraform modules can accomplish all of this.
jeremylevy 6y ago
I do use Terraform. Terraform is very good to create the building blocks of your architecture but not so much for the user-friendliness part. Let's take an example: You have an architecture with a CI/CD. Great. You add your CodePipeline and CodeBuild ressources to your plan. Perfect. For the CI part, you want your build to start on every commit on every non-master branch. Bad news: CodePipeline doesn't have support for multi-branch. So you will need to find a way to clone the default pipeline for each new branch. Repeat this for all the user-friendly features (GitHub check runs, env vars management, deployment monitoring/rollback...) for the three cloud providers for all the different architectures and you start to feel the difficulty (I can testify ;)).
- tlrobinson 6y ago
  I’m suggesting your tool could generate Terraform code and run the Terraform command “under the hood” so you don’t have to reimplement everything Terraform already does. Users wouldn’t need to know about Terraform (unless they wanted to. It might be nice to have a way to export the Terraform code)
  - jeremylevy 6y ago
    Sorry! My answer was certainly not very clear. I use Terraform “under the hood” for this project. What I was trying to explain is that the "hard part" is not the creation of the architecture components but the user-friendly features.

mattjoyce 6y ago

That does sound challenging. APIs and service change too. I can't imagine how complex that can get if something stops working. Still when you succeed perhaps the next goal is to simply specify what sort of compute function, the scale and risk tolerances, and let the patterns be assembled accordingly.

jeremylevy 6y ago
Yes, you're right. Even if you can lock the API version of the SDKs, you will still need to update all the architectures with the new features and changes. It's very challenging but I love working on it!

busted 6y ago

I've thought about this in the past. There's a modern way to build architecture, and cloud providers are in general only providing the building blocks and requiring you to put them together. It's unscalable to expect every company to hire someone smart enough to construct a good architecture (let alone the time) but at the same time, people who have been working on infrastructure long enough know that there's really only a handful of useful architectures that solve 90% of problems. I thought of it as CTO-as-a-service or CTO-in-a-box. Service meshes like istio.io start to solve a portion of this. Almost certainly there is endless complexity but I bet you can come up with something useful. Good luck!

jeremylevy 6y ago
Thanks! > people who have been working on infrastructure long enough know that there's really only a handful of useful architectures that solve 90% of problems I totally agree with you on that point. Also, given that architectures are created on your own cloud provider account you could easily add the missing parts.

derision 6y ago

Check out gruntwork.io

jeremylevy 6y ago
Gruntwork is very good to generate your infrastructure as code, but in my opinion it misses the "user-friendly" part. What do you think about it?

anewlanguage 6y ago

I've often found myself wanting something like this, a bootstrap to spin something relatively complex up quickly and then start customizing it. I think it would be nice if there was a way to export to the format native to whatever cloud you're targeting, e.g. CloudFoundry. This is something I've found missing in tools like Serverless Framework.

jeremylevy 6y ago
Thanks! For the export feature, It's something I've been thinking about for a while. Given that I use Terraform under the hood, I've been thinking about a feature to merge the base architecture plan with your custom plan. So you could have a catalog of "base" architectures and another catalog of "custom" architectures that you could spin up in seconds. What do you think about it?

lapnitnelav 6y ago

This so much. One of the main reason the big players dominate the Web (in the marketing space) so much is because SaaS is easier to handle (for most people) than having to deploy and maintain your infra. if you streamline the process so that your regular SMB can build their digital infrastructure and maintain their private data clouds, it would go a long way to help us all with privacy. A suite of open-source / on premise license one stop shop.

jeremylevy 6y ago
Thanks! I hope I won't disappoint you.

Addono 6y ago

Reminds me a bit of https://juju.is/ https://juju.is/. Haven't looked at it much, but it seems super powerfull and tries to solve at least some of the issues you mentioned.

jeremylevy 6y ago
Thanks! I didn't know Juju, I'll try it out!