7 ms·
Visualize AWS Cloud Diagrams
- techdragon 6y agoI love diagrams and pay for CloudCraft so I’m no stranger to the value of good diagram tools. But I looked at that cloud formation design and immediately thought “crap that’s probably going to cost way too much to run”. Anyone got a good up to date cloud formation to dollars per month tool and want to estimate it? (I’m on a mobile at the moment)
- codefined 6y agoThe only service that appears to cost a flat fee is 'Amazon Neptune', which on the cheapest plan costs ~10 cents an hour or ~$70/month. I imagine all the other costs (e.g. S3, Fargate, API Gateway, CloudFront) are negligible in comparison.
- mithunshanbhag 6y agoIt's $500+ per month. https://twitter.com/der_rehan/status/1308242717307174912 https://twitter.com/der_rehan/status/1308242717307174912
- scarface74 6y agoThe ElasticSearch cluster is a fixed cost also.
- weeeeelp 6y agoElasticsearch Service (managed clusters) has a variable price depending on node type/count and storage size. Also, you pay for network traffic.
- scarface74 6y agoIt’s “fixed” once you provision it. The other services like Lambda, APIGW, CodeBuild etc. vary according to usage. ES network traffic wouldn’t cost if you are only retrieving traffic in the same region.
- toksvaeth 6y agoHi, You can find an estimated cost breakdown here: https://docs.aws.amazon.com/solutions/latest/aws-perspective/overview.html#cost https://docs.aws.amazon.com/solutions/latest/aws-perspective... (Disclosure: My team developed this project)
- hlandau 6y agoIs this a joke? I feel like this is a parody of AWS, and the notion of using as many AWS services as possible to make something otherwise simple.
- eandre 6y agoUnfortunately not – this is what building modern cloud based software is like! This software complexity crisis is what we're trying to fix with Encore [1], if you're interested. [1] https://encore.dev https://encore.dev
- amadeuspagel 6y agoYour demo[1] returns a 502 error. [1]: https://play.encore.dev/ https://play.encore.dev/
- eandre 6y agoThanks, it's back up now.
- Hnrobert42 6y agoNot to mention this is something for which AWS already has all the data. They should just be providing this diagram as a free part of the AWS experience.
- degenerate 6y agoI've always found it annoying that they have all the data readily available to bill you for, but not to SHOW you, your overall utilization.
- stemuk 6y agoI have been creating most of my infrastructure diagrams in https://arcentry.com/ https://arcentry.com/ and this seems to me like an overcomplicated version of Arcentry.
- esseti 6y agoany good alternative? the service looks neat, but it has been discontinued
- itprofessional4 6y agodraw.io is a good alternative . By default it does not show icon for cloud services but if you click on "More Shapes" option at bottom left, you can add AWS,Azure and GCP and create all sorts for architecture diagrams. Plus, it is opensource and can run your own self hosted draw.io app from github.
- poooogles 6y agocloudcraft.io is OK
- jonbronson 6y agoLucidchart Cloud Insights is another player in this space. Full disclosure, I'm part of the team that created it: https://www.lucidchart.com/blog/lucidchart-cloud-insights https://www.lucidchart.com/blog/lucidchart-cloud-insights
- hbcondo714 6y agoWe are long time users of Lucidchart and were using the predecessor of Cloud Insights, AWS Import, for creating cloud diagrams of our existing AWS infrastructure. In February, a LucidChart account manager contacted us stating the AWS Import feature would be sunsetted. He offered to onboard our team onto Cloud Insights but we would have to upgrade to an Enterprise account at $199/license/year AND pay $4,000 a year for the Cloud Insights add-on that covers only up to 50 compute resources. We ended up keeping our account as-is and never heard back from the account manager. Lucidchart is a great product but we were really turned-off & disappointed they essentially took a feature away from us.
- esseti 6y agodoes it build a chart of the infrastrucuture? any infrastructure in AWS?
- dkroy 6y agoI really would welcome this on my current stack as someone who now primarily works in GCP and is constantly updating architecture diagrams. I love any tool that will help create automated assets to assist with documentation as they really help prevent tribal knowledge and reduce that constant overhead of keeping everything up to date.
- toppy 6y agoThis projekt circulated on HN few days ago: https://github.com/mingrammer/diagrams https://github.com/mingrammer/diagrams
- judheg 6y agoOOT, anyone knows tools from terraform description to these type of visualization?
- milquetoastaf 6y agoYou can use Terraform graph https://www.terraform.io/docs/commands/graph.html https://www.terraform.io/docs/commands/graph.html
- bdcravens 6y agoA named AWS service to visualize which named AWS services you are using is peak AWS.
- tidepod12 6y agoThis isn't a "named AWS service" in the same vein of typical services like S3, EC2, etc. This is a "AWS Solution Implementation" which is a list of community created "solutions" built using AWS. My strong suspicion is that this "AWS Perspective" is something that one of AWS's customers insisted on being built customized for their environment, and then the Solution Architects who designed it for the customer also wanted to get some "external credit" (AWS employees have to create a certain number of externally-facing blog posts/GitHub contributions/etc) so they uploaded it here.
- deleted 6y ago[deleted]
- pecosbill 6y ago"Simplify" seems to be missing from your Invent and Simplify (https://www.amazon.jobs/en/principles https://www.amazon.jobs/en/principles)
- capableweb 6y agoAlso "Frugality" and "Insist on the Highest Standards" seems to be missing from AWS Perspective (no pun intended, it's literally the name of the service)
- toksvaeth 6y agoHi there, I'm from AWS and my team are responsible for development of this project (AWS Perspective). It's important to note that Perspective is not a traditional diagramming tool like draw.io or Visio. Perspective is intended to display resources that already exist rather than draw new ones. To do this, Perspective inventories resources in your accounts (Lambdas, DBs, EC2 instances, ENIs, EBS volumes etc.) and can automatically create a diagrammatic representations of that data. Perspective also discovers the relationships between resources and will automatically expose them on the diagram. The underlying data sits in a graph database (Amazon Neptune). This allows you to explore the relationships in the graph (diagram) on the fly visually to discover how your resources interact with one another. We also have the early stages of a feature to show the estimated cost of the resources in the diagrams you create. The project is open source and our roadmap is public. I encourage you to check it out and add any commentary for features you'd like to see (or bugs you find!). https://github.com/awslabs/aws-perspective https://github.com/awslabs/aws-perspective I'm also happy to answer any questions.
- deleted 6y ago[deleted]
- viraptor 6y agoI may be missing some complexity that's not clear from the description, but since all the resources should be available in AWS Config already, why is it using so much extra infrastructure? Why isn't it a "point me at AWS Config dump, I'll give you an explorable static page/SVG" script instead? (The cost part would have to reach out to billing and/or cloudwatch, but still...) Or expressed a bit differently: what are the infra + 2 data stores doing that's a major improvement over processing raw AWS Config + CUR data which should easily fit in laptop memory. We're talking megabytes of data for a point in time snapshot, right?
- toksvaeth 6y agoHi, That's a good question. Whilst Perspective sources a lot of data from AWS Config, it also supplements and enriches the data from other sources where resources are not supported or where we wanted to show more detail. We'll be adding in other data sources in future, too. Additionally, the AWS Config query language is quite helpful but doesn't expose a graph of the resources. Our approach is to pull the individual relationship data out and represent those in a graph DB (Amazon Neptune). Backing on to a graph database gives us the ability to let users navigate through the relationships of interest on the fly, rather than dumping what would inevitably be a complicated mess on the screen if we just showed everything. Using this approach is also one of the ways we've tried to solve the UI/UX challenges presented in this space. Rather than say "show me all the things" that would lead to a complex mess of interconnections (and probably explode browsers) we instead encourage a workflow that starts with a resource or workload and explores outwards from there. It also allows us to implement some interesting features in future -- we intend to keep working on the project. The ElasticSearch cluster is used by the discovery process to locate resources based on the metadata we collect; it's part of the way we enrich the data and relationships beyond what we get from Config.
- Galanwe 6y agoDo you guys feel comfortable uploading your infrastructure code to a third party service? I sure wouldn't.
- ctvo 6y agoThe decentralized nature of Amazon / AWS rears its head again. It's much harder, I assume, to get security / teams to allow another team to crawl through its customer data (the Lambda team owns which customers have how many Lambdas running where, for example). Now imagine doing that for every AWS resource to form a graph of relationships. To get around that they deployed something that customers are wholly responsible for. Forcing customers to cover the bill for something AWS should provide out of the box, in this subpar experience where they also need to manage all the infrastructure, is the opposite of customer obsession.
- tidepod12 6y ago>To get around that they deployed something that customers are wholly responsible for. I doubt they were "getting around" anything. The customer being responsible for it is likely the entire point, because the customer who asked for this to be created probably specifically requested that (based on my experience working with AWS SAs). This post is an "AWS Solutions Implementation". This isn't a service like S3/EC2/Lambda/etc created to be an integral part of AWS. A "Solutions Implementation" is basically the output of a customer making a very specific request from AWS Solutions Architects to build something customized (eg, customer doesn't want to use AWS Config but they want to build something from scratch with Lambda and RDS that works the same way), and then the solutions architects decide to share it publicly just in case any other customers find the same niche solution useful. It isn't meant for wide appeal or to be used by everyone. I wouldn't be surprised if the fact that this post is getting decent attention means that AWS will use this solution as a proof of concept for investigating a future managed AWS service where the customer doesn't have to fully manage all of these moving pieces, but this post isn't that.
- ctvo 6y ago> I doubt they were "getting around" anything. The customer being responsible for it is likely the entire point, because the customer who asked for this to be created probably specifically requested that (based on my experience working with AWS SAs). I doubt a customer asked AWS: hey, can I get a graph of how the resources in my accounts all interconnect while maintaining the clusters for the data stores, handling upgrade, security, etc. myself and paying hundreds of dollars a month for the pleasure? The alternative is a customer asked for a way to see how resources were connected and how much it was costing them, and AWS, unable to deliver because of N hurdles (I mentioned security, but no doubt others), delivered this instead. Again: this should be a managed service and free.
- jeffcarter 6y agoShameless plug: I wrote AWSets[1], which can be used to do some of this (although the graphs aren't as pretty). It was written because at work we regularly interact with hundreds of AWS accounts and wanted something to quickly scan/catalog all of the resources (launch blog post explains more [2]). It's free to run locally (doesn't use or require Config), the data is normalized & output as JSON for you to do whatever you want with it. It has better coverage than similar tools (>250 resources & growing), and provides relationship mapping between them as well. It's still a work in progress and might have rough edges, so feel free to file issues to help make it better. [1] - https://github.com/trek10inc/awsets https://github.com/trek10inc/awsets [2] - https://www.trek10.com/blog/awsets-aws-resource-listing-made-easy https://www.trek10.com/blog/awsets-aws-resource-listing-made...
- chair6 6y agoFor a different angle on cloud infrastructure visualization, check out Abridge (https://abridge.io https://abridge.io). I'd love to get any feedback on the idea / site and even a trial, if you're interested! Abridge collects data across your configured set of AWS accounts & regions. It doesn't draw architecture diagrams, but instead will give you x-vs-y visualizations for a number of different relationships .. IAM users vs. groups, Lambda functions vs. runtimes, EC2 instances vs. keypair, etc. It'll also provide inventory tables / CSV dumps of the different resource types, and simple free search across all resources. Security-wise, Abridge collects data via cross-account trusts with the SecurityAudit role, and expires all collected data after 48 hours - more at https://abridge.io/security/ https://abridge.io/security/.
- drchopchop 6y agoWait, this graphing tool literally requires you to deploy Neptune, Elasticsearch, ECS, various VPC's/gateways, publish various reports to S3, run Lambda functions, and then after all that the minimum cost is $500+/month? Luckily it appears to be a "solution" instead of a core AWS product, otherwise I was about to say AWS has turned into a parody of itself.
- sciurus 6y agoI think I count 18 different AWS services in the diagram at https://d1.awsstatic.com/Solutions/Solutions%20Category%20Template%20Draft/Solution%20Architecture%20Diagrams/aws-perspective-architecture-diagram.f67026185e4f02d2d623289fa714d69d2c1f7af6.png https://d1.awsstatic.com/Solutions/Solutions%20Category%20Te...
- staticassertion 6y agoThis feels like a strange and expensive way to solve this problem. 1. Why do you need a graph database? You could almost certainly keep the entire thing in memory, who has >16GB of infrastructure? 2. 500+ dollars a month for a service that I'll use very infrequently? 3. I feel like all I actually want is something that takes CloudFormation and gives me a picture, for free, in a python script on my laptop. But ok, fine, what about things not in CF, and other metadata sure, but then see (1) and (2). If this were, say, one dollar a month (or serverless) and the infrastructure was totally abstracted away I'd be a lot more excited. edit: OK so (2) isn't really their fault, since this isn't an AWS service, but still. And 600 bucks isn't that bad for many companies, I just think there's a bit of a pricetag shock on this one.
- jcims 6y agoThe graph database is useful for external analysis of the relationships between your resources. I've built this in the past using just AWS Config and, while incredibly useful, it is extremely limited due to Config's very limited scope. If this covers a reasonable percentage of AWS services it would be more than cost justified at $1-2k/mo for our environment.
- staticassertion 6y agoYeah, thinking about it more I do feel that this is a justifiable cost.
- myroon5 6y ago> I feel like all I actually want is something that takes CloudFormation and gives me a picture, for free, in a python script on my laptop you might find this interesting then: https://github.com/aws-cloudformation/cfn-python-lint/pull/1411 https://github.com/aws-cloudformation/cfn-python-lint/pull/1... https://github.com/aws-cloudformation/aws-cfn-lint-visual-studio-code/pull/106 https://github.com/aws-cloudformation/aws-cfn-lint-visual-st...
- staticassertion 6y ago
- shmoogy 6y agoThis seems crazy for someone in a smaller account. I just want something that will chart out all my current resources allocated in a VPC/account and send to me once a week or so - so I can spot check if anything got accidentally left on. I kind of feel like you would need $20k+/month spend for it to be worth $500 a month to do this.