8 ms·

Hi there, I'm from AWS and my team are responsible for development of this project (AWS Perspective). It's important to note that Perspective is not a traditi

Hi there, I'm from AWS and my team are responsible for development of this project (AWS Perspective). It's important to note that Perspective is not a traditional diagramming tool like draw.io or Visio. Perspective is intended to display resources that already exist rather than draw new ones. To do this, Perspective inventories resources in your accounts (Lambdas, DBs, EC2 instances, ENIs, EBS volumes etc.) and can automatically create a diagrammatic representations of that data. Perspective also discovers the relationships between resources and will automatically expose them on the diagram. The underlying data sits in a graph database (Amazon Neptune). This allows you to explore the relationships in the graph (diagram) on the fly visually to discover how your resources interact with one another. We also have the early stages of a feature to show the estimated cost of the resources in the diagrams you create. The project is open source and our roadmap is public. I encourage you to check it out and add any commentary for features you'd like to see (or bugs you find!). https://github.com/awslabs/aws-perspective https://github.com/awslabs/aws-perspective I'm also happy to answer any questions.

deleted 6y ago
[deleted]
viraptor 6y ago
I may be missing some complexity that's not clear from the description, but since all the resources should be available in AWS Config already, why is it using so much extra infrastructure? Why isn't it a "point me at AWS Config dump, I'll give you an explorable static page/SVG" script instead? (The cost part would have to reach out to billing and/or cloudwatch, but still...) Or expressed a bit differently: what are the infra + 2 data stores doing that's a major improvement over processing raw AWS Config + CUR data which should easily fit in laptop memory. We're talking megabytes of data for a point in time snapshot, right?
- toksvaeth 6y ago
  Hi, That's a good question. Whilst Perspective sources a lot of data from AWS Config, it also supplements and enriches the data from other sources where resources are not supported or where we wanted to show more detail. We'll be adding in other data sources in future, too. Additionally, the AWS Config query language is quite helpful but doesn't expose a graph of the resources. Our approach is to pull the individual relationship data out and represent those in a graph DB (Amazon Neptune). Backing on to a graph database gives us the ability to let users navigate through the relationships of interest on the fly, rather than dumping what would inevitably be a complicated mess on the screen if we just showed everything. Using this approach is also one of the ways we've tried to solve the UI/UX challenges presented in this space. Rather than say "show me all the things" that would lead to a complex mess of interconnections (and probably explode browsers) we instead encourage a workflow that starts with a resource or workload and explores outwards from there. It also allows us to implement some interesting features in future -- we intend to keep working on the project. The ElasticSearch cluster is used by the discovery process to locate resources based on the metadata we collect; it's part of the way we enrich the data and relationships beyond what we get from Config.
  - viraptor 6y ago
    As with other comment, I should've been more clear. SVG / generated page doesn't mean no filtering. Just a point in time snapshot which doesn't require API calls once generated.
- tidepod12 6y ago
  Speculation on my part, but this seems to be much more geared towards data analysis of your environment rather than just having an SVG for reference. I'm assuming like most things AWS, this is not targeted at your small-sized startup running 2 VPCs, a handful of subnets, and 10-20 EC2 instances. This is likely built specifically for major enterprises running a hundred thousand instances, with hundreds of VPCs, thousands of subnets, hundreds of thousands of security groups, etc. In such environments, a static SVG isn't feasible and you might want something that allows more analytics instead, which I think is the entire point of using Neptune.
  - viraptor 6y ago
    I should've been more clear, but don't confused a static page and SVG with static content. I was thinking closer to a one page interactive app generated from a data snapshot.
viraptor 6y ago
Is there really no screenshot of the visualisation, or is it just hard to find / hidden on mobile? I assume the architecture diagram is made in another tool.
- toksvaeth 6y ago
  We just updated the GitHub README with some screenshots so you can see what it looks like. No, the diagrams of Perspective were not created with Perspective -- we had to stick to existing style guidelines.
jcims 6y ago
Do you have a list of supported services? I've built a very similar solution using AWS Config to feed Neo4j and it has been incredibly useful, but Config's coverage is so anemic that we are left with just a tiny portion of our infrastructure visible.
- toksvaeth 6y ago
  We sure do: https://docs.aws.amazon.com/solutions/latest/aws-perspective/appendix-c-supported-resources.html https://docs.aws.amazon.com/solutions/latest/aws-perspective... If there are more you'd like to see added, please raise an issue: https://github.com/awslabs/aws-perspective https://github.com/awslabs/aws-perspective
  - jcims 6y ago
    I think what i would really like to see (which may exist already) is a plugin architecture that allows users to easily extend with services they need. An eventing system that lets you hook node/edge events world be awesome as well. Will take a look tonight, really interested in this.

deleted 6y ago

[deleted]

viraptor 6y ago

I may be missing some complexity that's not clear from the description, but since all the resources should be available in AWS Config already, why is it using so much extra infrastructure? Why isn't it a "point me at AWS Config dump, I'll give you an explorable static page/SVG" script instead? (The cost part would have to reach out to billing and/or cloudwatch, but still...) Or expressed a bit differently: what are the infra + 2 data stores doing that's a major improvement over processing raw AWS Config + CUR data which should easily fit in laptop memory. We're talking megabytes of data for a point in time snapshot, right?

toksvaeth 6y ago
Hi, That's a good question. Whilst Perspective sources a lot of data from AWS Config, it also supplements and enriches the data from other sources where resources are not supported or where we wanted to show more detail. We'll be adding in other data sources in future, too. Additionally, the AWS Config query language is quite helpful but doesn't expose a graph of the resources. Our approach is to pull the individual relationship data out and represent those in a graph DB (Amazon Neptune). Backing on to a graph database gives us the ability to let users navigate through the relationships of interest on the fly, rather than dumping what would inevitably be a complicated mess on the screen if we just showed everything. Using this approach is also one of the ways we've tried to solve the UI/UX challenges presented in this space. Rather than say "show me all the things" that would lead to a complex mess of interconnections (and probably explode browsers) we instead encourage a workflow that starts with a resource or workload and explores outwards from there. It also allows us to implement some interesting features in future -- we intend to keep working on the project. The ElasticSearch cluster is used by the discovery process to locate resources based on the metadata we collect; it's part of the way we enrich the data and relationships beyond what we get from Config.
- viraptor 6y ago
  As with other comment, I should've been more clear. SVG / generated page doesn't mean no filtering. Just a point in time snapshot which doesn't require API calls once generated.
tidepod12 6y ago
Speculation on my part, but this seems to be much more geared towards data analysis of your environment rather than just having an SVG for reference. I'm assuming like most things AWS, this is not targeted at your small-sized startup running 2 VPCs, a handful of subnets, and 10-20 EC2 instances. This is likely built specifically for major enterprises running a hundred thousand instances, with hundreds of VPCs, thousands of subnets, hundreds of thousands of security groups, etc. In such environments, a static SVG isn't feasible and you might want something that allows more analytics instead, which I think is the entire point of using Neptune.
- viraptor 6y ago
  I should've been more clear, but don't confused a static page and SVG with static content. I was thinking closer to a one page interactive app generated from a data snapshot.

viraptor 6y ago

Is there really no screenshot of the visualisation, or is it just hard to find / hidden on mobile? I assume the architecture diagram is made in another tool.

toksvaeth 6y ago
We just updated the GitHub README with some screenshots so you can see what it looks like. No, the diagrams of Perspective were not created with Perspective -- we had to stick to existing style guidelines.

jcims 6y ago

Do you have a list of supported services? I've built a very similar solution using AWS Config to feed Neo4j and it has been incredibly useful, but Config's coverage is so anemic that we are left with just a tiny portion of our infrastructure visible.

toksvaeth 6y ago
We sure do: https://docs.aws.amazon.com/solutions/latest/aws-perspective/appendix-c-supported-resources.html https://docs.aws.amazon.com/solutions/latest/aws-perspective... If there are more you'd like to see added, please raise an issue: https://github.com/awslabs/aws-perspective https://github.com/awslabs/aws-perspective
- jcims 6y ago
  I think what i would really like to see (which may exist already) is a plugin architecture that allows users to easily extend with services they need. An eventing system that lets you hook node/edge events world be awesome as well. Will take a look tonight, really interested in this.