4 ms·

Literally working on an extremely similar project, albeit to power security; seems inevitable that an open-source solution would've popped up. Glad to see this.

Literally working on an extremely similar project, albeit to power security; seems inevitable that an open-source solution would've popped up. Glad to see this. Any exploration in doing this cross-cloud?

chris408 8y ago
Not sure, but for now AWS is the focus.
- alpha_squared 8y ago
  Not sure if I'm missing it in the documentation, but how does it keep in sync with changes in the account? Edit: OP changed response to remove certain references, presumably because something sensitive was revealed. Changing my response out of respect.
  - alexchantavy 8y ago
    Hi, I'm one of the devs on the project. Good question. As is, this does not keep anything in sync. To keep the graph in sync with changes in the account, simply set up a cronjob to run `cartography` whenever you would need a refresh. Each sync run should guarantee that you have the most up-to-date data. Here's how a sync works: when the sync starts, set a variable called `update_tag` to the current time. Then, pull all the data from your AWS account(s) and create Neo4j nodes and their relationships, making sure to set their `lastupdated` fields to `update_tag`. Finally, delete the left over nodes and relationships (i.e. those that do not have up-to-date `lastupdated` fields). This way the data stays fresh, and you can see this in the [cleanup jobs](https://github.com/lyft/cartography/tree/master/cartography/data/jobs/cleanup https://github.com/lyft/cartography/tree/master/cartography/...).
    - alpha_squared 8y ago
      Makes sense, thanks for the additional details! Our approach requires us to stay as real-time as possible, so we're actually using CloudWatch events to keep in sync -- the deletes become a little hard after that. I look forward to the progress of Cartography, though!
duked 8y ago
I'm working on something related to security too. I would love to connect but no email on your profile.
- alpha_squared 8y ago
  Sure, let's connect; you can shoot me a message here: andrew(a)armaneous.com
alexchantavy 8y ago
We're definitely open for collaboration to allow multi-cloud :-). This could simply be another [intel module](https://github.com/lyft/cartography/tree/master/cartography/intel https://github.com/lyft/cartography/tree/master/cartography/...).

chris408 8y ago

Not sure, but for now AWS is the focus.

alpha_squared 8y ago
Not sure if I'm missing it in the documentation, but how does it keep in sync with changes in the account? Edit: OP changed response to remove certain references, presumably because something sensitive was revealed. Changing my response out of respect.
- alexchantavy 8y ago
  Hi, I'm one of the devs on the project. Good question. As is, this does not keep anything in sync. To keep the graph in sync with changes in the account, simply set up a cronjob to run `cartography` whenever you would need a refresh. Each sync run should guarantee that you have the most up-to-date data. Here's how a sync works: when the sync starts, set a variable called `update_tag` to the current time. Then, pull all the data from your AWS account(s) and create Neo4j nodes and their relationships, making sure to set their `lastupdated` fields to `update_tag`. Finally, delete the left over nodes and relationships (i.e. those that do not have up-to-date `lastupdated` fields). This way the data stays fresh, and you can see this in the [cleanup jobs](https://github.com/lyft/cartography/tree/master/cartography/data/jobs/cleanup https://github.com/lyft/cartography/tree/master/cartography/...).
  - alpha_squared 8y ago
    Makes sense, thanks for the additional details! Our approach requires us to stay as real-time as possible, so we're actually using CloudWatch events to keep in sync -- the deletes become a little hard after that. I look forward to the progress of Cartography, though!

duked 8y ago

I'm working on something related to security too. I would love to connect but no email on your profile.

alpha_squared 8y ago
Sure, let's connect; you can shoot me a message here: andrew(a)armaneous.com

alexchantavy 8y ago

We're definitely open for collaboration to allow multi-cloud :-). This could simply be another [intel module](https://github.com/lyft/cartography/tree/master/cartography/intel https://github.com/lyft/cartography/tree/master/cartography/...).