8 ms·

Show HN: The Segment AWS Stack

237 points by calvinfo 10y ago

avitzurel 10y ago
This is great! I've been working on an open source stack that is somewhat similar and serves the same purpose. Allowing startups to bootstrap their stack on AWS. http://docs.the-startup-stack.com http://docs.the-startup-stack.com Great work on open sourcing this. I will see what I can do to contribute.
- avitzurel 10y ago
  Github organization is here: https://github.com/the-startup-stack https://github.com/the-startup-stack I am developing everything "in the open" so feel free to contribute / ask questions. I spent the last 3 months collecting usage information from startups using the-startup-stack and about to make another effort to commit all that knowledge back into the project.
tekronis 10y ago
One thing that I would have loved some more detail on is how are secrets and credentials being handled?
- conorgil145 10y ago
  Getting secrets and credentials correct seem to be the crux of most architectures. I'd also love to hear more from the Segment team on their approach with this setup!
moondev 10y ago
This is amazing. Kudos to the Segment team for putting in the effort to share and open source this. And also kudos for leveraging Terraform rather than CloudFormation. I may fork this to get it running with kubernetes on GCP
diegorbaquero 10y ago
Thank you for sharing, simple and detailed. However I'd like to comment that AWS is not cheap.
- misframer 10y ago
  What would you consider to be cheaper than AWS?
  - mark-ruwt 10y ago
    We were exclusively on AWS, but now are distributed across Linode, DigitalOcean, and Vultr. We're saving a lot, and getting better performance.
    - beat 10y ago
      Can you share a little what your stack looks like? And how are you doing infrastructure automation to build and reproduce it? I'm really interested in where you found ways to cut costs, and what kind of effort was involved.
      - mark-ruwt 10y ago
        Sure, think of it this way: If you had a perfectly clean install of your distro of choice, could you write a shell script that could build your server from scratch? If you can answer Yes to that (and you should), then you can build an if/then-heavy shell script that works with each of the APIs to create a perfectly clean install of your favorite distro. One script to create the clean slate machine. One script to build what you need. We have 9 "pods" around the globe, each with API servers (Java/Tomcat/Apache), static servers (Varnish), a MySQL slave, and an HAProxy Maître D'. With close to 60 servers, our monthly bills are less than $1,000, and we haven't had downtime in years. Spinning up a new server is just: sh build.sh atl api 1. Feel free to ping me if you want any more details: mark@areyouwatchingthis.com.
        mrsirduke 10y ago
beat 10y ago
This is so timely for me! I was struggling to build a repeatable stack with CloudFormation, started down the road of Terraform just last night. This will help me skip a lot of learning curve. Thank you!
fireworks10 10y ago
What tool did you use to make the flow chart, or is it custom? [0] https://segment.com/blog/the-segment-aws-stack/images/main.png https://segment.com/blog/the-segment-aws-stack/images/main.p...
- alexkappa 10y ago
  Thats the `terraform graph` command. [0] https://www.terraform.io/docs/commands/graph.html https://www.terraform.io/docs/commands/graph.html
  - avitzurel 10y ago
    `terraform graph` was used in the post but it's not the one he linked to. This seems custom to me, maybe Sketch or something.
    - fizx 10y ago
      Looks like http://www.graphviz.org/ http://www.graphviz.org/
ivan_ah 10y ago
Very nice. I didn't know about terraform, and seems like a very powerful combo... Can anyone comment as to the differences in functionality between ansible AWS playbooks and terraform? The Segment Stack is a good showcase of best-practices for AWS ... only one thing I didn't understand, why a NAT instance is needed in each subnet? https://github.com/segmentio/stack/blob/master/vpc/main.tf#L54 https://github.com/segmentio/stack/blob/master/vpc/main.tf#L... It seems a little wasteful. Couldn't you just allow traffic between the subnets?
- mikelinington 10y ago
  The biggest thing I've noticed is that terraform doesn't, out of the box, have any way of bringing in your existing resources, and sharing the [tfstate file] that has the current state of your world is its own challenge. The ec2 module of ansible has `count_tag`, and that was exactly what I wanted: "I want this many boxes that are identified by these tags." Played with an external tool call [terraforming] that creates a terraform setup from your existing boxes, but that, too, was way more than I wanted. Terraform is a very neat tool, but learning the AWS ecosystem _and_ this third-party tool that has its own very strong set of best practices was too much for me. If you're experienced, and especially if you're already using CloudFormation heavily, you can probably get a lot of mileage out of terraform. Doing a quick search to make sure I'm not crazy, it looks like terraform 0.7 will start to support importing, so that's something: https://github.com/hashicorp/terraform/issues/581 https://github.com/hashicorp/terraform/issues/581 [tfstate file]: https://www.terraform.io/docs/state/remote https://www.terraform.io/docs/state/remote [terraforming]: https://github.com/dtan4/terraforming https://github.com/dtan4/terraforming
  - jen20 10y ago
    I work on Terraform at HashiCorp - Terraform import is on the way, starting with 0.7. We have a multi-phase approach - first importing state with 0.7 and then moving on to generating configuration in subsequent releases. Many AWS resources are already supported for import, and we're intending to get full support over time. If there are specific resource types that you need that aren't currently supported, please open issues on the repository and we will endeavour to support them quickly!
joshpadnick 10y ago
So, my co-founder and I have basically been independently building a commercially supported alternative to this excellent open source package. But there are a few differences that make sense when offering this as a paid service: - We wrote our own terraform testing framework to validate that every change to our modules doesn't break functionality - We actively update our modules based on feedback from new client engagements - We provide commercial support for each module - We combine our modules with consulting and training as needed And of course, there are many similarities - We give 100% of the source code to our clients - Everything runs in the client's AWS account - Everything is self-documented, modularized, and can be combined/composed as the needs of different teams require I didn't mean for this to be a shameless plug; more just that I found it interesting to compare the open source vs. commercial approach to solving this same problem. Props to the Segment team for sharing this.
- beat 10y ago
  There's a lot of potential for such a service as a consultancy - advice and in-house customization included along with the software. As cool as the Segment stack is, it doesn't let people off the hook for designing effectively in the first place, or for taking a poor design and re-engineering it.
  - avitzurel 10y ago
    My initial approach with the-startup-stack[1] was exactly that. Building an open source solution with a "pro" level all setup included in it. I had a very hard time quantifying how much companies will pay for this and whether they even will. How I see it, you are either on Heroku (or other similar) and you don't care about anything except `git push` or you have a full blown stack. I know the middle ground between the two is where the stack is but I just couldn't figure out how many companies are actually experiencing those difficulties and how much are they willing to pay for the help. Since it's not a startup for me, it's just an open source project I decided not to worry about it, but would still be nice to get some input. [1] http://the-startup-stack.com http://the-startup-stack.com
    - 10y ago
deleted 10y ago
[deleted]
blahi 10y ago
I so much wish segment would offer their services on the Azure stack.
garysieling 10y ago
If you were to set this up and leave it running (say for a dev environment), what would your monthly bill look like?

237 points by calvinfo 10y ago

This is great! I've been working on an open source stack that is somewhat similar and serves the same purpose. Allowing startups to bootstrap their stack on AWS. http://docs.the-startup-stack.com http://docs.the-startup-stack.com Great work on open sourcing this. I will see what I can do to contribute.

avitzurel 10y ago
Github organization is here: https://github.com/the-startup-stack https://github.com/the-startup-stack I am developing everything "in the open" so feel free to contribute / ask questions. I spent the last 3 months collecting usage information from startups using the-startup-stack and about to make another effort to commit all that knowledge back into the project.

tekronis 10y ago

One thing that I would have loved some more detail on is how are secrets and credentials being handled?

conorgil145 10y ago
Getting secrets and credentials correct seem to be the crux of most architectures. I'd also love to hear more from the Segment team on their approach with this setup!

moondev 10y ago

This is amazing. Kudos to the Segment team for putting in the effort to share and open source this. And also kudos for leveraging Terraform rather than CloudFormation. I may fork this to get it running with kubernetes on GCP

diegorbaquero 10y ago

Thank you for sharing, simple and detailed. However I'd like to comment that AWS is not cheap.

misframer 10y ago
What would you consider to be cheaper than AWS?
- mark-ruwt 10y ago
  We were exclusively on AWS, but now are distributed across Linode, DigitalOcean, and Vultr. We're saving a lot, and getting better performance.
  - beat 10y ago
    Can you share a little what your stack looks like? And how are you doing infrastructure automation to build and reproduce it? I'm really interested in where you found ways to cut costs, and what kind of effort was involved.
    - mark-ruwt 10y ago
      Sure, think of it this way: If you had a perfectly clean install of your distro of choice, could you write a shell script that could build your server from scratch? If you can answer Yes to that (and you should), then you can build an if/then-heavy shell script that works with each of the APIs to create a perfectly clean install of your favorite distro. One script to create the clean slate machine. One script to build what you need. We have 9 "pods" around the globe, each with API servers (Java/Tomcat/Apache), static servers (Varnish), a MySQL slave, and an HAProxy Maître D'. With close to 60 servers, our monthly bills are less than $1,000, and we haven't had downtime in years. Spinning up a new server is just: sh build.sh atl api 1. Feel free to ping me if you want any more details: mark@areyouwatchingthis.com.
      - mrsirduke 10y ago

beat 10y ago

This is so timely for me! I was struggling to build a repeatable stack with CloudFormation, started down the road of Terraform just last night. This will help me skip a lot of learning curve. Thank you!

fireworks10 10y ago

What tool did you use to make the flow chart, or is it custom? [0] https://segment.com/blog/the-segment-aws-stack/images/main.png https://segment.com/blog/the-segment-aws-stack/images/main.p...

alexkappa 10y ago
Thats the `terraform graph` command. [0] https://www.terraform.io/docs/commands/graph.html https://www.terraform.io/docs/commands/graph.html
- avitzurel 10y ago
  `terraform graph` was used in the post but it's not the one he linked to. This seems custom to me, maybe Sketch or something.
  - fizx 10y ago
    Looks like http://www.graphviz.org/ http://www.graphviz.org/

ivan_ah 10y ago

Very nice. I didn't know about terraform, and seems like a very powerful combo... Can anyone comment as to the differences in functionality between ansible AWS playbooks and terraform? The Segment Stack is a good showcase of best-practices for AWS ... only one thing I didn't understand, why a NAT instance is needed in each subnet? https://github.com/segmentio/stack/blob/master/vpc/main.tf#L54 https://github.com/segmentio/stack/blob/master/vpc/main.tf#L... It seems a little wasteful. Couldn't you just allow traffic between the subnets?

mikelinington 10y ago
The biggest thing I've noticed is that terraform doesn't, out of the box, have any way of bringing in your existing resources, and sharing the [tfstate file] that has the current state of your world is its own challenge. The ec2 module of ansible has `count_tag`, and that was exactly what I wanted: "I want this many boxes that are identified by these tags." Played with an external tool call [terraforming] that creates a terraform setup from your existing boxes, but that, too, was way more than I wanted. Terraform is a very neat tool, but learning the AWS ecosystem _and_ this third-party tool that has its own very strong set of best practices was too much for me. If you're experienced, and especially if you're already using CloudFormation heavily, you can probably get a lot of mileage out of terraform. Doing a quick search to make sure I'm not crazy, it looks like terraform 0.7 will start to support importing, so that's something: https://github.com/hashicorp/terraform/issues/581 https://github.com/hashicorp/terraform/issues/581 [tfstate file]: https://www.terraform.io/docs/state/remote https://www.terraform.io/docs/state/remote [terraforming]: https://github.com/dtan4/terraforming https://github.com/dtan4/terraforming
- jen20 10y ago
  I work on Terraform at HashiCorp - Terraform import is on the way, starting with 0.7. We have a multi-phase approach - first importing state with 0.7 and then moving on to generating configuration in subsequent releases. Many AWS resources are already supported for import, and we're intending to get full support over time. If there are specific resource types that you need that aren't currently supported, please open issues on the repository and we will endeavour to support them quickly!

joshpadnick 10y ago

So, my co-founder and I have basically been independently building a commercially supported alternative to this excellent open source package. But there are a few differences that make sense when offering this as a paid service: - We wrote our own terraform testing framework to validate that every change to our modules doesn't break functionality - We actively update our modules based on feedback from new client engagements - We provide commercial support for each module - We combine our modules with consulting and training as needed And of course, there are many similarities - We give 100% of the source code to our clients - Everything runs in the client's AWS account - Everything is self-documented, modularized, and can be combined/composed as the needs of different teams require I didn't mean for this to be a shameless plug; more just that I found it interesting to compare the open source vs. commercial approach to solving this same problem. Props to the Segment team for sharing this.

beat 10y ago
There's a lot of potential for such a service as a consultancy - advice and in-house customization included along with the software. As cool as the Segment stack is, it doesn't let people off the hook for designing effectively in the first place, or for taking a poor design and re-engineering it.
- avitzurel 10y ago
  My initial approach with the-startup-stack[1] was exactly that. Building an open source solution with a "pro" level all setup included in it. I had a very hard time quantifying how much companies will pay for this and whether they even will. How I see it, you are either on Heroku (or other similar) and you don't care about anything except `git push` or you have a full blown stack. I know the middle ground between the two is where the stack is but I just couldn't figure out how many companies are actually experiencing those difficulties and how much are they willing to pay for the help. Since it's not a startup for me, it's just an open source project I decided not to worry about it, but would still be nice to get some input. [1] http://the-startup-stack.com http://the-startup-stack.com
  - 10y ago

deleted 10y ago

[deleted]

blahi 10y ago

I so much wish segment would offer their services on the Azure stack.

garysieling 10y ago

If you were to set this up and leave it running (say for a dev environment), what would your monthly bill look like?