24 ms·

Waypoint: Build, deploy, and release applications across any platform

549 points by jacobwg 6y ago

Veraticus 6y ago
I always love HashiCorp products! But... This feels like Otto v2 to me, and it seems like it hasn't actually solved the underlying problem with Otto: that it was simpler to just use and learn the underlying tools instead of a very specific DSL that transformed into them. If I have to look up how to use Waypoint to create Dockerfiles/Kubernetes manifests anyway, why not just learn how to use Dockerfiles and Kubernetes manifests? I'm pretty excited by Boundary, but I don't really see the point of this.
- mitchellh 6y ago
  Hello! Compared to Otto there are some major simplifications we made: * Waypoint does not own/manage any infrastructure, Otto attempted to manage the infrastructure. With Waypoint, you have to bring this yourself. * Waypoint has pluggable components for build/deploy/release. Otto focused in on our tools (mainly because we had to manage the infra, point 1). This makes it much easier to slide into existing ecosystems. On your question of abstractions: this same argument can be made for Terraform so I think my answer there would be the same but for applications. The idea is that Waypoint can give you a consistent workflow to use different tools, and additional value add features such as deployment URLs, exec, logs, etc. that work across platforms.
  - Veraticus 6y ago
    Thanks for the response Mitchell, and I love your work! You make an excellent point re: abstractions and consistent workflow. But for Terraform, you still need to know the underlying provider; to use Terraform in AWS I have to know what AWS resources I'm looking for, Azure resources in Azure, and so on. But Terraform adds value through abstraction. I don't need to learn the specific AWS/Azure calls; Terraform does it for me. Terraform provides a sane, consistent syntax. And it encourages a declarative workflow that the cloud providers themselves don't do very well/at all. I don't necessarily see Waypoint providing that same value. You need to know the underlying provider to know what you want to do with it, but the abstraction seems to make it more difficult to use that provider, not easier. But I am a devops professional, not an application developer, so I might just be the wrong market for it. Either way, congratulations on the release, and I'm excited to see where Waypoint goes from here.
- throwaway894345 6y ago
  > why not just learn how to use Dockerfiles and Kubernetes manifests? The problem I see with the entire Docker ecosystem is that the Docker/Dockerfile build system is fundamentally terrible, principally because the cache system assumes a linear dependency tree and build graphs are, well, graphs. To get reasonable build performance, you need a tool that understands this. This is something Nix gets right, and it even lets you build Docker images without the Docker/Dockerfile build system. I'd also like to see something that takes it a step further--build the docker image AND a set of kubernetes manifests that reference that docker image (a "kubernetes app package" if you will) such that you have a single artifact that represents your application that a "kubernetes package manager" can deploy (the current crop of k8s package managers altogether punt on the contents of docker images). I understand that this is a very different way of thinking about Kubernetes and Docker builds than what we practice today, but I'd really like to see this in practice or hear some debate about its merits (or lackthereof).
  - jacques_chester 6y ago
    Broadly, I agree, and I've written a lot about it in the past[0]. OCI images are not quite as bad as they used to be. Linear caching is no longer baked directly into the image format, instead it's an assumption that has been carried forward by Dockerfiles and docker build. Docker folks are working on at least the docker build part in buildkit[1]. In the meantime though I prefer Cloud Native Buildpacks, which are able to perform layer rebasing as an update operation. Disclosure: I have previously worked on buildpacks technology for Pivotal, now VMware. [0] https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POUaadcvK3ZNT9QEGDZhPk/edit https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU... [1] https://github.com/moby/buildkit https://github.com/moby/buildkit
  - nur0n 6y ago
    I believe BuildKit solves the dependency graph problem. Docker has shipped with it since 18.09. It is opt-in for now so you have to use e.g. `DOCKER_BUILDKIT=1 docker build ...`. https://github.com/moby/buildkit https://github.com/moby/buildkit
    - 6y ago
jgilias 6y ago
This feels a bit like that XKCD strip about fourteen competing standards. But I'd love to be proven wrong!
- CyanLite2 6y ago
  It's exactly that! https://xkcd.com/927/ https://xkcd.com/927/
- no_wizard 6y ago
  May be worth mentioning that Hashicorp has a lot more sway and influence in this arena than most who might have or are attempting or maybe thought to attempt this in the past. There may be something to be said for that. I know Google and AWS are pretty all in on kubernetes, but this does offer an alternative & a layer of abstraction on top of that. My bigger worry is all these deployment abstractions might lead to a scenario where when something breaks very badly, nobody really knows how to fix it other than to nuke the whole thing and try again. I feel like we're already there with that in a lot of ways with these devops systems. The weird kind of lock in they have to me is that they aren't enabling smaller teams to easily adopt these things, its aimed at either cloud providers or large insitutions that can afford to pay a dedicated engineer to manage these systems. I'd love to see something aimed at smaller teams that makes these orchestrations easy and multifunctional, and work across more platforms, like DO. Hashicorp et. el. know what the best practices are and how to keep things secure, just give me something that enforces those things on any fleet of servers/containers/whathaveyous I want to use. As a developer on a smaller team who spends part of my time working on these concerns, that would help immensely.
sushicalculus 6y ago
In my experience with HashiCorp products, things don't work like the page says (Vault) while other's are sometimes half-finished (Terraform will let you create, but not destroy resources). Edit: To clarify, I ran into some cases where the creation code for a resource existed, but not the destroy code. I might wait on this and read the Github issues for a few months before trying it
- aratno 6y ago
  Terraform does let you destroy resources: https://www.terraform.io/docs/commands/destroy.html https://www.terraform.io/docs/commands/destroy.html
  - sushicalculus 6y ago
    Yes, I know. I meant for some resources (the code for creating was added, but not for destroying those resources). I had to write a custom provider to temporarily fix it
  - kawsper 6y ago
    I wish Terraform still allowed destroy-time provisioners to access variables, it seems that it went away due to some refactoring, and it isn't coming back. https://github.com/hashicorp/terraform/issues/23679 https://github.com/hashicorp/terraform/issues/23679
- outime 6y ago
  I have been using Terraform since 0.11 and it's always allowed to destroy resources, what do you mean?
jacques_chester 6y ago
Hopefully someone at Hashicorp can double check my read: I feel this could be compared to Skaffold or Tilt, but with a heavier emphasis on a plugin architecture. Is that right?
- jambay 6y ago
  Hey Jacques! Good to chat with you again :) Yes, the plugin architecture is an essential part of Waypoint and we hope to see a large ecosystem of plugins. Rather than focuses exclusively on k8s, Waypoint is designed to center on the build, deploy, release workflow to any platform via plugins. There's a lot of plugins already built-in [1] and the Plugin SDK [2] makes it pretty straight-forward to build new ones. [1] https://www.waypointproject.io/plugins https://www.waypointproject.io/plugins [2] https://www.waypointproject.io/docs/extending-waypoint https://www.waypointproject.io/docs/extending-waypoint
  - jacques_chester 6y ago
    Nice to see you too! And thanks for the answer.
InTheArena 6y ago
Given the prevalence of Kubernetes, I am not sure more abstractions are needed. Kubernetes if anything is too complete in how it manages it's deployments, and all this does is add another level of abstraction on top of that, while encouraging people not to think about the operational part of the loop. For example, no health or readiness checks are defined as part of the deployment examples in this link. If you are still running multi-deployment with Ansible/ Puppet / Cloud formation, etc I see value, but i think most are probably better off focusing on going K8s.
- deleted 6y ago
  [deleted]
- ghayes 6y ago
  Managing Kubernetes configuration is a task unto itself. I'd say one of the best things that "deserves to be built" is a user-friendly layer that generates Kubernetes configuration that can start simple (e.g. total boilerplate) and be configured by hand as you have deeper requirements. Additionally, build-steps (e.g. how do your containers get built) etc can't be ignored. This all tallies up to me to about a day of engineering work to get a project setup with all the bells and whistles every time I want to start a new serious project on K8s. I definitely think Waypoint (or a similar product) could reduce this time to minutes, so long as I can customize more as the project evolves.
- darthrupert 6y ago
  As a System Architect, I would never be responsible for starting the use of Kubernetes. So there's a clear market for replacing it.
candiddevmike 6y ago
This seems handy for teams that aren't using Kubernetes. If you're already on Kubernetes, there are better tools out there like Argo, Scaffold, Harness, etc with a tighter focus. Curious to know why `waypoint test` is missing, that could be interesting...
GordonS 6y ago
Wow, this looks great! I'm a big fan of Azure DevOps for CI/CD, but I love the idea of something as powerful as Waypoint, but is open source and I can run it anywhere. I wonder if it might be a good tool for deploying infrastructure too - I miss something simpler than Ansible. I'll definitely be dabbling with it.
- N2B 6y ago
  Don’t get too attached to AzDO. Microsoft is trying their damndest to kill it, they’re doubling down on GitHub.
mitchellh 6y ago
Hello HN! I'm the founder of HashiCorp. Waypoint is our 2nd day HashiConf announcement and I'm excited to share and talk about it! Compared to Boundary, Waypoint is definitely weirder, it's trying to do things differently. I'll be around here to answer any questions. I think the most common question will be what is this and why? I cover that in detail on the intro page here so I recommend checking that out: https://www.waypointproject.io/docs/intro https://www.waypointproject.io/docs/intro Here are some major things Waypoint is trying to do: * Make it easier to just deploy. If you're bringing a Ruby app to Kubernetes for example, Waypoint uses buildpacks and an opinionated approach to deploying that app for you to Kubernetes. You don't need to write Dockerfiles or any YAML. You can `waypoint up` and go. If you have existing workflows already, you can use a plugin that just shells out to `kubectl`. The important thing here is you get a common workflow "waypoint up" and your future apps should be much easier to deploy. * Provide common debugging tools for deployments. Waypoint gives you `waypoint exec`, `waypoint logs`, etc. and these work on any platform Waypoint supports. So while K8S for example may provide similar functionality, you can now get identical behavior across K8S and serverless, or your VMs, etc. * Build a common workflow that we can plugin other tools around. This is similar to Terraform circa 2015. There wasn't a consistent way then to think about "infrastructure management" outside of a single target platform's tools. With Waypoint, we're trying to do something similar but for the application deployment lifecycle. As always, a disclaimer that Waypoint is a 0.1 and there is a lot we want to do! We have an exciting roadmap[1] that includes promotion, config syncing with KV stores, Vault and others, service brokering for databases, etc. And also, lots of jokes about Otto in the comments here. :) I think Waypoint and Otto similarities end at "they both deploy" (and Otto with HEAVY quotes around "deploy"). They're totally different tools, one didn't inspire the other, though we did make some major changes to avoid Waypoint hitting the same pitfalls as Otto. Super excited to share Waypoint today! [1]: https://www.waypointproject.io/docs/roadmap https://www.waypointproject.io/docs/roadmap
- solatic 6y ago
  Hi Mitchell, Could you please elaborate further on how you see Waypoint integrating with CI systems? It seems like you expect developers using Waypoint to deploy straight into production from their laptops, which is an anti-pattern, particularly for any company larger than a few people. Companies really need to make sure that code is checked into version control and that tests are run before code is deployed into production. How do you see Waypoint being used in companies where there are more structured pipelines to build code?
whoisjuan 6y ago
This is clearly the successor to Otto. I think the key point here is this: > "This workflow is consistent across any platform, including Kubernetes, Nomad, EC2, Google Cloud Run, and over a dozen more at launch. Waypoint can be extended with plugins to target any build/deploy/release logic." It makes a lot of sense. You provision your infrastructure with Terraform and deploy with Waypoint. This is basically Terraform but for Deployments. I think that's pretty cool. I'm wondering how clever is this though. If I have something running locally a deploy with Waypoint do they figure out all the configuration automatically? One big challenge that I have always had with fresh deployments into new environments it's making them work out of the box. I don't remember the last time I have deployed something without having to change a configuration or mess with the command line of my host to make it work.
Snappy 6y ago
So that's what they've been doing with all the Heroku people they've been hiring. http://hashiroku.com/ http://hashiroku.com/
codeviking 6y ago
This is pretty interesting. I spent the last year building an in-house, Kubernetes based PAAS that has a lot of similar functionality. What we don't do, however, is support multiple execution environments -- we're tightly coupled to Kubernetes. The fact that it's OSS makes it a compelling starting point for other organizations who are beginning their own in-house PAAS efforts (or restarting them). If I were starting over I'd definitely dig into this before deciding to roll my own. I know I'm certainly going to need to make some time to learn more about Waypoint. It might be too much pain to migrate to, as our in-house system is pretty mature -- but it'll at the very least serve as a source of inspiration. NIce work! Do you know how you plan to monetize this in the long-run? I assume it's via a managed offering, where the end-user isn't responsible for hosting the software themselves and instead pays a monthly rate (or pays per use). Knowing what that trajectory looks like we help me feel more comfortable using it, as it'd clarify how it will / will not change.
- jambay 6y ago
  I work at Product Mgmt at HashiCorp. We do hope to have Waypoint as a HashiCorp Cloud offering in the future. Yet we expect the have the core workflows available in open source. There may someday be features in a paid Enterprise edition, similar the approach we've used well with Vault.
  - codeviking 6y ago
    Gotcha, thanks!
ijelliti 6y ago
Great that make life easier for developers. Will be a keyword for logging in the future
kapilvt 6y ago
is this Otto 2.0 ;-) https://www.ottoproject.io https://www.ottoproject.io
- sushicalculus 6y ago
  Is Otto going away?
  - biggestlou 6y ago
    Work on Otto ceased many years ago
originalvichy 6y ago
Oh man, I got excited and thought you guys were finally gonna start developing a CI product. As a Jenkins heavy user I like how extensible it is but it could be simpler and work better and easier with tools like Nomad. I hope you guys hop into the CI server scene. A man can wish.
- gscho 6y ago
  After reading through some examples, I came away with the same thought. I hope that the roadmap extends this into the CI space further!
- sofixa 6y ago
  There are way too many CI tools out there, covering myriads of use cases and niches. Jenkins doesn't do anything better than any other tool.
geerlingguy 6y ago
From https://www.ottoproject.io https://www.ottoproject.io It was an ambitious project, and we feel that it has not lived up to expectations. Rather than having a public project that does not meet HashiCorp standards, we have decided to close source the project so we can rethink its design and implementation. The source code is still available for download, but Otto will no longer be actively maintained or supported. It looks like this is the successor to Otto, just with a bit of a different architecture. The thing I'm worried about in terms of considering trying/adopting it for any project is whether this will suffer the same fate, if it's not as successful as Hashicorp hopes. I don't want to learn new tools that give a small increase in efficiency if I don't have a guaranteed return on that effort and some reasonable confidence using that tool will help me in other areas too. Especially if the code behind the thing has the risk of going closed-source... it's kind of the Google problem (http://killedbygoogle.com http://killedbygoogle.com), and I really hope Hashicorp can avoid that (which they have, as they continue to support well-used even if not-flashy-anymore tools like Vagrant).
- mitchellh 6y ago
  The similarity between Otto and Waypoint are only skin deep: they both use the word "deploy." The way they deploy, their philosophies, etc. are COMPLETELY different. I talk about that in this comment a bit here: https://news.ycombinator.com/item?id=24790491 https://news.ycombinator.com/item?id=24790491 Waypoint is fully open source. There is no risk of it going closed source. If you are alluding to Otto here, Otto's source was never removed either, it is in the Git history of the project that's still on GitHub and we provided a one-click download link on the Otto website when we shut it down. EDIT to add: If you're afraid of us shutting down projects, please note Otto is the only project in our entire history we shut down like that. So we don't (or shouldn't!) have the same reputation around that that Google has. Terraform was effectively failing (no growth) for almost 12 months and we stayed committed to that.
  - paulryanrogers 6y ago
    The concern here is probably more that they become effectively dead without vendor support. While some projects escape that fate they seem to be rare.
kissgyorgy 6y ago
I'm not buying the premise that there is a need for yet-another-abstraction over a high level tool like Kubernetes. You need to learn your underlying deployment platform anyway, introducing yet another tool feels like a distraction.
- freehunter 6y ago
  I disagree for the most part. Developers shouldn’t have to know and understand their infrastructure. It’s great if they do, but making it a requirement is just adding complexity for complexity’s sake. Kubernetes is high level if you’re an infrastructure or ops person. It’s not high level if you’re a developer. This argument sounds an awful lot like “learn C before you learn JavaScript because you’re going to need to know C anyway” combined with “C is a high level language”. That might technically be a true statement but it’s only true for a very small number of people. In my opinion the the goal should always be to have a platform where developers don’t need to worry about the underlying infrastructure. Software development is hard enough as is, and Kubernetes isn’t exactly easy.
  - kissgyorgy 6y ago
    You can't use this tool if you don't know the platform you are deploying to.
jakupovic 6y ago
What I see here is that commands "logs" and "exec" are being commoditized and consumers (developers) are expecting these features out of their platform, whether it's kubernetes or anything else.
Axsuul 6y ago
Is this kind of like Capistrano on steroids?
- NikolaeVarius 6y ago
  Run through of docs really gives me capistrano vibes
- mitchellh 6y ago
  Cap was a motivation to a certain extent for sure. Both myself and Evan Phoenix (the first two core members of the team) come from heavy Ruby backgrounds and we yearned for the days of cap.
pachico 6y ago
If I understand how it works, there's something I find fundamentally wrong about it, which is that an application should not know where nor how it's deployed. It has to know how it's build, of course, but it should be deployment agnostic. Anyhow, I tend to like all Hashicorp projects and I'm grateful for their work.
fizixer 6y ago
What is that conf format? Looks neither YAML, nor JSON. (if your conf files are not a standard format, this project is likely not going to go anywhere IMHO). edit: Apparently non-standard [0]. [0] https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl
- jakelazaroff 6y ago
  Terraform uses HCL, and it’s done pretty well! I think it’s significantly more ergonomic than YAML and JSON (and even slightly more than TOML).
- sciurus 6y ago
  And yet terraform uses the same format and is the leader in its space?
- armon 6y ago
  We use HCL broadly across the HashiCorp portfolio. It was popularized by Terraform, but also used in Vault, Consul, Nomad, and Packer. Important to note, that HCL is designed to be completely interoperable with JSON. In practice, we find that HCL tends to be more human readable and writable and is significantly less verbose. However, if you are doing any sort of machine generation, it can be convenient to generate JSON and feed that in.
- yobert 6y ago
  HCL is a superset of JSON, so tools can still output json. having worked with HCL before for terraform, I can say it is way nicer for human-edited config files than either YAML or JSON.
choeger 6y ago
> You can use exec to open up a shell in your app Ok, maybe we don't exactly speak the same language, but shouldn't that be "a shell in your server running your app".
mrpippy 6y ago
As a not-that-old-but-still-grizzled desktop/embedded developer, I suggest this headline be changed to: "Waypoint: Build, deploy, and release _web_ applications across any _cloud_ platform"
- mkl 6y ago
  Yes, it's very misleading. I thought it would be about apps for desktop and mobile.
- rad_gruchalski 6y ago
  These don‘t have to be web applications. Maybe desktop applications should be referred to as ... „desktop applications“.
gavinray 6y ago
I'm late in the thread and this will probably get buried but: I think this looks amazing. Lot of comments here saying it's yet another useless abstraction. Sure, maybe it doesn't do anything you can't already do yourself, but the experience looks like one I would enjoy and could see myself using. Obviously I can't pass judgement that quickly so will reserve further comments until I can spend some time this weekend experimenting, but it's an experiment I look forward to.
- krimeo 6y ago
  Can't wait for the next layer on top of Waypoint which will allow me to build, deploy and release applications everywhere with one click!
  - Accujack 6y ago
    Or the one after that where I can do it from my Amazon smart speaker!
    - metadaemon 6y ago
      Alexa, Deploy Application. "Ordering Applebee's"
amw-zero 6y ago
How many wrappers of abstractions of wrappers of abstractions do we need?
antoncohen 6y ago
This is great. I'm glad a good company is backing an open source project like this. I've built abstractions like this at two companies, and I hope one day I can stop writing my own. Some comments: I see "promotion" on the roadmap, so this is probably in the works. But the current "Workspaces" each have their own build. I really want to be able to promote the same artifact from one environment (staging) to another (production). I'd also like to be able to choose the artifact build/version to deploy, not just what happens to be in my local repo. The concept of multiple environments also brings up the need to vary things by environment. App config (env vars) are obvious. But also settings like number of replicas or auto scaling min/max (auto scaling is also required). The biggest thing most of the tools in the space lack, like all the ones that try to copy the docker-compose.yml syntax, is standardized reusable app settings. Imagine I have two common types of app, "API services" and "background workers". They have common settings, like maybe they all default to using /healthz for health checks and auto-scaling at 70% CPU. Then within each group they vary, "API services" use internal load balancers and "background workers" don't. I don't want every individual "API service" app to have to say "my health check endpoint is /healthz" and "I run on port 8080 and need a load balancer". Those are the defaults, and if the app uses the defaults it shouldn't need to be configured. But at the same time I want the app to be able to override the defaults. Within a well standardized environment 90% of app infra settings (not env vars) are the same, or can use the same template (like the image is docker.example.com/svc/{app_name}:{build_id}). I want to be able to reuse the settings.
- asim 6y ago
  I think what you're looking for is micro => https://github.com/micro/micro https://github.com/micro/micro or the hosted version https://m3o.com https://m3o.com. Our goal is basically to abstract the entirety of this away to a common experience from the developers viewpoint.
  - antoncohen 6y ago
    Possibly, thanks, I'll look into it. One key is that I'm not looking for an abstraction, I'm looking to provide an abstraction. I run infrastructure teams that provide the infrastructure as a product to other engineering teams, including build and deployment systems. So I need a system that is flexible enough that I can create the abstraction I want to provide. Most of the tools in this space either abstract away everything, in an opinionated way that I don't agree with. Or they abstract away too little, so I would have to write a heavy abstraction over the abstraction. Waypoint, with its plugin system and if the HCL syntax is expanded to support reusable templates/modules, may provide a good building block for me.
hosh 6y ago
This reminds me a bit of Chef Habitat and Automate.
ris 6y ago
Another tool that seems quite heavily focused around making the simplest possible cases look snappy and simple. The simple cases aren't the problem. And when it comes to the complex cases, I have much more faith in the extensibility and power of Nix, which is designed as an actual real programming language to give users the power to solve problems the designers had never envisioned for themselves, than HCL, which seems to be arbitrarily restricted and half-thought-out at the best of times.
- arianvanp 6y ago
  Author of Waypoint seems to be interested in Nix https://github.com/hashicorp/waypoint/commit/f58c13d9fe3581a45b599fa90dcc9814b8c0dbe1 https://github.com/hashicorp/waypoint/commit/f58c13d9fe3581a... :')
jocatalin 6y ago
I’m glad to see new tooling targeting developers and the local development loop which was neglected because historically was hard to monetize. As an consultant involved in cloud migrations and CI/CD I do see the value of a common language for build/test/deploy, it’s a mess in the enterprises nowadays. I’m curious to learn what is Hashicorp’s opinion about CNAB (which somehow tries to solve the same problem). As Waypoint is complementary to CI tools, do you see this also complementary for the GitOps tooling which is gaining traction in the Kubernetes world?
asdev 6y ago
I think codified infrastructure has too much cognitive load/learning curve. It always gets messy, hard to maintain and understand, especially for new developers. Building a multi cloud UI tool is the future.
luord 6y ago
Well, now I know what I'll be doing over the weekend. Specially on using it in tandem with Nomad.
rickette 6y ago
I'm totally buying into the vision. Was discussing this with a co-worker a couple of months ago: there hasn't been much innovation w.r.t deploying apps within the last ~4 years (for on-premise at least). Still the same tools with bad UX, scripts, CLIs. Hope Waypoint can improve the status quo, it sounds like the missing glue.
- krimeo 6y ago
  Hopefully the next revolution is rediscovering the wonders of dynamic linking.
- rudasn 6y ago
  Which tools are you talking about? I'm new to this and I'm just looking around for better options. My use case is app releases and deployments across multiple on premise servers behind private vpn. Deployments are a very manual process, due to the vpn, but we use git to pull the project, docker for managing all the different services, and ansible to manage config per host and run deployment playbooks. What other/better opinions are there, for a very small team of fullstack debs?
rbirkby 6y ago
Waypoint feels similar to Skaffold mixed with some Eirini or Google’s dead kf project.
zapita 6y ago
The documentation says that Waypoint does not want to replace PaaS, but rather to integrate with both PaaS and lower-level infrastructure like Kubernetes. However, the documentation also says that Waypoint injects a “smart entrypoint” into the containers it builds, and that entrypoiny seems very intrusive. Among other things, it calls home to the waypoint server and registers on a “URL service” which will then route traffic back to it. That is basically a PaaS service mesh. So my understanding is that Waypoint, while claiming to be complementary to PaaS, is actually a trojan horse to inject its own PaaS into my infrastructure. I understand why that is valuable to Hashicorp; and I may even be interested in evaluating a Hashicorp PaaS. But I am not a fan of the way this is being snuck in. Injecting a service mesh into my stack is a major change; please be upfront about it so I can compare it to other PaaS.
- mr_toad 6y ago
  The waypoint server is run on your infrastructure.
  - zapita 6y ago
    I am aware of that. What’s your point?
- jambay 6y ago
  Hi, we are not intentionally hiding how Waypoint works. We attempted to make the Entrypoint functionality very clear in the documentation with explicit instructions for how you can turn off Entrypoint injection either during build time (it's never put into the artifact) or at deploy time (a configurable on-off switch). If you turn it off you lose features like the URL Service, logs and exec, but the build, deploy and release workflow is still there. https://www.waypointproject.io/docs/entrypoint/disable#disable-at-runtime https://www.waypointproject.io/docs/entrypoint/disable#disab... https://www.waypointproject.io/docs/entrypoint/disable#disable-at-runtime https://www.waypointproject.io/docs/entrypoint/disable#disab...
  - zapita 6y ago
    Thanks. My issue is not with the feature being hidden, but the positioning being misleading.
jsmeaton 6y ago
I will drill deeper into the documentation during the weekend, but if Mitchell is still around I have a few questions about how this might work for some apps I'm building. 1. Is there ECS support for Fargate, specifically, and how does waypoint exec (for a shell) work if Fargate is supported? We hacked a bastion container together using SSM remote activation, and it feels so brittle. 2. If I want to deploy multiple versions of my app into the same VPC is that possible? My use case is having 1 giant VPC/RDS that multiple app/feature versions will take a slice off and use. Rather than setting up a VPC + all services per staging/feature environment. 3. Is there support for dynamic URLs using native cloud tools? For example, I'd love to use ALB auto-provisioned certs and carve off dynamic URLs on a subdomain. Thanks :)
- mitchellh 6y ago
  1. ECS yes, Fargate I don't think so yet but we're looking into it. 2. Yes, you want this: https://www.waypointproject.io/docs/workspaces https://www.waypointproject.io/docs/workspaces 3. Yes: https://www.waypointproject.io/plugins/aws-ec2#aws-alb-releasemanager https://www.waypointproject.io/plugins/aws-ec2#aws-alb-relea... with more on the way.
  - jsmeaton 6y ago
    Thanks for the responses and pointing me in the right direction, I'm super excited about this project and I love the tools your company builds.
__jem 6y ago
I really like the focus on dev ergonomics, but as someone who works on a crufty older application, the problem I have is not deploying, it's managing a build system that needs to work around existing application code. If I could just refactor the application to use buildpacks, I wouldn't have a problem in the first place!
tpetry 6y ago
Man people are still deploying docker applications on a single server with docker-compose or docker-swarm for it's simplicity. Some of my personal projects are still deployed like this as it is really simple (it's docker with a very thin yaml), works exactly like on my machine and is against my expectations really stable. Is there some plan to support docker-compose or something like this? The "magic" why it is working so good for simple solutions is the service discovery. Since version 1.10 dns resolution by container names in created docker networks is possible which would be enough to replicate most of the functionality of docker-compose. And the feature to start multiple containers for an application is required. Is there some plan to support a lot more docker configuration options? Or is the docker mode just a small neat feature which will not receive much development as kubernetes and nomad will be the main deployment systems for most of the users?
- brujoand 6y ago
  How exactly are you deploying with docker compose? In my head the only approach would be to somehow sync the docker compose file(s) to a server, have them use the latest tag, and then have something like watchtower regularly check for updates on the images and restart the service. Is there some well defined way of doing this remotely from say a Gitlab pipeline?
  - tpetry 6y ago
    I am not doing this automatically, and waypoint seems to follow the same manual approach. I just ssh into my tiny vm and change the docker tag within the env and restart everything. This step is so small it could even be automated but i prefer to control the deployment manually. And every release has a different tag to be able to rollback to an old version, which is not easily doable if you just deploy the latest tag.

549 points by jacobwg 6y ago

I always love HashiCorp products! But... This feels like Otto v2 to me, and it seems like it hasn't actually solved the underlying problem with Otto: that it was simpler to just use and learn the underlying tools instead of a very specific DSL that transformed into them. If I have to look up how to use Waypoint to create Dockerfiles/Kubernetes manifests anyway, why not just learn how to use Dockerfiles and Kubernetes manifests? I'm pretty excited by Boundary, but I don't really see the point of this.

mitchellh 6y ago
Hello! Compared to Otto there are some major simplifications we made: * Waypoint does not own/manage any infrastructure, Otto attempted to manage the infrastructure. With Waypoint, you have to bring this yourself. * Waypoint has pluggable components for build/deploy/release. Otto focused in on our tools (mainly because we had to manage the infra, point 1). This makes it much easier to slide into existing ecosystems. On your question of abstractions: this same argument can be made for Terraform so I think my answer there would be the same but for applications. The idea is that Waypoint can give you a consistent workflow to use different tools, and additional value add features such as deployment URLs, exec, logs, etc. that work across platforms.
- Veraticus 6y ago
  Thanks for the response Mitchell, and I love your work! You make an excellent point re: abstractions and consistent workflow. But for Terraform, you still need to know the underlying provider; to use Terraform in AWS I have to know what AWS resources I'm looking for, Azure resources in Azure, and so on. But Terraform adds value through abstraction. I don't need to learn the specific AWS/Azure calls; Terraform does it for me. Terraform provides a sane, consistent syntax. And it encourages a declarative workflow that the cloud providers themselves don't do very well/at all. I don't necessarily see Waypoint providing that same value. You need to know the underlying provider to know what you want to do with it, but the abstraction seems to make it more difficult to use that provider, not easier. But I am a devops professional, not an application developer, so I might just be the wrong market for it. Either way, congratulations on the release, and I'm excited to see where Waypoint goes from here.
throwaway894345 6y ago
> why not just learn how to use Dockerfiles and Kubernetes manifests? The problem I see with the entire Docker ecosystem is that the Docker/Dockerfile build system is fundamentally terrible, principally because the cache system assumes a linear dependency tree and build graphs are, well, graphs. To get reasonable build performance, you need a tool that understands this. This is something Nix gets right, and it even lets you build Docker images without the Docker/Dockerfile build system. I'd also like to see something that takes it a step further--build the docker image AND a set of kubernetes manifests that reference that docker image (a "kubernetes app package" if you will) such that you have a single artifact that represents your application that a "kubernetes package manager" can deploy (the current crop of k8s package managers altogether punt on the contents of docker images). I understand that this is a very different way of thinking about Kubernetes and Docker builds than what we practice today, but I'd really like to see this in practice or hear some debate about its merits (or lackthereof).
- jacques_chester 6y ago
  Broadly, I agree, and I've written a lot about it in the past[0]. OCI images are not quite as bad as they used to be. Linear caching is no longer baked directly into the image format, instead it's an assumption that has been carried forward by Dockerfiles and docker build. Docker folks are working on at least the docker build part in buildkit[1]. In the meantime though I prefer Cloud Native Buildpacks, which are able to perform layer rebasing as an update operation. Disclosure: I have previously worked on buildpacks technology for Pivotal, now VMware. [0] https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POUaadcvK3ZNT9QEGDZhPk/edit https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU... [1] https://github.com/moby/buildkit https://github.com/moby/buildkit
- nur0n 6y ago
  I believe BuildKit solves the dependency graph problem. Docker has shipped with it since 18.09. It is opt-in for now so you have to use e.g. `DOCKER_BUILDKIT=1 docker build ...`. https://github.com/moby/buildkit https://github.com/moby/buildkit
  - 6y ago

jgilias 6y ago

This feels a bit like that XKCD strip about fourteen competing standards. But I'd love to be proven wrong!

CyanLite2 6y ago
It's exactly that! https://xkcd.com/927/ https://xkcd.com/927/
no_wizard 6y ago
May be worth mentioning that Hashicorp has a lot more sway and influence in this arena than most who might have or are attempting or maybe thought to attempt this in the past. There may be something to be said for that. I know Google and AWS are pretty all in on kubernetes, but this does offer an alternative & a layer of abstraction on top of that. My bigger worry is all these deployment abstractions might lead to a scenario where when something breaks very badly, nobody really knows how to fix it other than to nuke the whole thing and try again. I feel like we're already there with that in a lot of ways with these devops systems. The weird kind of lock in they have to me is that they aren't enabling smaller teams to easily adopt these things, its aimed at either cloud providers or large insitutions that can afford to pay a dedicated engineer to manage these systems. I'd love to see something aimed at smaller teams that makes these orchestrations easy and multifunctional, and work across more platforms, like DO. Hashicorp et. el. know what the best practices are and how to keep things secure, just give me something that enforces those things on any fleet of servers/containers/whathaveyous I want to use. As a developer on a smaller team who spends part of my time working on these concerns, that would help immensely.

sushicalculus 6y ago

In my experience with HashiCorp products, things don't work like the page says (Vault) while other's are sometimes half-finished (Terraform will let you create, but not destroy resources). Edit: To clarify, I ran into some cases where the creation code for a resource existed, but not the destroy code. I might wait on this and read the Github issues for a few months before trying it

aratno 6y ago
Terraform does let you destroy resources: https://www.terraform.io/docs/commands/destroy.html https://www.terraform.io/docs/commands/destroy.html
- sushicalculus 6y ago
  Yes, I know. I meant for some resources (the code for creating was added, but not for destroying those resources). I had to write a custom provider to temporarily fix it
- kawsper 6y ago
  I wish Terraform still allowed destroy-time provisioners to access variables, it seems that it went away due to some refactoring, and it isn't coming back. https://github.com/hashicorp/terraform/issues/23679 https://github.com/hashicorp/terraform/issues/23679
outime 6y ago
I have been using Terraform since 0.11 and it's always allowed to destroy resources, what do you mean?

jacques_chester 6y ago

Hopefully someone at Hashicorp can double check my read: I feel this could be compared to Skaffold or Tilt, but with a heavier emphasis on a plugin architecture. Is that right?

jambay 6y ago
Hey Jacques! Good to chat with you again :) Yes, the plugin architecture is an essential part of Waypoint and we hope to see a large ecosystem of plugins. Rather than focuses exclusively on k8s, Waypoint is designed to center on the build, deploy, release workflow to any platform via plugins. There's a lot of plugins already built-in [1] and the Plugin SDK [2] makes it pretty straight-forward to build new ones. [1] https://www.waypointproject.io/plugins https://www.waypointproject.io/plugins [2] https://www.waypointproject.io/docs/extending-waypoint https://www.waypointproject.io/docs/extending-waypoint
- jacques_chester 6y ago
  Nice to see you too! And thanks for the answer.

InTheArena 6y ago

Given the prevalence of Kubernetes, I am not sure more abstractions are needed. Kubernetes if anything is too complete in how it manages it's deployments, and all this does is add another level of abstraction on top of that, while encouraging people not to think about the operational part of the loop. For example, no health or readiness checks are defined as part of the deployment examples in this link. If you are still running multi-deployment with Ansible/ Puppet / Cloud formation, etc I see value, but i think most are probably better off focusing on going K8s.

deleted 6y ago
[deleted]
ghayes 6y ago
Managing Kubernetes configuration is a task unto itself. I'd say one of the best things that "deserves to be built" is a user-friendly layer that generates Kubernetes configuration that can start simple (e.g. total boilerplate) and be configured by hand as you have deeper requirements. Additionally, build-steps (e.g. how do your containers get built) etc can't be ignored. This all tallies up to me to about a day of engineering work to get a project setup with all the bells and whistles every time I want to start a new serious project on K8s. I definitely think Waypoint (or a similar product) could reduce this time to minutes, so long as I can customize more as the project evolves.
darthrupert 6y ago
As a System Architect, I would never be responsible for starting the use of Kubernetes. So there's a clear market for replacing it.

candiddevmike 6y ago

This seems handy for teams that aren't using Kubernetes. If you're already on Kubernetes, there are better tools out there like Argo, Scaffold, Harness, etc with a tighter focus. Curious to know why `waypoint test` is missing, that could be interesting...

GordonS 6y ago

Wow, this looks great! I'm a big fan of Azure DevOps for CI/CD, but I love the idea of something as powerful as Waypoint, but is open source and I can run it anywhere. I wonder if it might be a good tool for deploying infrastructure too - I miss something simpler than Ansible. I'll definitely be dabbling with it.

N2B 6y ago
Don’t get too attached to AzDO. Microsoft is trying their damndest to kill it, they’re doubling down on GitHub.

mitchellh 6y ago

Hello HN! I'm the founder of HashiCorp. Waypoint is our 2nd day HashiConf announcement and I'm excited to share and talk about it! Compared to Boundary, Waypoint is definitely weirder, it's trying to do things differently. I'll be around here to answer any questions. I think the most common question will be what is this and why? I cover that in detail on the intro page here so I recommend checking that out: https://www.waypointproject.io/docs/intro https://www.waypointproject.io/docs/intro Here are some major things Waypoint is trying to do: * Make it easier to just deploy. If you're bringing a Ruby app to Kubernetes for example, Waypoint uses buildpacks and an opinionated approach to deploying that app for you to Kubernetes. You don't need to write Dockerfiles or any YAML. You can `waypoint up` and go. If you have existing workflows already, you can use a plugin that just shells out to `kubectl`. The important thing here is you get a common workflow "waypoint up" and your future apps should be much easier to deploy. * Provide common debugging tools for deployments. Waypoint gives you `waypoint exec`, `waypoint logs`, etc. and these work on any platform Waypoint supports. So while K8S for example may provide similar functionality, you can now get identical behavior across K8S and serverless, or your VMs, etc. * Build a common workflow that we can plugin other tools around. This is similar to Terraform circa 2015. There wasn't a consistent way then to think about "infrastructure management" outside of a single target platform's tools. With Waypoint, we're trying to do something similar but for the application deployment lifecycle. As always, a disclaimer that Waypoint is a 0.1 and there is a lot we want to do! We have an exciting roadmap[1] that includes promotion, config syncing with KV stores, Vault and others, service brokering for databases, etc. And also, lots of jokes about Otto in the comments here. :) I think Waypoint and Otto similarities end at "they both deploy" (and Otto with HEAVY quotes around "deploy"). They're totally different tools, one didn't inspire the other, though we did make some major changes to avoid Waypoint hitting the same pitfalls as Otto. Super excited to share Waypoint today! [1]: https://www.waypointproject.io/docs/roadmap https://www.waypointproject.io/docs/roadmap

solatic 6y ago
Hi Mitchell, Could you please elaborate further on how you see Waypoint integrating with CI systems? It seems like you expect developers using Waypoint to deploy straight into production from their laptops, which is an anti-pattern, particularly for any company larger than a few people. Companies really need to make sure that code is checked into version control and that tests are run before code is deployed into production. How do you see Waypoint being used in companies where there are more structured pipelines to build code?

whoisjuan 6y ago

This is clearly the successor to Otto. I think the key point here is this: > "This workflow is consistent across any platform, including Kubernetes, Nomad, EC2, Google Cloud Run, and over a dozen more at launch. Waypoint can be extended with plugins to target any build/deploy/release logic." It makes a lot of sense. You provision your infrastructure with Terraform and deploy with Waypoint. This is basically Terraform but for Deployments. I think that's pretty cool. I'm wondering how clever is this though. If I have something running locally a deploy with Waypoint do they figure out all the configuration automatically? One big challenge that I have always had with fresh deployments into new environments it's making them work out of the box. I don't remember the last time I have deployed something without having to change a configuration or mess with the command line of my host to make it work.

Snappy 6y ago

So that's what they've been doing with all the Heroku people they've been hiring. http://hashiroku.com/ http://hashiroku.com/

codeviking 6y ago

This is pretty interesting. I spent the last year building an in-house, Kubernetes based PAAS that has a lot of similar functionality. What we don't do, however, is support multiple execution environments -- we're tightly coupled to Kubernetes. The fact that it's OSS makes it a compelling starting point for other organizations who are beginning their own in-house PAAS efforts (or restarting them). If I were starting over I'd definitely dig into this before deciding to roll my own. I know I'm certainly going to need to make some time to learn more about Waypoint. It might be too much pain to migrate to, as our in-house system is pretty mature -- but it'll at the very least serve as a source of inspiration. NIce work! Do you know how you plan to monetize this in the long-run? I assume it's via a managed offering, where the end-user isn't responsible for hosting the software themselves and instead pays a monthly rate (or pays per use). Knowing what that trajectory looks like we help me feel more comfortable using it, as it'd clarify how it will / will not change.

jambay 6y ago
I work at Product Mgmt at HashiCorp. We do hope to have Waypoint as a HashiCorp Cloud offering in the future. Yet we expect the have the core workflows available in open source. There may someday be features in a paid Enterprise edition, similar the approach we've used well with Vault.
- codeviking 6y ago
  Gotcha, thanks!

ijelliti 6y ago

Great that make life easier for developers. Will be a keyword for logging in the future

kapilvt 6y ago

is this Otto 2.0 ;-) https://www.ottoproject.io https://www.ottoproject.io

sushicalculus 6y ago
Is Otto going away?
- biggestlou 6y ago
  Work on Otto ceased many years ago

originalvichy 6y ago

Oh man, I got excited and thought you guys were finally gonna start developing a CI product. As a Jenkins heavy user I like how extensible it is but it could be simpler and work better and easier with tools like Nomad. I hope you guys hop into the CI server scene. A man can wish.

gscho 6y ago
After reading through some examples, I came away with the same thought. I hope that the roadmap extends this into the CI space further!
sofixa 6y ago
There are way too many CI tools out there, covering myriads of use cases and niches. Jenkins doesn't do anything better than any other tool.

geerlingguy 6y ago

From https://www.ottoproject.io https://www.ottoproject.io It was an ambitious project, and we feel that it has not lived up to expectations. Rather than having a public project that does not meet HashiCorp standards, we have decided to close source the project so we can rethink its design and implementation. The source code is still available for download, but Otto will no longer be actively maintained or supported. It looks like this is the successor to Otto, just with a bit of a different architecture. The thing I'm worried about in terms of considering trying/adopting it for any project is whether this will suffer the same fate, if it's not as successful as Hashicorp hopes. I don't want to learn new tools that give a small increase in efficiency if I don't have a guaranteed return on that effort and some reasonable confidence using that tool will help me in other areas too. Especially if the code behind the thing has the risk of going closed-source... it's kind of the Google problem (http://killedbygoogle.com http://killedbygoogle.com), and I really hope Hashicorp can avoid that (which they have, as they continue to support well-used even if not-flashy-anymore tools like Vagrant).

mitchellh 6y ago
The similarity between Otto and Waypoint are only skin deep: they both use the word "deploy." The way they deploy, their philosophies, etc. are COMPLETELY different. I talk about that in this comment a bit here: https://news.ycombinator.com/item?id=24790491 https://news.ycombinator.com/item?id=24790491 Waypoint is fully open source. There is no risk of it going closed source. If you are alluding to Otto here, Otto's source was never removed either, it is in the Git history of the project that's still on GitHub and we provided a one-click download link on the Otto website when we shut it down. EDIT to add: If you're afraid of us shutting down projects, please note Otto is the only project in our entire history we shut down like that. So we don't (or shouldn't!) have the same reputation around that that Google has. Terraform was effectively failing (no growth) for almost 12 months and we stayed committed to that.
- paulryanrogers 6y ago
  The concern here is probably more that they become effectively dead without vendor support. While some projects escape that fate they seem to be rare.

kissgyorgy 6y ago

I'm not buying the premise that there is a need for yet-another-abstraction over a high level tool like Kubernetes. You need to learn your underlying deployment platform anyway, introducing yet another tool feels like a distraction.

freehunter 6y ago
I disagree for the most part. Developers shouldn’t have to know and understand their infrastructure. It’s great if they do, but making it a requirement is just adding complexity for complexity’s sake. Kubernetes is high level if you’re an infrastructure or ops person. It’s not high level if you’re a developer. This argument sounds an awful lot like “learn C before you learn JavaScript because you’re going to need to know C anyway” combined with “C is a high level language”. That might technically be a true statement but it’s only true for a very small number of people. In my opinion the the goal should always be to have a platform where developers don’t need to worry about the underlying infrastructure. Software development is hard enough as is, and Kubernetes isn’t exactly easy.
- kissgyorgy 6y ago
  You can't use this tool if you don't know the platform you are deploying to.

jakupovic 6y ago

What I see here is that commands "logs" and "exec" are being commoditized and consumers (developers) are expecting these features out of their platform, whether it's kubernetes or anything else.

Axsuul 6y ago

Is this kind of like Capistrano on steroids?

NikolaeVarius 6y ago
Run through of docs really gives me capistrano vibes
mitchellh 6y ago
Cap was a motivation to a certain extent for sure. Both myself and Evan Phoenix (the first two core members of the team) come from heavy Ruby backgrounds and we yearned for the days of cap.

pachico 6y ago

If I understand how it works, there's something I find fundamentally wrong about it, which is that an application should not know where nor how it's deployed. It has to know how it's build, of course, but it should be deployment agnostic. Anyhow, I tend to like all Hashicorp projects and I'm grateful for their work.

fizixer 6y ago

What is that conf format? Looks neither YAML, nor JSON. (if your conf files are not a standard format, this project is likely not going to go anywhere IMHO). edit: Apparently non-standard [0]. [0] https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl

jakelazaroff 6y ago
Terraform uses HCL, and it’s done pretty well! I think it’s significantly more ergonomic than YAML and JSON (and even slightly more than TOML).
sciurus 6y ago
And yet terraform uses the same format and is the leader in its space?
armon 6y ago
We use HCL broadly across the HashiCorp portfolio. It was popularized by Terraform, but also used in Vault, Consul, Nomad, and Packer. Important to note, that HCL is designed to be completely interoperable with JSON. In practice, we find that HCL tends to be more human readable and writable and is significantly less verbose. However, if you are doing any sort of machine generation, it can be convenient to generate JSON and feed that in.
yobert 6y ago
HCL is a superset of JSON, so tools can still output json. having worked with HCL before for terraform, I can say it is way nicer for human-edited config files than either YAML or JSON.

choeger 6y ago

> You can use exec to open up a shell in your app Ok, maybe we don't exactly speak the same language, but shouldn't that be "a shell in your server running your app".

mrpippy 6y ago

As a not-that-old-but-still-grizzled desktop/embedded developer, I suggest this headline be changed to: "Waypoint: Build, deploy, and release _web_ applications across any _cloud_ platform"

mkl 6y ago
Yes, it's very misleading. I thought it would be about apps for desktop and mobile.
rad_gruchalski 6y ago
These don‘t have to be web applications. Maybe desktop applications should be referred to as ... „desktop applications“.

gavinray 6y ago

I'm late in the thread and this will probably get buried but: I think this looks amazing. Lot of comments here saying it's yet another useless abstraction. Sure, maybe it doesn't do anything you can't already do yourself, but the experience looks like one I would enjoy and could see myself using. Obviously I can't pass judgement that quickly so will reserve further comments until I can spend some time this weekend experimenting, but it's an experiment I look forward to.

krimeo 6y ago
Can't wait for the next layer on top of Waypoint which will allow me to build, deploy and release applications everywhere with one click!
- Accujack 6y ago
  Or the one after that where I can do it from my Amazon smart speaker!
  - metadaemon 6y ago
    Alexa, Deploy Application. "Ordering Applebee's"

amw-zero 6y ago

How many wrappers of abstractions of wrappers of abstractions do we need?

antoncohen 6y ago

This is great. I'm glad a good company is backing an open source project like this. I've built abstractions like this at two companies, and I hope one day I can stop writing my own. Some comments: I see "promotion" on the roadmap, so this is probably in the works. But the current "Workspaces" each have their own build. I really want to be able to promote the same artifact from one environment (staging) to another (production). I'd also like to be able to choose the artifact build/version to deploy, not just what happens to be in my local repo. The concept of multiple environments also brings up the need to vary things by environment. App config (env vars) are obvious. But also settings like number of replicas or auto scaling min/max (auto scaling is also required). The biggest thing most of the tools in the space lack, like all the ones that try to copy the docker-compose.yml syntax, is standardized reusable app settings. Imagine I have two common types of app, "API services" and "background workers". They have common settings, like maybe they all default to using /healthz for health checks and auto-scaling at 70% CPU. Then within each group they vary, "API services" use internal load balancers and "background workers" don't. I don't want every individual "API service" app to have to say "my health check endpoint is /healthz" and "I run on port 8080 and need a load balancer". Those are the defaults, and if the app uses the defaults it shouldn't need to be configured. But at the same time I want the app to be able to override the defaults. Within a well standardized environment 90% of app infra settings (not env vars) are the same, or can use the same template (like the image is docker.example.com/svc/{app_name}:{build_id}). I want to be able to reuse the settings.

asim 6y ago
I think what you're looking for is micro => https://github.com/micro/micro https://github.com/micro/micro or the hosted version https://m3o.com https://m3o.com. Our goal is basically to abstract the entirety of this away to a common experience from the developers viewpoint.
- antoncohen 6y ago
  Possibly, thanks, I'll look into it. One key is that I'm not looking for an abstraction, I'm looking to provide an abstraction. I run infrastructure teams that provide the infrastructure as a product to other engineering teams, including build and deployment systems. So I need a system that is flexible enough that I can create the abstraction I want to provide. Most of the tools in this space either abstract away everything, in an opinionated way that I don't agree with. Or they abstract away too little, so I would have to write a heavy abstraction over the abstraction. Waypoint, with its plugin system and if the HCL syntax is expanded to support reusable templates/modules, may provide a good building block for me.

hosh 6y ago

This reminds me a bit of Chef Habitat and Automate.

ris 6y ago

Another tool that seems quite heavily focused around making the simplest possible cases look snappy and simple. The simple cases aren't the problem. And when it comes to the complex cases, I have much more faith in the extensibility and power of Nix, which is designed as an actual real programming language to give users the power to solve problems the designers had never envisioned for themselves, than HCL, which seems to be arbitrarily restricted and half-thought-out at the best of times.

arianvanp 6y ago
Author of Waypoint seems to be interested in Nix https://github.com/hashicorp/waypoint/commit/f58c13d9fe3581a45b599fa90dcc9814b8c0dbe1 https://github.com/hashicorp/waypoint/commit/f58c13d9fe3581a... :')

jocatalin 6y ago

I’m glad to see new tooling targeting developers and the local development loop which was neglected because historically was hard to monetize. As an consultant involved in cloud migrations and CI/CD I do see the value of a common language for build/test/deploy, it’s a mess in the enterprises nowadays. I’m curious to learn what is Hashicorp’s opinion about CNAB (which somehow tries to solve the same problem). As Waypoint is complementary to CI tools, do you see this also complementary for the GitOps tooling which is gaining traction in the Kubernetes world?

asdev 6y ago

I think codified infrastructure has too much cognitive load/learning curve. It always gets messy, hard to maintain and understand, especially for new developers. Building a multi cloud UI tool is the future.

luord 6y ago

Well, now I know what I'll be doing over the weekend. Specially on using it in tandem with Nomad.

rickette 6y ago

I'm totally buying into the vision. Was discussing this with a co-worker a couple of months ago: there hasn't been much innovation w.r.t deploying apps within the last ~4 years (for on-premise at least). Still the same tools with bad UX, scripts, CLIs. Hope Waypoint can improve the status quo, it sounds like the missing glue.

krimeo 6y ago
Hopefully the next revolution is rediscovering the wonders of dynamic linking.
rudasn 6y ago
Which tools are you talking about? I'm new to this and I'm just looking around for better options. My use case is app releases and deployments across multiple on premise servers behind private vpn. Deployments are a very manual process, due to the vpn, but we use git to pull the project, docker for managing all the different services, and ansible to manage config per host and run deployment playbooks. What other/better opinions are there, for a very small team of fullstack debs?

rbirkby 6y ago

Waypoint feels similar to Skaffold mixed with some Eirini or Google’s dead kf project.

zapita 6y ago

The documentation says that Waypoint does not want to replace PaaS, but rather to integrate with both PaaS and lower-level infrastructure like Kubernetes. However, the documentation also says that Waypoint injects a “smart entrypoint” into the containers it builds, and that entrypoiny seems very intrusive. Among other things, it calls home to the waypoint server and registers on a “URL service” which will then route traffic back to it. That is basically a PaaS service mesh. So my understanding is that Waypoint, while claiming to be complementary to PaaS, is actually a trojan horse to inject its own PaaS into my infrastructure. I understand why that is valuable to Hashicorp; and I may even be interested in evaluating a Hashicorp PaaS. But I am not a fan of the way this is being snuck in. Injecting a service mesh into my stack is a major change; please be upfront about it so I can compare it to other PaaS.

mr_toad 6y ago
The waypoint server is run on your infrastructure.
- zapita 6y ago
  I am aware of that. What’s your point?
jambay 6y ago
Hi, we are not intentionally hiding how Waypoint works. We attempted to make the Entrypoint functionality very clear in the documentation with explicit instructions for how you can turn off Entrypoint injection either during build time (it's never put into the artifact) or at deploy time (a configurable on-off switch). If you turn it off you lose features like the URL Service, logs and exec, but the build, deploy and release workflow is still there. https://www.waypointproject.io/docs/entrypoint/disable#disable-at-runtime https://www.waypointproject.io/docs/entrypoint/disable#disab... https://www.waypointproject.io/docs/entrypoint/disable#disable-at-runtime https://www.waypointproject.io/docs/entrypoint/disable#disab...
- zapita 6y ago
  Thanks. My issue is not with the feature being hidden, but the positioning being misleading.

jsmeaton 6y ago

I will drill deeper into the documentation during the weekend, but if Mitchell is still around I have a few questions about how this might work for some apps I'm building. 1. Is there ECS support for Fargate, specifically, and how does waypoint exec (for a shell) work if Fargate is supported? We hacked a bastion container together using SSM remote activation, and it feels so brittle. 2. If I want to deploy multiple versions of my app into the same VPC is that possible? My use case is having 1 giant VPC/RDS that multiple app/feature versions will take a slice off and use. Rather than setting up a VPC + all services per staging/feature environment. 3. Is there support for dynamic URLs using native cloud tools? For example, I'd love to use ALB auto-provisioned certs and carve off dynamic URLs on a subdomain. Thanks :)

mitchellh 6y ago
1. ECS yes, Fargate I don't think so yet but we're looking into it. 2. Yes, you want this: https://www.waypointproject.io/docs/workspaces https://www.waypointproject.io/docs/workspaces 3. Yes: https://www.waypointproject.io/plugins/aws-ec2#aws-alb-releasemanager https://www.waypointproject.io/plugins/aws-ec2#aws-alb-relea... with more on the way.
- jsmeaton 6y ago
  Thanks for the responses and pointing me in the right direction, I'm super excited about this project and I love the tools your company builds.

__jem 6y ago

I really like the focus on dev ergonomics, but as someone who works on a crufty older application, the problem I have is not deploying, it's managing a build system that needs to work around existing application code. If I could just refactor the application to use buildpacks, I wouldn't have a problem in the first place!

tpetry 6y ago

Man people are still deploying docker applications on a single server with docker-compose or docker-swarm for it's simplicity. Some of my personal projects are still deployed like this as it is really simple (it's docker with a very thin yaml), works exactly like on my machine and is against my expectations really stable. Is there some plan to support docker-compose or something like this? The "magic" why it is working so good for simple solutions is the service discovery. Since version 1.10 dns resolution by container names in created docker networks is possible which would be enough to replicate most of the functionality of docker-compose. And the feature to start multiple containers for an application is required. Is there some plan to support a lot more docker configuration options? Or is the docker mode just a small neat feature which will not receive much development as kubernetes and nomad will be the main deployment systems for most of the users?

brujoand 6y ago
How exactly are you deploying with docker compose? In my head the only approach would be to somehow sync the docker compose file(s) to a server, have them use the latest tag, and then have something like watchtower regularly check for updates on the images and restart the service. Is there some well defined way of doing this remotely from say a Gitlab pipeline?
- tpetry 6y ago
  I am not doing this automatically, and waypoint seems to follow the same manual approach. I just ssh into my tiny vm and change the docker tag within the env and restart everything. This step is so small it could even be automated but i prefer to control the deployment manually. And every release has a different tag to be able to rollback to an old version, which is not easily doable if you just deploy the latest tag.