7 ms·

Not sure how I feel about the HTTPS/TLS related bits. These days anything I write in Go uses plain HTTP, and the TLS is done by a reverse proxy of some variety

Not sure how I feel about the HTTPS/TLS related bits. These days anything I write in Go uses plain HTTP, and the TLS is done by a reverse proxy of some variety that does some other stuff with the traffic too including security headers, routing for different paths to different services, etc. I never run a go web application "bare", public facing, and manually supplying cert files.

bayindirh 2y ago
While I understand the sentiment, this makes bare installations too hard. A big project not handling HTTPS themselves (like docmost), adds tons of complexity on the server side. Now, I have to install that service as a container to isolate that, then need to add a reverse proxy on top, etc. That leads to resource inflation when I just want to use a small VM for that single task. Now, instead I deploy a whole infrastructure to run that small thing.
- NhanH 2y ago
  Handling https in the project also adds tons of complexity in the long run though: tls/ssl library versions, cert handling. Instead of having one way to deal with all of them (at the proxy layer, or sometimes at network layer), I have to deal with individual software way of managing those
  - kortilla 2y ago
    I think you’re vastly overestimating the complexity of pointing a TLS library to a CA.
    - thayne 2y ago
      For a single application, it's not too bad. When you have dozens of applications that all have different mechanisms to install a CA, rotate certs, etc. And some of those don't have a good way to automate rotating the certs, then it becomes a pain.
      - bayindirh 2y ago
        I don't think so. We run a horde of machines, and run plethora of services, which are custom and/or has a very narrow install base due to the niche they serve. 99% of them use system-wide PKI store for CA and their certificates, which is under /etc. All of them have configuration options for these folders, and have short certificate lives because of the operational regulations we have in place. At worst case, we distribute them with saltstack. Otherwise we just use scp, maybe a small wrapper script. Managing them doesn't take any measurable time for us. ...and we have our own CA on top of it.
        thayne 2y ago
        > 99% of them use system-wide PKI store for CA and their certificates, which is under /etc. Consider yourself lucky then. For self-hosted third party software, I've seen requirements to provide it in an environment variable, upload it to a web form over plain http on localhost, specify an AWS secret service secret that contains it, put it in a specific location that is bind mounted into a container, create a custom image (both VM and container), etc.
    - hnlmorg 2y ago
      Their point isn’t about the complexity of installing a certificate. It’s doing it successfully and securely at scale. Everything is easy until you have to do it at scale.
    - 9dev 2y ago
      Do that for a bunch of different applications and you hit interesting issues. For example the Java TLS stack, which doesn’t accept a PEM certificate on its own, but needs the full certificate chain. Kibana, however, requires the full certificate chain including the root certificate, which isn’t usually a part of the certificate itself, and Elasticsearch complains about an invalid certificate if you point it to the same one. So even for two apps from the same vendor, which are commonly deployed together, you need bespoke TLS file variants. Scale that to more applications, and you’ll find out the hard way that you are vastly underestimating the complexity of operating a software ecosystem.
      - kortilla 2y ago
        I’ve done it. What you’re describing is like an hour of work. Moving TLS outside of the application is possibly the dumbest reason to spend the resources and complexity on a side car.
  - bayindirh 2y ago
    I mean, tons of "old-school" services handle these things fine for the last two decades, at least. It can't be that hard. It's just a TLS library, and some files in a specific format at the end of the day.
    - redundantly 2y ago
      In my experience, most people have an extremely hard time wrapping their minds around how to configure TLS/HTTPS services and fail completely at understanding how it works.
  - ptman 2y ago
    https://pkg.go.dev/golang.org/x/crypto/acme/autocert https://pkg.go.dev/golang.org/x/crypto/acme/autocert
- yjftsjthsd-h 2y ago
  > Now, I have to install that service as a container to isolate that, then need to add a reverse proxy on top, etc. Why? I've run plenty of normal non-containerized apps that bind localhost:1234 and then are reverse proxied by nginx or caddy or whatever. (I agree that you would need a reverse proxy, I think that's kinda the point, it's the container thing I don't get)
  - rjh29 2y ago
    Global vs local variables.
  - bayindirh 2y ago
    Because some of the applications are "container native" and do not support configuration of IP/Port binding. Why? UNIX philosophy and working traditions be damned. Exhibit A: Docmost: https://docmost.com/docs/self-hosting/environment-variables https://docmost.com/docs/self-hosting/environment-variables I can understand the reverse proxy, but I want to run thing on a small VPS or a Raspberry Pi, etc. I want to use minimum resources so I can run more things per server. If this thing was being installed at work, I can build a great wall in front of it, but for personal things, I rather have less moving parts, and just deploy and forget the thing.
    - sofixa 2y ago
      > Because some of the applications are "container native" and do not support configuration of IP/Port binding. Why? UNIX philosophy and working traditions be damned. I'm not sure I follow. Docmost runs in a container, on a port which is configurable. By default, as all containers, that port is local to the container. The container orchestrator (be it docker CLI, docker-compose, Swarm, Nomad, Kubernetes, Podman) is the one who you instruct to expose a port from the container network to your host network. Docker tries to be smart and easy and will expose it on all networks by default, but even on it it's configurable, let alone on the more advanced options.
      - bayindirh 2y ago
        Let me try to explain myself clearer. First, any enterprise/homelab installation with significant resources is outside of my scope. I don't care about them. When you don't pay for the infra/power cost, all bets are off. You can install quadruple redundant 300 node K8S cluster sitting on two different data centers in different cities connected to their own power generation equipment for a simple docmost instance. I'm talking about small fry installations, for personal reasons, on small equipment. Think Raspi5/N100 NUC boxes as the target hardware. I have limited resources, want no heat/noise at my home or want to pay as small as possible for a good VPS. Now, for something like docmost, I need a container runtime, and ingress controller/a reverse proxy, because I want to open this service to a VPN or web somehow. I don't have much qualms with containers, as long as there are no cut corners. We'll talk about two examples here. First is Wiki.js. Comes with 3 containers. DB, Wiki, Updater. OK. They are not much resource heavy, so acceptable. On top, Wiki container handles its own HTTPS, TLS certificates via Let's Encrypt (and allows custom ones, as well). It's the nice one. I can expose it, or hide it, I can handle my certificates or it can handle itself. Nice, batteries included. Install in 10 seconds, update occasionally, enjoy. Second is docmost. It's a nice tool. Allows me to create a space and put a bunch of people in to collaborate. Install for desktop in 10 seconds & Works. Nice. I want to open it to outside. Forward a couple of ports? No. There's no HTTPS. Now I need to terminate it. Traefik? Too big for the job. Apache, doable but needs half an hour from get go. Can I limit it to localhost for local installations, for myself? No. As long as I run it, it binds to 0.0.0.0. So, if I want to install it locally to my desktop, I still need to add a firewall in front of it, because whole LAN can see it, and no, I don't trust any devices on my network (not people, but the apps). This is my problem. I want batteries included solutions, which can adapt to my circumstances. I don't have the desire to run n servers for m services, because they are too lazy to be flexible to adapt their environments. It's not I can't do it, it's the contrary, but I don't want to, because I want to live my life and have time for people and things I care. The services I use must be able to adaptable to my environment. This is how bare-metal services, daemons in UNIX world work. I can fit a whole rack of services in a powerful enough server and isolate them if required. It saves space, power, time and sanity. Littering services with services in boxes and putting more boxes on top of that because their developers are lazy makes my blood boil.
- deleted 2y ago
  [deleted]
- nirui 2y ago
  > Now, I have to install that service as a container to isolate that, then need to add a reverse proxy on top, etc. You can setup a Traefik (or some other ingress service) instance in a container and let it handle all the reverse proxying thingies for you. And if you do it right, the services should automatically register to the ingress service as they start up, and a port/HTTP route should be automatically assigned to them. Do it in the old bare is harder and probably will always be harder, since you will be directly interact with OS facilities that was probably designed for something else than what you might try to run. Container management service such as Docker and Kubernetes abstracted away a lot of these complexities. Funny enough, Traefik is written in Go... guess we've gone some (maybe not full) cycle on this one.
  - bayindirh 2y ago
    Honestly, doing things on bare server and interacting with OS is easier because it involves less moving parts and everything is in a more accessible state. Containers are not bad per se, but cutting corners just because "this will run in a container, so they can just add another HTTPS terminator" is just carelessness IMHO. Because not all of us have homelabs at home to install an onion of services to run a simple service open to outside. A good example of this is Wiki.js. It's desinged as container native, but handles its own ingress, HTTPS and Let's Encrypt certificates. I have no qualms to it, but when another tool just cuts corners and tells you that "It's easy to install, but bring your own secure ingress layer on top", it gets ugly. Because it adds moving parts, most importantly wastes resources for a 3 person installation on small hardware, etc. Keep in mind, these are tools designed for small user-bases. They're not enterprise software. On my day job, we call 80 machine clusters "small". But this is not about things I install/manage at my job.
    - treflop 2y ago
      While I am not ready to recommend that everyone install Traefik, this is false. You can get a single node Docker “cluster” going with Traefik in 15 seconds. There is no maintenance except updating occasionally. It doesn’t use much more resources. You do not need to install any third party tools. There is no onion of services. You literally just boot up Traefik plus your app. This has been doable since at least 2019 by just installing Docker via your OS’ package manager. I’ve started using containers before 99% of people and so got to see the fundamentals build up. You do not need to skip directly to “Kubernetes.” That’s like needing to wash your clothes so you skip directly to buying an industrial washing machine and then lamenting how all washing machines are overkill.
ongy 2y ago
I suspect this is partially from google's internal 0 trust cluster networking. I.e. even if the communication is entirely between components inside a k8s (or borg) cluster, it should be authenticated and encrypted. In this model, there may be a reverse proxy at the edge of the cluster, but the communication between this service and the internal services wouls still be https. With systems like cert-manager it's also incredibly easy to supply every in-cluster process with a certificate form the cluster-internal CA. -- Googler, not related to this project
- cyberpunk 2y ago
  Why wouldn’t you use istio or cilium for this?
  - grogenaut 2y ago
    Why add another layer if you aren't already using istio or cilium?
    - cyberpunk 2y ago
      Because it’s zero configuration auto mtls between all the services in your cluster (or intra-node if cillium) instead of managing a tls cert for every service?
      - sofixa 2y ago
        Zero to little configuration at point of use, but a lot of upfront configuration, maintenance, fun issues when you need something slightly less traditional (e.g. something that needs raw TCP or heavens forbid, UDP). Different trade offs for different situations.
        cyberpunk 2y ago
        I still think it’s far less than managing tls per service. Every component needs a different tls configuration, vs one time installing istio. Raw TCP is supported by istio even with mtls, you just have to match in your VirtualServices on SNI instead of Host header. We routinely mix tcp and http services on the same external ports, with mtls for both. UDP I don’t really see how is relevant to a conversation on tls
effdee 2y ago
The phrase "SSL added and removed here" from an NSA slide comes to mind.
- pushupentry1219 2y ago
  To be clear I meant something like Caddy, or nginx not a solution like cloudflare or GCP doing my TLS
  - arccy 2y ago
    once you outgrow a single machine, unsecured network connections become an issue again
karmarepellent 2y ago
I have use cases for both approaches (letting a reverse proxy handle TLS, letting the application listen on an external socket and handling TLS in the application). I find is is easier to configure an application with a reverse proxy in front when different paths require e.g. different cache-control response headers. At the end of the day I do not want to replicate all the logic that nginx (and others) already provide when it integrates well with the application at the back. Other commenters suggest that both ways (with or without additional reverse proxy) add "tons of complexity". I don't see why. Using a reverse proxy is what we have done for a while now. Installation and configuration (with a reasonable amount of hardening) is not complex and there exist a lot of resources to make it easier. And leaving the reverse proxy out and handling TLS in the application itself should not be "complex" either. Just parse a certificate and private key and supply them to whatever web framework you happen to use.
- eptcyka 2y ago
  And implement cert reloading if your application reaches any kind of respectable uptime.
KelvinFineBoy69 2y ago
[dead]
Thorrez 2y ago
What HTTPS/TLS related bits are you talking about? The code looks like it supports your use case. It has a "BehindProxy" setting: https://github.com/google/go-safeweb/blob/c2d1215a6a2445915c6971347b2f4bd0c582a519/safehttp/plugins/hsts/hsts.go#L70 https://github.com/google/go-safeweb/blob/c2d1215a6a2445915c...

bayindirh 2y ago

While I understand the sentiment, this makes bare installations too hard. A big project not handling HTTPS themselves (like docmost), adds tons of complexity on the server side. Now, I have to install that service as a container to isolate that, then need to add a reverse proxy on top, etc. That leads to resource inflation when I just want to use a small VM for that single task. Now, instead I deploy a whole infrastructure to run that small thing.

NhanH 2y ago
Handling https in the project also adds tons of complexity in the long run though: tls/ssl library versions, cert handling. Instead of having one way to deal with all of them (at the proxy layer, or sometimes at network layer), I have to deal with individual software way of managing those
- kortilla 2y ago
  I think you’re vastly overestimating the complexity of pointing a TLS library to a CA.
  - thayne 2y ago
    For a single application, it's not too bad. When you have dozens of applications that all have different mechanisms to install a CA, rotate certs, etc. And some of those don't have a good way to automate rotating the certs, then it becomes a pain.
    - bayindirh 2y ago
      I don't think so. We run a horde of machines, and run plethora of services, which are custom and/or has a very narrow install base due to the niche they serve. 99% of them use system-wide PKI store for CA and their certificates, which is under /etc. All of them have configuration options for these folders, and have short certificate lives because of the operational regulations we have in place. At worst case, we distribute them with saltstack. Otherwise we just use scp, maybe a small wrapper script. Managing them doesn't take any measurable time for us. ...and we have our own CA on top of it.
      - thayne 2y ago
        > 99% of them use system-wide PKI store for CA and their certificates, which is under /etc. Consider yourself lucky then. For self-hosted third party software, I've seen requirements to provide it in an environment variable, upload it to a web form over plain http on localhost, specify an AWS secret service secret that contains it, put it in a specific location that is bind mounted into a container, create a custom image (both VM and container), etc.
  - hnlmorg 2y ago
    Their point isn’t about the complexity of installing a certificate. It’s doing it successfully and securely at scale. Everything is easy until you have to do it at scale.
  - 9dev 2y ago
    Do that for a bunch of different applications and you hit interesting issues. For example the Java TLS stack, which doesn’t accept a PEM certificate on its own, but needs the full certificate chain. Kibana, however, requires the full certificate chain including the root certificate, which isn’t usually a part of the certificate itself, and Elasticsearch complains about an invalid certificate if you point it to the same one. So even for two apps from the same vendor, which are commonly deployed together, you need bespoke TLS file variants. Scale that to more applications, and you’ll find out the hard way that you are vastly underestimating the complexity of operating a software ecosystem.
    - kortilla 2y ago
      I’ve done it. What you’re describing is like an hour of work. Moving TLS outside of the application is possibly the dumbest reason to spend the resources and complexity on a side car.
- bayindirh 2y ago
  I mean, tons of "old-school" services handle these things fine for the last two decades, at least. It can't be that hard. It's just a TLS library, and some files in a specific format at the end of the day.
  - redundantly 2y ago
    In my experience, most people have an extremely hard time wrapping their minds around how to configure TLS/HTTPS services and fail completely at understanding how it works.
- ptman 2y ago
  https://pkg.go.dev/golang.org/x/crypto/acme/autocert https://pkg.go.dev/golang.org/x/crypto/acme/autocert
yjftsjthsd-h 2y ago
> Now, I have to install that service as a container to isolate that, then need to add a reverse proxy on top, etc. Why? I've run plenty of normal non-containerized apps that bind localhost:1234 and then are reverse proxied by nginx or caddy or whatever. (I agree that you would need a reverse proxy, I think that's kinda the point, it's the container thing I don't get)
- rjh29 2y ago
  Global vs local variables.
- bayindirh 2y ago
  Because some of the applications are "container native" and do not support configuration of IP/Port binding. Why? UNIX philosophy and working traditions be damned. Exhibit A: Docmost: https://docmost.com/docs/self-hosting/environment-variables https://docmost.com/docs/self-hosting/environment-variables I can understand the reverse proxy, but I want to run thing on a small VPS or a Raspberry Pi, etc. I want to use minimum resources so I can run more things per server. If this thing was being installed at work, I can build a great wall in front of it, but for personal things, I rather have less moving parts, and just deploy and forget the thing.
  - sofixa 2y ago
    > Because some of the applications are "container native" and do not support configuration of IP/Port binding. Why? UNIX philosophy and working traditions be damned. I'm not sure I follow. Docmost runs in a container, on a port which is configurable. By default, as all containers, that port is local to the container. The container orchestrator (be it docker CLI, docker-compose, Swarm, Nomad, Kubernetes, Podman) is the one who you instruct to expose a port from the container network to your host network. Docker tries to be smart and easy and will expose it on all networks by default, but even on it it's configurable, let alone on the more advanced options.
    - bayindirh 2y ago
      Let me try to explain myself clearer. First, any enterprise/homelab installation with significant resources is outside of my scope. I don't care about them. When you don't pay for the infra/power cost, all bets are off. You can install quadruple redundant 300 node K8S cluster sitting on two different data centers in different cities connected to their own power generation equipment for a simple docmost instance. I'm talking about small fry installations, for personal reasons, on small equipment. Think Raspi5/N100 NUC boxes as the target hardware. I have limited resources, want no heat/noise at my home or want to pay as small as possible for a good VPS. Now, for something like docmost, I need a container runtime, and ingress controller/a reverse proxy, because I want to open this service to a VPN or web somehow. I don't have much qualms with containers, as long as there are no cut corners. We'll talk about two examples here. First is Wiki.js. Comes with 3 containers. DB, Wiki, Updater. OK. They are not much resource heavy, so acceptable. On top, Wiki container handles its own HTTPS, TLS certificates via Let's Encrypt (and allows custom ones, as well). It's the nice one. I can expose it, or hide it, I can handle my certificates or it can handle itself. Nice, batteries included. Install in 10 seconds, update occasionally, enjoy. Second is docmost. It's a nice tool. Allows me to create a space and put a bunch of people in to collaborate. Install for desktop in 10 seconds & Works. Nice. I want to open it to outside. Forward a couple of ports? No. There's no HTTPS. Now I need to terminate it. Traefik? Too big for the job. Apache, doable but needs half an hour from get go. Can I limit it to localhost for local installations, for myself? No. As long as I run it, it binds to 0.0.0.0. So, if I want to install it locally to my desktop, I still need to add a firewall in front of it, because whole LAN can see it, and no, I don't trust any devices on my network (not people, but the apps). This is my problem. I want batteries included solutions, which can adapt to my circumstances. I don't have the desire to run n servers for m services, because they are too lazy to be flexible to adapt their environments. It's not I can't do it, it's the contrary, but I don't want to, because I want to live my life and have time for people and things I care. The services I use must be able to adaptable to my environment. This is how bare-metal services, daemons in UNIX world work. I can fit a whole rack of services in a powerful enough server and isolate them if required. It saves space, power, time and sanity. Littering services with services in boxes and putting more boxes on top of that because their developers are lazy makes my blood boil.
deleted 2y ago
[deleted]
nirui 2y ago
> Now, I have to install that service as a container to isolate that, then need to add a reverse proxy on top, etc. You can setup a Traefik (or some other ingress service) instance in a container and let it handle all the reverse proxying thingies for you. And if you do it right, the services should automatically register to the ingress service as they start up, and a port/HTTP route should be automatically assigned to them. Do it in the old bare is harder and probably will always be harder, since you will be directly interact with OS facilities that was probably designed for something else than what you might try to run. Container management service such as Docker and Kubernetes abstracted away a lot of these complexities. Funny enough, Traefik is written in Go... guess we've gone some (maybe not full) cycle on this one.
- bayindirh 2y ago
  Honestly, doing things on bare server and interacting with OS is easier because it involves less moving parts and everything is in a more accessible state. Containers are not bad per se, but cutting corners just because "this will run in a container, so they can just add another HTTPS terminator" is just carelessness IMHO. Because not all of us have homelabs at home to install an onion of services to run a simple service open to outside. A good example of this is Wiki.js. It's desinged as container native, but handles its own ingress, HTTPS and Let's Encrypt certificates. I have no qualms to it, but when another tool just cuts corners and tells you that "It's easy to install, but bring your own secure ingress layer on top", it gets ugly. Because it adds moving parts, most importantly wastes resources for a 3 person installation on small hardware, etc. Keep in mind, these are tools designed for small user-bases. They're not enterprise software. On my day job, we call 80 machine clusters "small". But this is not about things I install/manage at my job.
  - treflop 2y ago
    While I am not ready to recommend that everyone install Traefik, this is false. You can get a single node Docker “cluster” going with Traefik in 15 seconds. There is no maintenance except updating occasionally. It doesn’t use much more resources. You do not need to install any third party tools. There is no onion of services. You literally just boot up Traefik plus your app. This has been doable since at least 2019 by just installing Docker via your OS’ package manager. I’ve started using containers before 99% of people and so got to see the fundamentals build up. You do not need to skip directly to “Kubernetes.” That’s like needing to wash your clothes so you skip directly to buying an industrial washing machine and then lamenting how all washing machines are overkill.

ongy 2y ago

I suspect this is partially from google's internal 0 trust cluster networking. I.e. even if the communication is entirely between components inside a k8s (or borg) cluster, it should be authenticated and encrypted. In this model, there may be a reverse proxy at the edge of the cluster, but the communication between this service and the internal services wouls still be https. With systems like cert-manager it's also incredibly easy to supply every in-cluster process with a certificate form the cluster-internal CA. -- Googler, not related to this project

cyberpunk 2y ago
Why wouldn’t you use istio or cilium for this?
- grogenaut 2y ago
  Why add another layer if you aren't already using istio or cilium?
  - cyberpunk 2y ago
    Because it’s zero configuration auto mtls between all the services in your cluster (or intra-node if cillium) instead of managing a tls cert for every service?
    - sofixa 2y ago
      Zero to little configuration at point of use, but a lot of upfront configuration, maintenance, fun issues when you need something slightly less traditional (e.g. something that needs raw TCP or heavens forbid, UDP). Different trade offs for different situations.
      - cyberpunk 2y ago
        I still think it’s far less than managing tls per service. Every component needs a different tls configuration, vs one time installing istio. Raw TCP is supported by istio even with mtls, you just have to match in your VirtualServices on SNI instead of Host header. We routinely mix tcp and http services on the same external ports, with mtls for both. UDP I don’t really see how is relevant to a conversation on tls

effdee 2y ago

The phrase "SSL added and removed here" from an NSA slide comes to mind.

pushupentry1219 2y ago
To be clear I meant something like Caddy, or nginx not a solution like cloudflare or GCP doing my TLS
- arccy 2y ago
  once you outgrow a single machine, unsecured network connections become an issue again

karmarepellent 2y ago

I have use cases for both approaches (letting a reverse proxy handle TLS, letting the application listen on an external socket and handling TLS in the application). I find is is easier to configure an application with a reverse proxy in front when different paths require e.g. different cache-control response headers. At the end of the day I do not want to replicate all the logic that nginx (and others) already provide when it integrates well with the application at the back. Other commenters suggest that both ways (with or without additional reverse proxy) add "tons of complexity". I don't see why. Using a reverse proxy is what we have done for a while now. Installation and configuration (with a reasonable amount of hardening) is not complex and there exist a lot of resources to make it easier. And leaving the reverse proxy out and handling TLS in the application itself should not be "complex" either. Just parse a certificate and private key and supply them to whatever web framework you happen to use.

eptcyka 2y ago
And implement cert reloading if your application reaches any kind of respectable uptime.

KelvinFineBoy69 2y ago

[dead]

Thorrez 2y ago

What HTTPS/TLS related bits are you talking about? The code looks like it supports your use case. It has a "BehindProxy" setting: https://github.com/google/go-safeweb/blob/c2d1215a6a2445915c6971347b2f4bd0c582a519/safehttp/plugins/hsts/hsts.go#L70 https://github.com/google/go-safeweb/blob/c2d1215a6a2445915c...