3 ms·

This is a tangent, but your setup sounds interesting to me — would you share more about how to configure such for myself?

1vuio0pswjnm7 2mo ago
Many years ago I started to describe how it works in an HN comment and some reply complained about the idea of terminating TLS, i.e., decrypting, and then re-encrypting. Obviously this sacrifices something, e.g, speed, in order to gain _control_ But this is what Cloudflare does and no one seems to mind Large companies also do this to protect their LANs I'm not running a CDN, only a small home LAN. I'm only procesing a small amount of traffic on a personal computer. This setup is fast enough for me, it's not slow at all The basic configuration is generally: 1. Configure DNS to point to the local proxy listening address #1, a local address, e.g., using a wildcard in a zone file 2. Configure the proxy to terminate TLS, "do stuff", and then forward to proxy UNIX socket path #2 or proxy listening address #2 3. After doing the stuff, the proxy then sends the traffic over the internet The "do stuff" part is personal. It depends on what one wants to do. There are seemingly endless possibilities It's not likely the constantly changing configurations I use would be suitable for others. It's all based on personal preferences and usage habits I rarely use a graphical browser, for example I don't make piecemeal remote DNS queries like most www users. (IME, most A RR's stay the same over long periods.) I get bulk DNS data periodicallly from a variety of sources and load it into the proxy's memory. When I make an HTTP request there is either no DNS lookup because I'm using the IP address of the proxy or there is a single, local DNS lookup which returns the address of the proxy. There is no access to remote DNS When I first decided to start inspecting own TLS traffic by terminating and re-encrypting, I initially tested the idea using socat After I saw that it worked, I started using other software like haproxy I never expected this approach would work well enough but many years have gone by and I'm still using it. The configurations I use are much longer and more complicated than any sample I have ever seen on the www It's funny that Cloudflare is decrypting and re-encryting _other peoples'_ traffic, and this is thought to be AOK, but aside from large companies few people seem interested in doing this with their _own_ traffic on their _own_ computers on their _own_ networks It can be useful, IMHO
- mlrtime 2mo ago
  How much speed are you actually sacrificing? I'd assume modern hardware can decrypt/encrypt TLS very fast. Side note: I remember working for a company in early 2000's that had Sparc III 1U servers we had to buy crypto accelerators to do this until I had to convince them to use dell/linux.
- dwedge 2mo ago
  This sounds like exactly the kind of thing I'd waste a weekend setting up but what do you actually do with the decrypted traffic? What do you inject? I know you said it's personal but maybe some basic ideas. Do you find any websites or services that fail because of cert pinning or similar? Why do you restrict dns caching to periodic intervals, just for external privacy? In terms of the speed I doubt the time to decrypt and encrypt tls is noticeable in modern times, especially given how slow websites have become. It's not like a load balanced website behind cloudflare isn't already doing this 3 times
  - 1vuio0pswjnm7 2mo ago
    "... but what do you actually do with the decrypted traffic?" I read it On the most basic level, consider logging I can add response headers with additional information about the request and these are stored in the log I can create a comprehensive log of every HTTP request, stored in a text file or SQL database I store these logs in an unpopular compression format that allows searching tar'd and compressed files using regex without decompressing them (This is not possible using zstd, gzip, etc.) I use this "browsing history" with a variety of programs Compare this with the design paradigm of storing history inside each individual program, as is the case with a so-called "modern" graphical web browser Other basic proxy functions that I use are (a) control over HTTP headers, including cookies, (b) control over TLS settings, including SNI, and (c) rewriting URLs The settings are site-specific and/or URL-specific. Lists of websites that have special requirements, e.g., need for SNI, specific HTTP headers, TLS version, HTTP versions, etc., are stored in text files. These are mapped into the proxy's memory. Not every website has the same requirements I also use CDN IP address ranges to make decisions about what requirements a website might have Compare this with trying to control these things from within each individual program. In some cases, it is not possible. Or compare with trying to control these things with Javascript. Generally I don't use a graphical browser. Nor do I use NodeJS. I use Lua. As such, Javascript is not the best solution for me Having lists of which websites have some specific requirement(s) allows me to learn about the sites I visit in ways that a so-called "modern" web browser does not The same goes for controlling DNS. I learn about where website operators outsource DNS, how DNS providers (mis)configure their services and when DNS data changes or remains static for years Perhaps the most important proxy function for me is the ability to avoid DNS lookups. The IP address data is stored in an SQL database using a simple, custom format, extracted to a text file and mapped into the proxy's memory Using a proxy allowed me to move away from storing DNS data in, and serving it from, zone files This is only a partial list of functions that I use. I like to experiment Proxies today can do much more than basic functions like the ones I mentioned For example, injecting Javascript or modifying an HTML tag before it reaches a browser is trivial, albeit inefficient from a purist perspective This is not something I need to do most of the time. I have relatively simple needs. Most times, I'm not using a graphical browser. I'm using the command line in textmode, no X11, etc. Cloudflare has a different use case What does Cloudflare actually do with the decrypted traffic. Only CF knows

1vuio0pswjnm7 2mo ago

Many years ago I started to describe how it works in an HN comment and some reply complained about the idea of terminating TLS, i.e., decrypting, and then re-encrypting. Obviously this sacrifices something, e.g, speed, in order to gain _control_ But this is what Cloudflare does and no one seems to mind Large companies also do this to protect their LANs I'm not running a CDN, only a small home LAN. I'm only procesing a small amount of traffic on a personal computer. This setup is fast enough for me, it's not slow at all The basic configuration is generally: 1. Configure DNS to point to the local proxy listening address #1, a local address, e.g., using a wildcard in a zone file 2. Configure the proxy to terminate TLS, "do stuff", and then forward to proxy UNIX socket path #2 or proxy listening address #2 3. After doing the stuff, the proxy then sends the traffic over the internet The "do stuff" part is personal. It depends on what one wants to do. There are seemingly endless possibilities It's not likely the constantly changing configurations I use would be suitable for others. It's all based on personal preferences and usage habits I rarely use a graphical browser, for example I don't make piecemeal remote DNS queries like most www users. (IME, most A RR's stay the same over long periods.) I get bulk DNS data periodicallly from a variety of sources and load it into the proxy's memory. When I make an HTTP request there is either no DNS lookup because I'm using the IP address of the proxy or there is a single, local DNS lookup which returns the address of the proxy. There is no access to remote DNS When I first decided to start inspecting own TLS traffic by terminating and re-encrypting, I initially tested the idea using socat After I saw that it worked, I started using other software like haproxy I never expected this approach would work well enough but many years have gone by and I'm still using it. The configurations I use are much longer and more complicated than any sample I have ever seen on the www It's funny that Cloudflare is decrypting and re-encryting _other peoples'_ traffic, and this is thought to be AOK, but aside from large companies few people seem interested in doing this with their _own_ traffic on their _own_ computers on their _own_ networks It can be useful, IMHO

mlrtime 2mo ago
How much speed are you actually sacrificing? I'd assume modern hardware can decrypt/encrypt TLS very fast. Side note: I remember working for a company in early 2000's that had Sparc III 1U servers we had to buy crypto accelerators to do this until I had to convince them to use dell/linux.
dwedge 2mo ago
This sounds like exactly the kind of thing I'd waste a weekend setting up but what do you actually do with the decrypted traffic? What do you inject? I know you said it's personal but maybe some basic ideas. Do you find any websites or services that fail because of cert pinning or similar? Why do you restrict dns caching to periodic intervals, just for external privacy? In terms of the speed I doubt the time to decrypt and encrypt tls is noticeable in modern times, especially given how slow websites have become. It's not like a load balanced website behind cloudflare isn't already doing this 3 times
- 1vuio0pswjnm7 2mo ago
  "... but what do you actually do with the decrypted traffic?" I read it On the most basic level, consider logging I can add response headers with additional information about the request and these are stored in the log I can create a comprehensive log of every HTTP request, stored in a text file or SQL database I store these logs in an unpopular compression format that allows searching tar'd and compressed files using regex without decompressing them (This is not possible using zstd, gzip, etc.) I use this "browsing history" with a variety of programs Compare this with the design paradigm of storing history inside each individual program, as is the case with a so-called "modern" graphical web browser Other basic proxy functions that I use are (a) control over HTTP headers, including cookies, (b) control over TLS settings, including SNI, and (c) rewriting URLs The settings are site-specific and/or URL-specific. Lists of websites that have special requirements, e.g., need for SNI, specific HTTP headers, TLS version, HTTP versions, etc., are stored in text files. These are mapped into the proxy's memory. Not every website has the same requirements I also use CDN IP address ranges to make decisions about what requirements a website might have Compare this with trying to control these things from within each individual program. In some cases, it is not possible. Or compare with trying to control these things with Javascript. Generally I don't use a graphical browser. Nor do I use NodeJS. I use Lua. As such, Javascript is not the best solution for me Having lists of which websites have some specific requirement(s) allows me to learn about the sites I visit in ways that a so-called "modern" web browser does not The same goes for controlling DNS. I learn about where website operators outsource DNS, how DNS providers (mis)configure their services and when DNS data changes or remains static for years Perhaps the most important proxy function for me is the ability to avoid DNS lookups. The IP address data is stored in an SQL database using a simple, custom format, extracted to a text file and mapped into the proxy's memory Using a proxy allowed me to move away from storing DNS data in, and serving it from, zone files This is only a partial list of functions that I use. I like to experiment Proxies today can do much more than basic functions like the ones I mentioned For example, injecting Javascript or modifying an HTML tag before it reaches a browser is trivial, albeit inefficient from a purist perspective This is not something I need to do most of the time. I have relatively simple needs. Most times, I'm not using a graphical browser. I'm using the command line in textmode, no X11, etc. Cloudflare has a different use case What does Cloudflare actually do with the decrypted traffic. Only CF knows