4 ms·

Yes, go based programs observe the proxy variable unless specifically compiled not to. But that doesn't break applications, they just ignore the env var. Go's d

Yes, go based programs observe the proxy variable unless specifically compiled not to. But that doesn't break applications, they just ignore the env var. Go's default package manager doesn't seem to work at all with HTTP Toolkit. Here's what I did. * Intercept Tab > Fresh terminal. * In the terminal, do your usual stuff, it intercepts curl, etc. * If you try to use go's package manager, example: `go get golang.org/x/oauth2` It errors out with ``` go get: module golang.org/x/oauth2: reading http://127.0.0.1:8000/golang.org/x/oauth2/@v/list http://127.0.0.1:8000/golang.org/x/oauth2/@v/list: 500 Server error ``` Ideally it shouldn't break the application, just ignore if it can't intercept. > absolutely not a hard guarantee for applications that actively ignore standard proxy configuration (handling that is very hard, and definitely out of scope here). I encountered a usecase where this was needed and LD_PRELOAD trick (used by proxychains) etc failed because the application was statically compiled. I ended up using https://github.com/hmgle/graftcp https://github.com/hmgle/graftcp which somehow manages to force tcp traffic through a socks5 proxy.

pimterry 5y ago
Thanks, that's super useful. > If you try to use go's package manager, example: `go get golang.org/x/oauth2` I just tested, and `go get golang.org/x/oauth2` seems to work fine for me, I can see all the requests being happily intercepted immediately: https://imgur.com/a/Cb1y9Q2 https://imgur.com/a/Cb1y9Q2 Can you see the 500 in HTTP Toolkit, and any more info there (in the body or as an error at the top) related to that? Or can you see a "certificate rejected" message? If nothing turns up there at all then yes, something must be overriding the proxy configuration. Maybe you have some other Go package manager configuration that conflicts with this? I'd be very interested to know about that if so, I'm sure there's others with the same thing. It's always very hard to know if my configuration is representative of normal devs for any given language/tool. Probably best to debug this outside of a HN thread though :-). You can file a proper issue about this at https://github.com/httptoolkit/httptoolkit/issues/new https://github.com/httptoolkit/httptoolkit/issues/new, I'd love to know what's going on there and get this fixed. > I ended up using https://github.com/hmgle/graftcp https://github.com/hmgle/graftcp which somehow manages to force tcp traffic through a socks5 proxy. Really interesting, thanks! I'll look into that.
- IceWreck 5y ago
  This is the error I see in the response body in HTTP Toolkit. ``` Error: Passthrough loop detected. This probably means you're sending a request directly to a passthrough endpoint, which is forwarding it to the target URL, which is a passthrough endpoint, which is forwarding it to the target URL, which is a passthrough endpoint... You should either explicitly mock a response for this URL (http://127.0.0.1:8000/golang.org/x/oauth2/@v/list http://127.0.0.1:8000/golang.org/x/oauth2/@v/list), or use the server as a proxy, instead of making requests to it directly. ```
  - pimterry 5y ago
    That's really interesting! That's what appears if you HTTP Toolkit receives a request, proxies to the provided destination, and then finds itself on the other end of its own outgoing socket. This avoids the simplest infinite proxy loops. That'll make if you try to proxy a request to `127.0.0.1:8000` via HTTP Toolkit for example. That shouldn't be happening here though - there's no way Go should be using the proxy address as its target URL. I.e. to see this result, it must be sending a request like: GET /golang.org/x/oauth2/@v/list Host: 127.0.0.1:8000 You can probably see the full details in HTTP Toolkit's request data. Any chance you've added a rule in HTTP Toolkit that's redirecting traffic back to itself? That could do this, and would show up as two identical requests in HTTP Toolkit. If not (you can reset the rules by hitting the trash icon on the mock page) then this means something in your Go setup is either configured very strangely or its interpreting HTTP Toolkit's configuration incorrectly. Sorry I can't be more helpful, I have no idea why `go get` would send a request like that! If you can pin it down to something I can reproduce (e.g. a specific Go version or config setting or something) than I'm happy to investigate further, but it seems to work correctly everywhere I've tested so far.
    - IceWreck 5y ago
      > Any chance you've added a rule in HTTP Toolkit that's redirecting traffic back to itself. No, I literally downloaded it after seeing the post on HN and started a terminal with it for testing. Default settings. I'm using Fedora 35 if it helps. Then I started using that terminal for something else and thats when I realized go get was failing.

pimterry 5y ago

Thanks, that's super useful. > If you try to use go's package manager, example: `go get golang.org/x/oauth2` I just tested, and `go get golang.org/x/oauth2` seems to work fine for me, I can see all the requests being happily intercepted immediately: https://imgur.com/a/Cb1y9Q2 https://imgur.com/a/Cb1y9Q2 Can you see the 500 in HTTP Toolkit, and any more info there (in the body or as an error at the top) related to that? Or can you see a "certificate rejected" message? If nothing turns up there at all then yes, something must be overriding the proxy configuration. Maybe you have some other Go package manager configuration that conflicts with this? I'd be very interested to know about that if so, I'm sure there's others with the same thing. It's always very hard to know if my configuration is representative of normal devs for any given language/tool. Probably best to debug this outside of a HN thread though :-). You can file a proper issue about this at https://github.com/httptoolkit/httptoolkit/issues/new https://github.com/httptoolkit/httptoolkit/issues/new, I'd love to know what's going on there and get this fixed. > I ended up using https://github.com/hmgle/graftcp https://github.com/hmgle/graftcp which somehow manages to force tcp traffic through a socks5 proxy. Really interesting, thanks! I'll look into that.

IceWreck 5y ago
This is the error I see in the response body in HTTP Toolkit. ``` Error: Passthrough loop detected. This probably means you're sending a request directly to a passthrough endpoint, which is forwarding it to the target URL, which is a passthrough endpoint, which is forwarding it to the target URL, which is a passthrough endpoint... You should either explicitly mock a response for this URL (http://127.0.0.1:8000/golang.org/x/oauth2/@v/list http://127.0.0.1:8000/golang.org/x/oauth2/@v/list), or use the server as a proxy, instead of making requests to it directly. ```
- pimterry 5y ago
  That's really interesting! That's what appears if you HTTP Toolkit receives a request, proxies to the provided destination, and then finds itself on the other end of its own outgoing socket. This avoids the simplest infinite proxy loops. That'll make if you try to proxy a request to `127.0.0.1:8000` via HTTP Toolkit for example. That shouldn't be happening here though - there's no way Go should be using the proxy address as its target URL. I.e. to see this result, it must be sending a request like: GET /golang.org/x/oauth2/@v/list Host: 127.0.0.1:8000 You can probably see the full details in HTTP Toolkit's request data. Any chance you've added a rule in HTTP Toolkit that's redirecting traffic back to itself? That could do this, and would show up as two identical requests in HTTP Toolkit. If not (you can reset the rules by hitting the trash icon on the mock page) then this means something in your Go setup is either configured very strangely or its interpreting HTTP Toolkit's configuration incorrectly. Sorry I can't be more helpful, I have no idea why `go get` would send a request like that! If you can pin it down to something I can reproduce (e.g. a specific Go version or config setting or something) than I'm happy to investigate further, but it seems to work correctly everywhere I've tested so far.
  - IceWreck 5y ago
    > Any chance you've added a rule in HTTP Toolkit that's redirecting traffic back to itself. No, I literally downloaded it after seeing the post on HN and started a terminal with it for testing. Default settings. I'm using Fedora 35 if it helps. Then I started using that terminal for something else and thats when I realized go get was failing.