4 ms·

I think this is very true. The most vexing problems I've run into lately have all been because something is going wrong in a step I don't have much control ove

I think this is very true. The most vexing problems I've run into lately have all been because something is going wrong in a step I don't have much control over. Like in one case the proxy was stripping out the Authorization header for PUT requests (seriously). Or in another the corporate-signed certs used by all the on-prem systems weren't validating on AWS because they don't have a valid root cert. I really wish I had a "debugger" that could trace the interconnections between all the systems my app depends on.

spockz 5y ago
In both cases you would have been helped by proper error reporting and by tracing. (See OpenTelemetry) For the first the remote server could say there is no Authorization header where tracing confirms it was there when sent by the client but not by the proxy. For the second it would be really nice to see more detailed logging on the server side about why a particular peer certificate was rejected. My biggest gripe with current OpenSSL and Java TLS implementations is this lack of detailed reporting.
- wilburTheDog 5y ago
  In the first case the request worked when executed in Postman but failed when executed from a browser, which said it was a CORS error. Using the logs we traced it up to the authorization step in APIG, which wasn't being executed (because auth header was required to execute it) and so wasn't generating logs. And that's where I was stumped, because it was not a part of my mental model that request headers could disappear between client and APIG. We eventually realized the browser uses the proxy and postman doesn't, so we asked the networking group and they admitted they were removing the auth header in the proxy. Regardless of the details this is one of those situations I wasn't able to solve by just thinking about it. If I had a 'debugger' that could step through these API requests it would have been much easier to figure this one out. I don't know enough about OpenTelemetry to say if it would have helped or not.

spockz 5y ago

In both cases you would have been helped by proper error reporting and by tracing. (See OpenTelemetry) For the first the remote server could say there is no Authorization header where tracing confirms it was there when sent by the client but not by the proxy. For the second it would be really nice to see more detailed logging on the server side about why a particular peer certificate was rejected. My biggest gripe with current OpenSSL and Java TLS implementations is this lack of detailed reporting.

wilburTheDog 5y ago
In the first case the request worked when executed in Postman but failed when executed from a browser, which said it was a CORS error. Using the logs we traced it up to the authorization step in APIG, which wasn't being executed (because auth header was required to execute it) and so wasn't generating logs. And that's where I was stumped, because it was not a part of my mental model that request headers could disappear between client and APIG. We eventually realized the browser uses the proxy and postman doesn't, so we asked the networking group and they admitted they were removing the auth header in the proxy. Regardless of the details this is one of those situations I wasn't able to solve by just thinking about it. If I had a 'debugger' that could step through these API requests it would have been much easier to figure this one out. I don't know enough about OpenTelemetry to say if it would have helped or not.