4 ms·

Using the “X-AMZN-Trace-Id” Header for Request Tracing Through AWS Load Balancer

82 points by rmason 5y ago

Spivak 5y ago
I think this the AWS docs explain this fairly well. https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-request-tracing.html https://docs.aws.amazon.com/elasticloadbalancing/latest/appl... The format for the field is X-Amzn-Trace-Id: Field=version-time-id and you're allowed to add custom fields that don't follow this format and they will be preserved.
booleanbetrayal 5y ago
I recently implemented support for this in the prometheus-exporter gem. https://github.com/discourse/prometheus_exporter/pull/150 https://github.com/discourse/prometheus_exporter/pull/150 What's annoying about the encoded time value is version 1 (the only version still) is in seconds precision. Makes the value non-viable for any sort of queue time metric. Better than nothing, but would have been nice if it were micro.
wdb 5y ago
Have a look at this propagator from Opentelemetry: https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/propagators/aws/xray/propagator.go https://github.com/open-telemetry/opentelemetry-go-contrib/b... If you extract this trace id and use it as your request id you can follow it from the load balancer all the way to the end.
kortex 5y ago
I wonder if the system propagates "baggage" headers. OpenTelemetry uses this header to propagate span/trace IDs. https://www.w3.org/TR/baggage/ https://www.w3.org/TR/baggage/
guidoism 5y ago
Request ID! I added that to the Amazon web servers about 20 years ago! My code probably doesn’t exist anymore but it’s funny to see it referenced on HN this many years later.
- an_account_name 5y ago
  If it’s in APLCore it’s still running somewhere, you can be sure
  - guidoism 5y ago
    Oh man, I gotta lot of code in APLCore. I can't believe that's still around too. Wow.
uhhhhhhhhhhhhhh 5y ago
Wasnt there an rfc recommending against the use of X- prefix headers? I guess that didnt hold
- OJFord 5y ago
  Yes, 'deprecated' in RFC6648 (2012): https://datatracker.ietf.org/doc/html/rfc6648 https://datatracker.ietf.org/doc/html/rfc6648 `X-AMZN-something` at least though probably predates that; a mix of `X-AMZN` and `AMZN` would be annoying. (Although I suppose they could say 'We've deprecated `X-`, it'll be stripped and still work, but please don't use it in new code' applying to all of them, old and new.)

82 points by rmason 5y ago

I think this the AWS docs explain this fairly well. https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-request-tracing.html https://docs.aws.amazon.com/elasticloadbalancing/latest/appl... The format for the field is X-Amzn-Trace-Id: Field=version-time-id and you're allowed to add custom fields that don't follow this format and they will be preserved.

booleanbetrayal 5y ago

I recently implemented support for this in the prometheus-exporter gem. https://github.com/discourse/prometheus_exporter/pull/150 https://github.com/discourse/prometheus_exporter/pull/150 What's annoying about the encoded time value is version 1 (the only version still) is in seconds precision. Makes the value non-viable for any sort of queue time metric. Better than nothing, but would have been nice if it were micro.

wdb 5y ago

Have a look at this propagator from Opentelemetry: https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/propagators/aws/xray/propagator.go https://github.com/open-telemetry/opentelemetry-go-contrib/b... If you extract this trace id and use it as your request id you can follow it from the load balancer all the way to the end.

kortex 5y ago

I wonder if the system propagates "baggage" headers. OpenTelemetry uses this header to propagate span/trace IDs. https://www.w3.org/TR/baggage/ https://www.w3.org/TR/baggage/

guidoism 5y ago

Request ID! I added that to the Amazon web servers about 20 years ago! My code probably doesn’t exist anymore but it’s funny to see it referenced on HN this many years later.

an_account_name 5y ago
If it’s in APLCore it’s still running somewhere, you can be sure
- guidoism 5y ago
  Oh man, I gotta lot of code in APLCore. I can't believe that's still around too. Wow.

uhhhhhhhhhhhhhh 5y ago

Wasnt there an rfc recommending against the use of X- prefix headers? I guess that didnt hold

OJFord 5y ago
Yes, 'deprecated' in RFC6648 (2012): https://datatracker.ietf.org/doc/html/rfc6648 https://datatracker.ietf.org/doc/html/rfc6648 `X-AMZN-something` at least though probably predates that; a mix of `X-AMZN` and `AMZN` would be annoying. (Although I suppose they could say 'We've deprecated `X-`, it'll be stripped and still work, but please don't use it in new code' applying to all of them, old and new.)