11 ms·
AWS X-Ray – Distributed Tracing System
- koolba 10y agoSo is this AWS's New Relic competitor?
- mmcclellan 10y agothat was my thought. Though no Python yet: > AWS X-Ray supports tracing for applications that are written in Node.js, Java, and .NET.
- Rapzid 10y agoAnd no Ruby. Edit: The blog post mentions they will be adding this to the other SDK's.
- wsh91 10y agoWell, there is Stackdriver Trace for free already on Google Cloud Platform. (Disclosure: I work on GCP, albeit not on that team.)
- brazzledazzle 10y agoAny idea how they compare?
- tonyhb 10y agoAnd Stackdriver is Zipkin compatible, which is awesome. X-Ray doesn't seem to support Zipkin nor Opentrace which is limiting.
- ranman 10y agoThis blog post has more info: https://aws.amazon.com/blogs/aws/aws-x-ray-see-inside-of-your-distributed-application/ https://aws.amazon.com/blogs/aws/aws-x-ray-see-inside-of-you... I'll be talking about this on the https://twitch.tv/aws https://twitch.tv/aws stream at 12:30 pacific if you guys want to ask questions / learn more. (I WORK AT AWS)
- simook 10y ago^ Confirmed
- Trisell 10y agofixed!
- DenisM 10y agoIs X-Ray suitable to debug individual incidents, tracing a path of individual user, or only to get an overall view of where the time is being spent?
- thekemkid 10y agoIs "black box" tracing available, and feasible for production systems without a large perf impact?
- jmacfan 10y agoK-Ray? I think it's supposed to be X-Ray. Great attention to detail
- grandalf 10y agoI know someone who goes by the nickname k-ray
- Thaxll 10y agohttps://aws.amazon.com/xray/details/ https://aws.amazon.com/xray/details/
- btashton 10y agoNo Python support?
- EugeneOZ 10y agoNo Rust support, what is even more unexpected.
- allengeorge 10y agoWhy is that unexpected? As much as I love Rust - it's not exactly a top-10 language right now. If there's no Python support - that shocks me.
- EugeneOZ 10y agoIt's the language #1.
- Xorlev 10y agoHooray, hosted Zipkin. Would be really nice to have a opentracing backend that sends to AWS X-Ray.
- pritianka 10y agoAn OpenTracing <> Aws X-Ray integration would rock. I work on the project and would love to collaborate with anyone interested.
- andrewvc 10y agoI haven't used many of the new AWS services. Someone tell me, what's the quality level? I'll be surprised if all the new AWS stuff works that well given how divided their focus must be now.
- ranman 10y ago(I WORK FOR AWS) Each service has it's own dedicated service team. I've used some (not all) of the new services and I believe there is room to improve but I didn't think they were unstable or anything like that.
- brazzledazzle 10y agoI know there's no obligation to invest in services that don't make money, so please don't take this as some kind of expectation of that. A dedicated team doesn't always mean there's an equal distribution of quality and innovation. You can have a dedicated team that's so small they only have time to do bug fixes or a team with less talent/experience because everyone wants to move to teams working on the projects with the most organizational momentum.
- ranman 10y agoTBH I think you have a point there: the hot new thing is serverless and lambda so everyone I know really wants to work on those teams. I see where you're coming from for sure. There are still a lot of talented and passionate people at Amazon who want to execute on other products. The S3 team is still growing 10 years later and they're still doing (IMO) cool stuff. I'm a devangelist though so I don't always get to see the inside of every team (not enough time). I do know that we all drink the customer focused kool-aide and we genuinely believe in it. Sorry if this all sounds like marketing.
- brazzledazzle 10y agoIt doesn't sound like marketing to me and I appreciate your response. You could have just as easily ignored my comment.
- ropiku 10y agoLooks like it doesn't support all languages. Not sure if users can contribute plugins. > You can use X-Ray with applications written in Java, Node.js, and .NET that are deployed on these services. Support for AWS Lambda is coming soon.
- henrygrew 10y agoI was expecting python support
- devbug 10y agoIf you're interested in distributed system tracing there's a lot going on. As a starting point, I would recommend reading Google's paper on their project "Dapper." [1] It's essentially the core of most distributed tracing systems. At least those I've encountered. There's a lot of tooling out there that take their cues from Dapper. I've recently been looking into integrating OpenZipkin[2] with our systems. I see at as a more viable alternative (no tie in!) to Yet Another Propriety Thing in AWS (YAPTA). There's other as well, like AppDash. Recently there has been a push towards an open-standard for the collection side called OpenTracing[3]. I came across it when investigating LightStep[4]. Ideally that means no vendor lock-in, which of course has lots of knock-on effects. If you know of any, I'd love to be pointed in the direction of _different_ and not just divergent techniques. [1]: http://static.googleusercontent.com/media/research.google.com/en//archive/papers/dapper-2010-1.pdf http://static.googleusercontent.com/media/research.google.co... [2]: http://zipkin.io http://zipkin.io [3]: http://opentracing.io http://opentracing.io [4]: http://lightstep.com http://lightstep.com - Impressive team behind this.
- Roritharr 10y agoThose are the comments I come here for. I was reading this and dreaded the lock-in. Thanks for pointing out sane alternatives!
- tonyarkles 10y agoMy masters thesis is X-trace derived, as well as a paper we published at ICPE a few years ago. I'm on mobile so don't have a link handy. You can probably guess my name from my username.
- benzewdu 10y agohttps://ecommons.usask.ca/xmlui/bitstream/handle/10388/ETD-2015-06-2185/ARKLES-THESIS.pdf https://ecommons.usask.ca/xmlui/bitstream/handle/10388/ETD-2...
- SEJeff 10y agoAppdash[5] :) [5] https://github.com/sourcegraph/appdash https://github.com/sourcegraph/appdash
- res0nat0r 10y agoIs this comparable to Dynatrace?
- the_aLgo 10y agoNo, X-Ray is no APM solution (yet). For example, it is missing End-User Experience Monitoring (for more information see http://www.apmdigest.com/gartners-5-dimensions-of-apm http://www.apmdigest.com/gartners-5-dimensions-of-apm). The similarity to Dynatrace is that - like the PurePath - X-Ray enables distributed tracing. But the approach is different: whereas Dynatrace instruments your application using an agent^, X-Ray needs you to instrument an application using the provided SDK. Moreover, X-Ray seems to be limited to applications communicating via HTTP(S). ^ in the most cases
- xrd 10y agoIncredible. My book talks about writing tests to help build reliable systems but having a dynamic run time tool like this is an astounding addition to a developer's toolset.
- zelon88 10y agoSo how much did AWS have to pay to buy the entire front page of HN today?
- heartbreak 10y agoSome context is that today is one of the days for Amazon re:Invent 2016. https://reinvent.awsevents.com https://reinvent.awsevents.com
- pritianka 10y agoLol yeah, I think it's more about it being Re:invent. FWIW, I've been receiving a lot of pings on this stuff since I work on OpenTracing. So I think it's believable that this reached the top on its own.
- simlevesque 10y agoJust when I finished setting up NewRelic on Elastic BeanStalk :P
- deepnotderp 10y agoDamn, AWS is on a tear RN.
- polskibus 10y agoIs this offering similar to Azure's Application Insights? https://azure.microsoft.com/pl-pl/services/application-insights/ https://azure.microsoft.com/pl-pl/services/application-insig...
- honkhonkpants 10y agoHrmm name collision with Google's previously published XRay function tracing system http://research.google.com/pubs/pub45287.html http://research.google.com/pubs/pub45287.html
- Zikes 10y agoSame for AWS Shield & Google's Project Shield.
- dboreham 10y agoUgh. Of course I read the comments first to see if the article is worth reading and it takes me 10 minutes to realize this is not Amazon's 3D rendering as a service, service..
- loganbertram 10y agoHonest question: why is there so much AWS news up today? I mean, X-Ray is particularly exciting for me, but there are currently 5/30 stories on the front page which are basically just product announcements. I'm pretty new, but is this normal? I thought the basic upvote criteria was meant to be that we should focus on upvoting articles of some depth as well as interest.
- winthrowe 10y agoAWS major conference/product showcase is going on this week. https://reinvent.awsevents.com/ https://reinvent.awsevents.com/ Google, Apple, Microsoft, and Amazon each dominate the frontpage for several days when their respective developer conferences are ongoing or just completed.
- seanp2k2 10y agoAWS is having their developer conference this week: https://reinvent.awsevents.com https://reinvent.awsevents.com
- iamed2 10y agoThere's a big Amazon event (AWS re:Invent) and they are announcing an extraordinary number of new tools. Think an Apple or Google event but one where they announce a new product every 15 minutes. People with many different kinds of jobs are excited for different reasons. It's double-Christmas for people who work extensively with AWS.
- deleted 10y ago[deleted]
- ShakataGaNai 10y agoToday is the second day of AWS Re:Invent. So just like when Apple releases products and the site is slammed with Apple news, these two days are Amazon days. Don't worry, tomorrow will be back to normal.
- inthewoods 10y agoSignificant blow to all APM players in the space. Approximately 50% of New Relic's revenue comes from AWS. Datadog just announced their distributed tracing system at $25/host. X-Ray is orders of magnitude cheaper than either - and per trace pricing is something that would be very tough for anyone else to do and plays well to Lambda efforts. X-Ray seems like a pretty basic service at this point but in my opinion the writing is on the wall for other APM providers. Amazon chose this announcement for a major slot so I expect they'll be investing in the service. As Bezos is often quoted as saying "your margin is my opportunity."
- DenisM 10y agoTurns out X-Ray is sampling service, it drops data. You can use it to debug recurring problems, but it's no use when debugging a particular incident with the customer on the phone. Bummer. To provide a performant and cost-effective experience, X-Ray does not collect data for every request that is sent to an application. Instead, it collects data for a statistically significant number of requests. X-Ray should not be used as an audit or compliance tool because it does not guarantee data completeness. https://aws.amazon.com/xray/faqs/ https://aws.amazon.com/xray/faqs/
- dkuebric 10y agoYeah, there's a big difference between something that's pure distributed tracing (this) and something that does metrics/monitoring as well (TraceView, New Relic, ...). Sampling is a valid way to make distributed tracing scalable and performant, but it can also limit the use-cases for the data. I imagine this being used for episodic debug cases, where one could turn up the sample rate and pay only for the traces captured/queried during an incident. But you would need to do your monitoring and trending separately. I wouldn't be surprised if they eventually start integrating this better with CloudWatch for that reason, though it doesn't seem to be doing any of that today. Disclosure: I work on TraceView (traceview.solarwinds.com) which is distributed tracing based APM product. We're inspired by Google Dapper and x-trace, both mentioned elsewhere in this thread.
- DenisM 10y ago> difference between something that's pure distributed tracing (this) and something that does metrics/monitoring as well (TraceView, New Relic, ...) I don't see the distinction you're making, as neither seems to record all traces for accurate audit. Am I missing it?
- dkuebric 10y agoI was speaking to a more general monitoring approach where you might want to know p99 latency, request volume, error rate, etc, for each service to use in alerting and trending. This is a common use-case for application monitoring that isn't addressed well by a pure tracing approach. If you're looking specifically for the 100% audit trail case, I'd look at DynaTrace or potentially Instana.
- spo81rty 10y agoSome have mentioned is this full APM and does this compete with New Relic, Dynatrace, Stackify, Appdynamics, App Insights, etc. Those products are primarily based on code profiling. For example, at Stackify we automatically profile key methods for dozens of common dependencies and frameworks to understand their usage and performance. Every SQL, NoSQL, caching, queuing providers and many other things. Plus app errors, logs, etc. So the best I can tell from the AWS blog and docs is the answer is no its not a full APM. It appears to track how long a web request takes and any usage of the AWS services via their SDK. More of a lightweight service mapping of AWS services. So SQL database or HTTP calls probably aren't tracked. In the future could they expand it? Sure. But for now it seems limited compared to a full blown APM product. Although this could be help for identifying performance problems with AWS services. Matt - Founder of Stackify