4 ms·

Sure, and I thought twice before putting that in :) We of course monitor our 95th & 99th primarily, for reasons that you said. But I didn't quote those because

by dalyons 11y ago

Sure, and I thought twice before putting that in :) We of course monitor our 95th & 99th primarily, for reasons that you said. But I didn't quote those because they're usually blown out by slow db queries, n+1 selects,n+1 cache gets, etc, issues that happen regardless of language. often they're on rarely used end points, where the cost-benefit of optimising isn't worth it. I feel median gives a resonable proxy for the language/framework execution capabilities.

jsmthrowaway 11y ago
That response actually illustrates my point, oddly enough; median has comforted you to organizationally disregard the super interesting data you're getting from the percentile aggregations. Those blowouts are far more interesting than you're saying, I wager. p99 blowout is generally an operational smell, even on low-RPS endpoints.
- bumbledraven 11y ago
  Grandparent's point is that 99th percentile latency is, for their application, unrelated to the ruby-vs-Go comparison.
- dalyons 11y ago
  I think we might be talking past each other a little :) The median has absolutely not "comforted" us, nor do we disregard the percentile data. In fact, we have a whole perf team basically dedicated to looking at 95/9th & other slow transaction trace data, and fixing these. In big projects, serving 10's of millions of users, with a handful of devs, you've gotta pick your battles, you cant fix everything always. We do our best, just like anyone else. So obviously we agree with the importance of percentiles. Dont make assumptions :) And all of this is rather irrelevant to the main point now, which is overall _language_ speed.
  - jsmthrowaway 11y ago
    It's not, actually, and I'm not sure why you think I'm talking past you. Your claim was that Ruby is comparable performance-wise to Go in your application, and you cited median latency to make that case. I pointed out that interpreted languages often have a long tail and that the long tail is more interesting and undermines the comparison. You replied that I am correct and, indeed, you do have a long tail but you wrote it off as "stuff that happens" regardless of language. I am replying that the "stuff that happens" is the interesting stuff that undermines the flawed comparison you attempted to make, and I disagree that all of it can be written off to language-independent concerns; if you investigate it further, you might find some of it comes from the choice of language itself. I have not wandered off topic into irrelevancy nor talked past you. This is still addressing the point you attempted to make by citing median latency in defense of the performance characteristics of your chosen language, in order to assuage general hesitance to adopt Ruby over performance concerns. I think you'll find that your metrics do not support your data if you dig into that p99. (I share that experience from handling tens of millions of users myself, which is all I can admit to publicly, so that request re: assumptions goes both ways.) To recap: claim submitted with flawed supporting data, thread addressing why the supporting data is flawed. No talking past taking place.
    - dalyons 11y ago
      Never claimed ruby was as fast as go. Our go services run at about 5-15ms(excluding 95/99 etc), so they're def faster(they do a lot less too!). I'm just saying ruby (or other dynamic langs, i dont mind, I dont have a "chosen language") can make a decently fast product API, and thats all. And of course we've investigated further... (this is anecdotal i know), but of all the degenerate 99th cases we've investigated/solved, I can only think of 2 or 3 that were lang related, rather than logic, db, n+1s, etc. If you're curious, those were: - degenerate GC behaviour in a specific circumstance - JSON serialization (specifically crossing the C/ruby boundary shiteloads of times to serialize an object) Sure these issues sucked, but we mitigate/sidestepped those issues for the most part. It's all a trade off when you're trying to decide what to use for your project; right tool for the job etc. Im not evalganizing for or against anything. Curious what would you propose instead to roughly compare speeds of a product API? (Keeping in mind my anec-data about nearly all of 99ths being a logic/db issue). Comparing 99ths (from my experience) would be more like comparing which codebase has more bugs, because thats how we treat degenerate cases.

by dalyons 11y ago

jsmthrowaway 11y ago

That response actually illustrates my point, oddly enough; median has comforted you to organizationally disregard the super interesting data you're getting from the percentile aggregations. Those blowouts are far more interesting than you're saying, I wager. p99 blowout is generally an operational smell, even on low-RPS endpoints.

bumbledraven 11y ago
Grandparent's point is that 99th percentile latency is, for their application, unrelated to the ruby-vs-Go comparison.
dalyons 11y ago
I think we might be talking past each other a little :) The median has absolutely not "comforted" us, nor do we disregard the percentile data. In fact, we have a whole perf team basically dedicated to looking at 95/9th & other slow transaction trace data, and fixing these. In big projects, serving 10's of millions of users, with a handful of devs, you've gotta pick your battles, you cant fix everything always. We do our best, just like anyone else. So obviously we agree with the importance of percentiles. Dont make assumptions :) And all of this is rather irrelevant to the main point now, which is overall _language_ speed.
- jsmthrowaway 11y ago
  It's not, actually, and I'm not sure why you think I'm talking past you. Your claim was that Ruby is comparable performance-wise to Go in your application, and you cited median latency to make that case. I pointed out that interpreted languages often have a long tail and that the long tail is more interesting and undermines the comparison. You replied that I am correct and, indeed, you do have a long tail but you wrote it off as "stuff that happens" regardless of language. I am replying that the "stuff that happens" is the interesting stuff that undermines the flawed comparison you attempted to make, and I disagree that all of it can be written off to language-independent concerns; if you investigate it further, you might find some of it comes from the choice of language itself. I have not wandered off topic into irrelevancy nor talked past you. This is still addressing the point you attempted to make by citing median latency in defense of the performance characteristics of your chosen language, in order to assuage general hesitance to adopt Ruby over performance concerns. I think you'll find that your metrics do not support your data if you dig into that p99. (I share that experience from handling tens of millions of users myself, which is all I can admit to publicly, so that request re: assumptions goes both ways.) To recap: claim submitted with flawed supporting data, thread addressing why the supporting data is flawed. No talking past taking place.
  - dalyons 11y ago
    Never claimed ruby was as fast as go. Our go services run at about 5-15ms(excluding 95/99 etc), so they're def faster(they do a lot less too!). I'm just saying ruby (or other dynamic langs, i dont mind, I dont have a "chosen language") can make a decently fast product API, and thats all. And of course we've investigated further... (this is anecdotal i know), but of all the degenerate 99th cases we've investigated/solved, I can only think of 2 or 3 that were lang related, rather than logic, db, n+1s, etc. If you're curious, those were: - degenerate GC behaviour in a specific circumstance - JSON serialization (specifically crossing the C/ruby boundary shiteloads of times to serialize an object) Sure these issues sucked, but we mitigate/sidestepped those issues for the most part. It's all a trade off when you're trying to decide what to use for your project; right tool for the job etc. Im not evalganizing for or against anything. Curious what would you propose instead to roughly compare speeds of a product API? (Keeping in mind my anec-data about nearly all of 99ths being a logic/db issue). Comparing 99ths (from my experience) would be more like comparing which codebase has more bugs, because thats how we treat degenerate cases.