agallego

19 ms·

1. 121.▲
   by agallego 6y ago

   ah yeah no doubt. I've read the recent gc code in the jvm tree. it's beautiful. We just had a ton years of experience with c++ and it was a good fit for seastar.io - this all came from me messing around with https://git
2. 122.▲
   by agallego 6y ago

   who cares about lower percentage tiers in the world of big data? what does the p30 or 50 even matter at large scale? See Gil Tene's talk on how not to measure latency. https://www.youtube.com/watch?v=lJ8ydIuPFeU What m
3. 123.▲
   by agallego 6y ago

   I understand tail latencies - https://www.youtube.com/watch?v=WdFYY3vEcxo - my prev open source project smf ( https://github.com/smfrpc/smf ) uses gill tene's HDR histogram. I think you have a very
4. 124.▲
   by agallego 6y ago

   We wrote it in c++ on top of seastar.io
5. 125.▲
   by agallego 6y ago

   +1K on this. tail latencies are bad.
6. 126.▲
   by agallego 6y ago

   p99 latency. No startup times of course. This is for pushing a simple 2 petabyte workload. No AOT compilers, just download kafka bin distribution 2.4.1 Just launching 6 or 7 of these. bin/kafka-run-class.sh org.apache.kafka.tools.P
7. 127.▲
   by agallego 6y ago

   The JVM is fast. (though I'd say the best thing about the JVM is the amazingly large test suite it maintains). I think he was alluding to the fact it is much easier to develop predictable systems in a language that forces you to deal w
8. 128.▲
   by agallego 6y ago

   alex@vectorized.io - let's chat. i'm looking for a senior member atm.
9. 129.▲
   by agallego 6y ago

   correct. we use one protocol, just raft - both for metadata and for data. Raft is really easy to parallelize and dispatch to multiple followers async. I measured recently on 3 i3.8xlarge instances which give you 1.2GB/s - and i got aro
10. 130.▲
    by agallego 6y ago

    on it.
11. 131.▲
    by agallego 6y ago

    Thanks! Yeh I think the same, specially with the rise of core count. I think an i3.metal is at 96vcpus. I estimate a machine like that should hold around 400K partitions just fine with us. I should measure this weekend.
12. 132.▲
    by agallego 6y ago

    Yeah. Based on seastar. W.r.t OSS doing research on it atm. I had a VP at a major cloud say to me "we will try to use your tech" in person. So doing research before I make the decision
13. 133.▲
    by agallego 6y ago

    Agreed. We're working on this exactly - https://vectorized.io/redpanda/ 10x faster. API compat. No jvm. I also know of other private impls of it. Just makes sense.
14. 134.▲
    Google open sources updated tcmalloc(github.com)

    4 points by agallego 7y ago|0 comments
15. 135.▲
    by agallego 7y ago

    Here is a web developer pretending to understand a multi generational gap of power. The complexities of this armed conflict cannot be possibly explained in a line as rude as `colombians don't learn from their own mistakes`. For those
16. 136.▲
    by agallego 7y ago

    Note that you still have the same number of fault domains if you read the actual proposal. It does not simplify the system. It is simply replacing the ZK ensemble w/ its own raft impl. Same number of JVMs, same operational complexity.
17. 137.▲
    by agallego 8y ago

    great post!
18. 138.▲
    by agallego 8y ago

    cool project. However, the callback has an odd interface. why not pass a universal ref T&& vs a const T& most of your code is like this. ``` auto resp = HttpResponse::newHttpJsonResponse(json); callback(resp); ``` which me
19. 139.▲
    by agallego 8y ago

    i have a prototype of that same idea working w/ seastar.
20. 140.▲
    Intel’s implementation of C++17 parallel algorithms(github.com)

    4 points by agallego 8y ago|0 comments
21. 141.▲
    by agallego 8y ago

    abseil, folly and seastar are pretty great.
22. 142.▲
    by agallego 8y ago

    This has no c++ content. only C. the mistakes highlighted are just not representative of a team of C devs. the resource leak is specially a bad example because there better examples where go safety is a real benefit.
23. 143.▲
    by agallego 8y ago

    did you think of using the Go apache beam API ? any comparisons with oss alternatives?
24. 144.▲
    by agallego 8y ago

    ... i guess I was tired when i wrote it. I meant to write 29th! thanks!!
25. 145.▲
    by agallego 8y ago

    I see what you are saying. I did run it 100s of times, but didn't record it. Problem with benchmarking is, that it is incredibly slow haha. I'll run it for a few hours with a couple other tips ppl sent me and have a follow up on t
26. 146.▲
    by agallego 8y ago

    No no. - capped freqs: stddev 768x. in the non-turbo-to-turbo direction - in the other direction it is 2.3x. So was just acknowledging that there are 20% of samples in that table with your previous point
27. 147.▲
    by agallego 8y ago

    ~20%. The difference however is a max of ~2.3x in that direction. 100% of the code is there, what do you suggest is 'messed' up. I probably ran the benchmarks 100s of times over 7 days trying to understand. That particular I poste
28. 148.▲
    by agallego 8y ago

    Thanks for summarizing it better than I did. I'll edit and make that a bit more clear.
29. 149.▲
    by agallego 8y ago

    Thanks. That machine has a server processor and ecc memory 64GB. Specs are pretty similar to a small server. But yeah. Hardware has changed a ton :)
30. 150.▲
    by agallego 8y ago

    Very cool!