agallego

11 ms·

1. 91.▲
   by agallego 6y ago

   thank you! :)
2. 92.▲
   by agallego 6y ago

   Sweet thanks! passing this to the designer.
3. 93.▲
   by agallego 6y ago

   correct. xfs is just what we test all the time.
4. 94.▲
   by agallego 6y ago

   i haven't tested myself. the kafka ecosystem is huge. but we should test.
5. 95.▲
   by agallego 6y ago

   not sure how you got here tho, maybe i miss explained something. This is talking about filesystem api not about kafka api. the kernel does something similar, standard filesystem things just purpose built for our use case.
6. 96.▲
   by agallego 6y ago

   great questions. (1) arrow is great! currently, it does not, but yes it will when we move out of nodejs impl into our own v8 isolates inside an alien thread (seastar concept) (2) stateful but only for a single partition (3) will be released
7. 97.▲
   by agallego 6y ago

   Haha. Too good.
8. 98.▲
   by agallego 6y ago

   Absolutely. If you use transactions, we don't support it yet. We are in active development here. Notice that folks love speed but the operational simplicity of having one binary/fault domain makes a lot of our enterprise users use
9. 99.▲
   by agallego 6y ago

   Its a bit deeper. Custom strategies for reading and writing have a huge impact. (orders of magnitude for specific workloads rather than generic page cache scheduling) Optimizing for fragmented buffers. Etc. See the LinkedIn vid from the blo
10. 100.▲
    by agallego 6y ago

    Sure. Gists are No use of page cache instead custom built read ahead and write behind strategy Io scheduling groups from seastar Custom intra cluster RPC to optimize for fragmented buffers native support (most storage work is small material
11. 101.▲
    by agallego 6y ago

    Indeed. See the segment appender work in the github with the chunk cache which sums up our write behind strategy. The reads are more complex since it involves understanding io priority scheduling etc
12. 102.▲
    by agallego 6y ago

    So many goodies to come. First we need to finish multi partition transactions that we don't yet support. We need to balance eager cluster balancing based on saturation (network, cpu, disk, etc) with other performance improvements. As i
13. 103.▲
    by agallego 6y ago

    Mostly for sparse file system support. We dispatch IO eagerly and out of order.
14. 104.▲
    by agallego 6y ago

    Why, you can run on a single core and 1G of ram. We just give you optimal prod settings for saturating boxes. See rpk container
15. 105.▲
    by agallego 6y ago

    Most kafka connectors just use the kafka API. So should "just work" (tm). Hehe. Let me know otherwise
16. 106.▲
    by agallego 6y ago

    Will do. Thank you. Do you have any thoughts here. How would you say it differently. We tried that on the landing page
17. 107.▲
    by agallego 6y ago

    Hi there! Sure thing. I basically was playing with an RPC framework to make my old company project fast in 2017 see github.com/smfrpc/smf I wanted to play with dpdk then. When I started this in 2019 I wanted to use a framework tha
18. 108.▲
    by agallego 6y ago

    Hi there. Of course you can make money using it. The main restriction is to not run redpanda as a service. Otherwise have fun. Its the same as cockroachdb in that regard
19. 109.▲
    by agallego 6y ago

    Let me know if you run into issues - vectorized.io/slack the entire eng team is there.
20. 110.▲
    by agallego 6y ago

    Checkout this _extremely_ detailed work that explains exactly that - https://vectorized.io/kafka-redpanda-availability/ To quote my colleague Denis - The results fit the theory. Raft’s (and Redpanda’s) performance is
21. 111.▲
    by agallego 6y ago

    I would say, building an scalable raft implementation has been by far the hardest thing. Mostly because we started from scratch and bypassed the page-cache with our read-ahead and write-behind strategies, etc. so building tiers of caches, e
22. 112.▲
    by agallego 6y ago

    also i'd note one more thing which is the WASM API is going to give us an opportunity to add more enterprise value. i.e.: think GDPR compliance as long as your data streams are in JSON or some predefined format. WASM basically allows u
23. 113.▲
    by agallego 6y ago

    We had success with very large customers (enterprise) and also some success with finance company (looking for 0 data loss). The idea is that we can give the kafka API compat system with 10x lower tail latencies for free and still monetize t
24. 114.▲
    by agallego 6y ago

    We kind of started on a different note. We wanted to solve some fundamental problems (no Zookeeper - similar to KIP-500) and most of all we wanted a single binary we can ship around, something that is easy to run. I think people LOVE the ka
25. 115.▲
    by agallego 6y ago

    I'm Alex, the founder. Happy to answer any questions.
26. 116.▲
    by agallego 6y ago

    hi wickerm - not sure if this applies to you - but we released an scholarship for this kind of thing for under represented groups ( https://vectorized.io/scholarship/ )
27. 117.▲
    by agallego 6y ago

    ceo here; you get: 1. 1500 bucks a month. 2. mentorship from ppl that have built very large systems before 3. hack on _any_ project you want Bonus: You keep all your intellectual property. we don't care. we just want to help under repr
28. 118.▲
    by agallego 6y ago

    Thank you! Will do! <3
29. 119.▲
    by agallego 6y ago

    very different, in just about every way. Type safety, deduced by the compiler instead of the stringly-typed %d and friends for starters.
30. 120.▲
    by agallego 6y ago

    thanks, though what we sell is operational simplicity. speed is nice, but not the main benefit. a single binary that's easy to run is what CIO seem to be interested. though we are young. fingers crossed it works :)