agallego

8 ms·

31.
by agallego 3y ago
When you add compaction, indexing, recovery, tiered storage, etc. some things become harder to reason about wrt systems resources if you are embedded.
32.
by agallego 3y ago
For long term storage I agree too. The reason we invented our byoc was so that (1) you own your storage and (2) we only charge you for value add
33.
by agallego 3y ago
We’re about to released a revamped wasm and new sdk with prev lessons learned. Should be cool
34.
by agallego 3y ago
alex here, original author of redpanda is hard to respond to a 6-part blog series content - released all at once - on an HN thread. - what we can deterministically show is data loss on apache kafka with no fsync() [shouldn't be a surpr
35.
by agallego 4y ago
I tried in 2017 writing it in rust and found some compiler bugs. I also found compiler bugs in c++ tho to be honest, but I felt more comfortable in c++ so decided to write the first version of it in c++. The huge advantage is that storage e
36.
by agallego 4y ago
this is a bad take. people work on problems they find interesting. it follows that you write about them because you can. the author has worked on kernel hypervisors, databases and now doing stuff around code generation. btw their code is ap
37.
by agallego 4y ago
we use it for redpanda https://github.com/redpanda-data/redpanda - other langs are go for k8s integration, python (dev prod tooling), ... js for ui that's mostly it.
38.
by agallego 4y ago
I think this is a good architecture to focus on the developer experience. A rust runtime with a Python front end. Very cool to see
39.
by agallego 4y ago
I’m not sure this is true. I’ve probably spoken with 500+ teams myself and by and large folks use default settings.
40.
by agallego 4y ago
kyle is very friendly and I recommend reaching out. we can't and wouldn't disclose any pricing that is not public information. would be unethical on my part. all i can say is we wish to continue our work with him indefinitely as l
41.
by agallego 4y ago
interested! :D
42.
by agallego 4y ago
hi newman314 - i mentioned in the tweet this was a mistake and offered an apology there, an sdr reached out to you, when i realized that i apologize. no ill intent. feel free to test this with a fake github account. my tweets automatically
43.
by agallego 4y ago
totally different approaches tho. people have tried what you proposed many times before and for some scale succeeded. hard to compare at all when you dig into the details. expect a companion post. this was super fun to partner with kyle on
44.
by agallego 5y ago
perf is important, but predictability of the SSD perf is honestly just (if not more) valuable, specially when you have to build SLA's for your products.
45.
by agallego 5y ago
a small note on redpanda. Is not a `rewrite`. It is an entire new storage engine, different consensus protocol, different on-disk format, different internal RPC, no external dependencies, ...., in fact when i started it it was using flatbuf
46.
by agallego 5y ago
Hehe Awesome. Good to hear :) For wasm we are collecting feedback so def comment on that GitHub thread. There is a blog post describing the API preview.
47.
by agallego 5y ago
yes sir. should just work w/ no changes. just change the docker img name to redpanda. let me know if anything comes up.
48.
by agallego 5y ago
this is cool. i wanted to try this w/ redpanda (i'm a maintainer https://github.com/redpanda-data/redpanda ), but I didn't see it using testcontainers or docker or anything like that.
49.
by agallego 5y ago
dunno what to say. the answer doesn't change.
50.
by agallego 5y ago
i got a notification in slack. there is less organization than you think :D. if we were, we'd have done this w/ the last 15 posts.
51.
by agallego 5y ago
I'm pretty excited to see the wasm chat too. very cool. I think that wasm is going to do to server side what javascript did for the web in the 90's
52.
by agallego 5y ago
I accidentally deleted the parent thread, when I meant to hit edit. but, my name is alex & I am the founder. happy to answer questions.
53.
by agallego 5y ago
Yeah, we went from low throughput to multi-gigabyte/second workloads, for example consuming the NYSE entire firehose+downstream analytics.
54.
by agallego 5y ago
for the record, you can do compaction with redpanda - all the same compaction strategies you know w/ kafka work w/ redpanda. https://github.com/vectorizedio/redpanda/tree/dev/src/v&#x2
55.
by agallego 5y ago
Happy to answer q’s on redpanda
56.
by agallego 6y ago
hi @sgt - yup SASL_SSL due to the client protos have to be compatible, including the hack you are thinking of. from a user perspective, existing SASL + SSL + SCRAM will be released next wednesday - so no code changes.
57.
by agallego 6y ago
that makes total sense. I agree. though in the future, the kafka proto will be impl details. i.e.: once you start moving to http proxy with typed schemas.
58.
by agallego 6y ago
From a feature list perspective on the readme it basically is what we did at Redpanda (dev here)
59.
by agallego 6y ago
oh odd. what setup to repro the js errors. i'll fix.
60.
by agallego 6y ago
redpanda is not just a performance (latency) gain. The main thing people pay us money for is not performance, it is operational simplicity - no zk, no jvm, nothing to tune ( https://vectorized.io/docs/autotune ) . one bi

More ›