apurvamehta

5 ms·

1.
OpenData – Open-Source and Object Store Native Databases(opendata.dev)
1 points by apurvamehta 3mo ago|0 comments
2.
Kafka's Broken Promise: There Is No Goldilocks Log(opendata.dev)
6 points by apurvamehta 3mo ago|0 comments
3.
Our first customers were the exception(apurvamehta.com)
3 points by apurvamehta 4mo ago|0 comments
4.
by apurvamehta 4mo ago
It's a demonstration. Kafka is a very popular input for Clickhouse. So are data lakes. The point is neither need Kafka at all.
5.
Ingesting 1Gbps of logs into ClickHouse for $180/month(opendata.dev)
18 points by apurvamehta 4mo ago|3 comments
6.
by apurvamehta 5mo ago
Thanks! opendata contributor here. We're heavily inspired by Turbopuffer. I'd say we are comparable to them when they launched in terms of perf and scale. But they've obviously invested heavily since then, so we're not g
7.
OpenData Vector: MIT-Licensed Vector Search on Object Storage(opendata.dev)
48 points by apurvamehta 5mo ago|5 comments
8.
OpenData Buffer: HA pipelines without Kafka(opendata.dev)
13 points by apurvamehta 5mo ago|1 comments
9.
by apurvamehta 6mo ago
+1 to what @agavra said. It's awesome to see you here @valyala. Your writing and talks about timeseries databases were a great inspriratino for us. I recall one of your earlier talks about the data layout design of VM. Opendata Timeser
10.
by apurvamehta 6mo ago
yes. this is current issue. there are two solutions: 1. the reason it's slow as you select more series over longer periods of time is that the series has to be pulled for each time bucket in the range, and then the samples have to be p
11.
by apurvamehta 6mo ago
Agreed. VictoriaMetrics is indeed a very compelling offering. The disk-less approach is significantly simpler to operate, which I think is the biggest difference. Running opendata's version yourself has fewer moving pieces, and standar
12.
by apurvamehta 6mo ago
Good call out, updated the intro with a summary of the cost benefit. Thanks for the feedback!
13.
OpenData Timeseries: Prometheus-compatible metrics on object storage(opendata.dev)
18 points by apurvamehta 6mo ago|15 comments
14.
Sorted string tables (SST) from first principles(bitsxpages.com)
71 points by apurvamehta 9mo ago|11 comments
15.
Four Horsemen: Lessons from Distributed System Apocalypse(responsive.dev)
3 points by apurvamehta 2y ago|0 comments
16.
The Why, When, and How of Kafka Applications(responsive.dev)
1 points by apurvamehta 2y ago|0 comments
17.
Founder's Letter: Responsive and the Future of Stream Processing(responsive.dev)
4 points by apurvamehta 2y ago|0 comments
18.
by apurvamehta 9y ago
From the very same blog post: > Is this Magical Pixie Dust I can sprinkle on my application? > No, not quite. Exactly-once processing is an end-to-end guarantee and the application has to be designed to not violate the property as we
19.
by apurvamehta 9y ago
Personally, I would welcome Aphyr trying to break the Kafka EOS guarantees. Like all software, there will be bugs, and the exposure will only make it stronger and more viable. I write as one of the implementors of exactly once guarantees in
20.
by apurvamehta 9y ago
Points 1 & 2. There is no direct communication between a producer and consumer. The producer writes to the broker, the consumer reads from the broker. There is a detailed flow diagram for the producer side operations in the article, and
21.
by apurvamehta 13y ago
We did our experiments directly on hardware. I don't think that AWS VMs simulate multiple physical sockets. If they don't then this article will not apply to them.
22.
by apurvamehta 13y ago
Wow.. that's great to know. We will definitely investigate this approach. Thanks for sharing! :)
23.
by apurvamehta 13y ago
That would be true if we were using C++. Unfortunately, all our code is in Scala and we use Java NIO libraries to memory map our files. AFAIK, they don't give us the option on using these POSIX calls.
24.
by apurvamehta 13y ago
Yes. It was a blunder. The post has been updated to reflect this.
25.
by apurvamehta 13y ago
Yikes. I meant that they dropped TO 1/4th the original.
26.
by apurvamehta 13y ago
Thanks, the 400% number is wrong. It was a last minute edit.. I should learn not to do that. I have updated the post to say that the error rates have dropped by 1/4th.
27.
by apurvamehta 13y ago
This is exactly right :)
28.
by apurvamehta 13y ago
Hi, post author here. > Also, is there a reason not to use large pages directly for the mmap'd sets if you know you're going to have them hot at all times? (I assume they read the entire file on start?) We could use large pages
29.
by apurvamehta 14y ago
> $1,200/month or less gets you your own private room in a shared apartment in NYC, Boston, or just about anywhere else. $1400/month gets a one-bed apartment in Mountain View. I used to pay $700/mo for a private room and bathroom in a
30.
by apurvamehta 15y ago
I feel like this is overkill. I have been using RailsReady ( https://github.com/joshfng/railsready ) to setup several Mac and Ubuntu boxes and it has never been more than one click.

More ›