exAspArk

7 ms·

31.
by exAspArk 2y ago
ClickHouse is definitely a popular choice nowadays. I'm curious whether you self-host ClickHouse or use their Cloud? We wanted to make BemiDB as simple to run as possible with a single binary and object storage (vs large machines, big
32.
by exAspArk 2y ago
Fair point. We think that BemiDB currently can be useful when used with small and medium Postgres databases. Running complex analytics queries on Postgres can work, but it usually requires tuning it and adding indexes tailored to these quer
33.
by exAspArk 2y ago
We love ParadeDB and their team. Their primary focus is search (Elasticsearch on Postgres), but they also have the pg_analytics Postgres extension (foreign data wrappers and embedded DuckDB). The biggest difference is in a Postgres extensio
34.
by exAspArk 2y ago
^ This! Here is the link that briefly describes pros and cons of different alternatives for analytics https://github.com/BemiHQ/BemiDB#alternatives
35.
by exAspArk 2y ago
The most common approach is to read Postgres data in DuckDB https://duckdb.org/docs/extensions/postgres.html
36.
by exAspArk 2y ago
Thanks! We love the pg_moooncake extension (and pg_duckdb used under the hood). Although our approaches are slightly different. Long-term, we want to allow anyone to use BemiDB by using native Postgres logical replication without installing
37.
by exAspArk 2y ago
Thank you! The pg_analytics Postgres extension partially supports different file formats. We bet big on Iceberg open table format, which uses Parquet data files under the hood. Our initial approach is to do periodic full table resyncing. Th
38.
by exAspArk 2y ago
I agree that DuckDB may sometimes be buggy because it's being actively developed with a large surface area (a universal embeddable query engine that works with different storage layers if I were to simplify). However, DuckDB (non-profi
39.
by exAspArk 2y ago
Yes, we want to use the approach like you described! We'll likely wait until enough changes are accumulated by using 2 configurable thresholds: time (like 30s) and size (like 100MB)
40.
by exAspArk 2y ago
We ran some benchmarks (TPC-H, designed for OLAP) with ~10M records https://github.com/BemiHQ/BemiDB#benchmark The BemiDB storage layer produced ~300MB columnar Parquet files (with ZSTD compression) vs 1.6GB of data in
41.
by exAspArk 2y ago
For now, BemiDB supports only full Postgres table data re-sync. We plan to enable real-time data syncing from Postgres into S3 by using logical replication (CDC), which is much more reliable than PG NOTIFY. We use logical replication and th
42.
by exAspArk 2y ago
Thank you! Yes, absolutely! 1) You could use BemiDB to sync your Postgres data (e.g., partition time-series tables) to S3 in Iceberg format. Iceberg is essentially a "table" abstraction on top of columnar Parquet data files with a
43.
Show HN: BemiDB – Postgres read replica optimized for analytics(github.com)
209 points by exAspArk 2y ago|117 comments
44.
It's Time to Rethink Event Sourcing(blog.bemi.io)
4 points by exAspArk 2y ago|0 comments
45.
Choosing the Right Audit Trail Approach in Ruby(blog.bemi.io)
46 points by exAspArk 2y ago|7 comments
46.
How Change Data Capture Powers Modern Apps(blog.bemi.io)
2 points by exAspArk 2y ago|0 comments
47.
The Ultimate Guide to PostgreSQL Data Change Tracking(exaspark.medium.com)
3 points by exAspArk 3y ago|0 comments
48.
Bemi: Automatic PostgreSQL Audit Trail(bemi.io)
2 points by exAspArk 3y ago|0 comments
49.
Show HN: Bemi – context-aware data change tracking for Prisma(github.com)
3 points by exAspArk 3y ago|0 comments
50.
Show HN: Bemi for TypeORM – automatic data change tracking(github.com)
4 points by exAspArk 3y ago|0 comments
51.
Show HN: Bemi – data versioning and time travel for PostgreSQL(bemi.io)
14 points by exAspArk 3y ago|1 comments
52.
Show HN: Bemi, enabling Event Sourcing for any database(bemi.io)
6 points by exAspArk 3y ago|0 comments
53.
Forking AngelList(angellist.com)
2 points by exAspArk 4y ago|0 comments
54.
Top Ways to Implement Real-Time Rich Text Editor (Ranked by Complexity)(exaspark.medium.com)
3 points by exAspArk 6y ago|0 comments
55.
by exAspArk 7y ago
Hyre | Full-Stack Developer | Toronto, Canada | ONSITE, REMOTE Canada Hyre is a marketplace that connects event organizers such as hotels and venues with event staff such as wait and bar staff. Essentially, an Uber-like model for the $80B+
56.
by exAspArk 7y ago
Hyre | Full-Stack Developer | Toronto, Canada | ONSITE, VISA Hyre is a marketplace that connects event organizers such as hotels and venues with event staff such as wait and bar staff. Essentially, an Uber-like model for the $80B+ event sta
57.
by exAspArk 7y ago
Hyre | Full-Stack Developer | Toronto, Canada | REMOTE, ONSITE, VISA Hyre is a marketplace that connects event organizers such as hotels and venues with event staff such as wait and bar staff. Essentially, an Uber-like model for the $80B+ e
58.
by exAspArk 7y ago
We started by building some new functionality with Elixir. For example, a new read-only GraphQL API. Later, we might begin performing mutations and writes, serving all HTTP requests with Phoenix, and handling asynchronous jobs. Although Eli
59.
by exAspArk 7y ago
Hyre | Full-Stack Developer | Toronto, Canada | REMOTE, ONSITE, VISA Hyre is a marketplace that helps connect event organizers (hotel chains such as Marriott and Hilton, exhibition centers, etc.) with event staff (supervisors, bartenders, w
60.
Improving Browser Performance 10x(engineering.universe.com)
10 points by exAspArk 7y ago|0 comments

More ›