3 ms·
I wish there was some API that abstracted the DB and all technical details and you could connect nodes to it that are specific databases with specific capabilit
by endisneigh 5y ago
I wish there was some API that abstracted the DB and all technical details and you could connect nodes to it that are specific databases with specific capabilities and it would delegate as necessary.
- paulryanrogers 5y agoFDW may be a step in that direction
- srcreigh 5y agoYou're basically talking about Airflow/Airbyte/custom ETL + database expertise. The only way to get efficient performance is expertise, expertise is expensive, ETLs are a given when you have expertise... Just hire a DB consultant or two and you're all set.
- vbezhenar 5y agoODBC, JDBC.
- mns06 5y agoYou might want to look into debezium. We use it to extract the change log from a generic OLTP database into Materialize, a view maintenance engine. Combining that data with event streams in Kafka is very powerful for us.
- Too 5y agoQuery language and data modeling for a db highly depends on if it is relational, graph, time series, denormalized or KV. Don't think this would be possible beyond what's already available in form of ORMs. Even getting SQL dialects to agree is a challenge some times.