4 ms·

What would HN commenters define as a truly modern data stack right now? Mostly asking what this chart https://a16z.com/emerging-architectures-for-modern-data-in

by extr 3y ago

What would HN commenters define as a truly modern data stack right now? Mostly asking what this chart https://a16z.com/emerging-architectures-for-modern-data-infrastructure/ https://a16z.com/emerging-architectures-for-modern-data-infr... would look like in 2024.

mr_toad 3y ago
> What would HN commenters define as a truly modern data stack right now? Just don’t spend too much time debating what it is, or the analysts who need numbers yesterday will develop their own solutions before you’ve even published your A3 architecture diagram. A bird in the hand, as they say.
rch 3y ago
- Postgres and Kafka - Airflow and Flink - Kudu+Impala or Clickhouse - Iceberg/Parquet or Delta - Ozone(HDFS) or Ceph - Spark Rapids and/or Ray+Metaflow - Open Metadata or Atlas+Ranger
- fifilura 3y ago
  TBH that sounds like a heap of buzzwords not useful for anyone (but maybe an architect promoting his career).
  - rch 3y ago
    Pretty much, but updated for 2024.
te_chris 3y ago
IT DEPENDS. But for standard ecom/ops stuff (and this is heavily biased by the fact this is what I know and have deployed with low overhead): Big query. This is fed by an ETL pipeline of the stuff you need. dbt to ELT into useful, reportable tables. Something to visualise - I still like Mode, but there's lots of new BI toys around. KISS. This is extremely low maintenance at the actual requirement levels of most businesses with < 50 employees.
neighbour 3y ago
- Fivetran - Snowflake/BigQuery/Redshift - dbt - Looker/Metabase/Tableau