9 ms·
What is the difference between a ML feature store and a low-latency OLAP DB platform/data warehouse? I see many similarities between both, like the possibility
by giovannibonetti 3y ago
What is the difference between a ML feature store and a low-latency OLAP DB platform/data warehouse? I see many similarities between both, like the possibility of performing aggregation of large data sets in a very short time.
- uoaei 3y agoFeature stores are more for fast read and moderate write/update for ML training and inference flows. Good organization and fast query of relatively clean data. Data warehouse is more for relatively unstructured or blobby data with moderate read access and capacity for massive files. OLAP is mostly for feeding streaming and event-driven flows, including but not limited to ML.
- jamesblonde 3y agoYou need the columnar store for both training data and batch inference data. If you have a batch ML system that works with time series data, the feature store will help you create point in time correct training data snapshots from the mutable feature datab(no future data leakagae), as well as batch inference data. For real-time ml systems, it give uou row oriented retrival latencies for features. Most importantly, it helps modularize your ML system into feature pipelines training pipelines, and inference pipelines. No onolithic ML pipelines.
- nikhilsimha 3y agothe ability generate training sets against historical inferences to back-test new features another one is the focus on pushing as much compute to the write-side as possible (within Chronon) - specially joins and aggregations. OLAP databases and even graph databases don't scale well to high read traffic. Even when they do, the latencies are very high.
- giovannibonetti 3y agoYou may want to take a look at Starrocks [1]. It is an open-source DB [2] that competes with Clickhouse [3] and claims to scale well – even with joins – to handle use cases like real-time and user-facing analytics, where most queries should run in a fraction of a second. [1] https://www.starrocks.io/ https://www.starrocks.io/ [2] https://github.com/StarRocks/starrocks https://github.com/StarRocks/starrocks [3] https://www.starrocks.io/blog/starrocks-vs-clickhouse-the-quest-for-analytical-database-performance https://www.starrocks.io/blog/starrocks-vs-clickhouse-the-qu...
- nikhilsimha 3y agoWe did and gave up due to scalability limitations. Fundamentally most of the computation needs to happen before the read request is sent.
- jvican 3y agoHey! I work on the ML Feature Infra at Netflix, operating a similar system to Chronon but with some crucial differences. What other alternatives aside from Starrocks did you evaluate as potential replacements prior to building Chronon? Curious if you got to try Tecton or Materialize.com.
- nikhilsimha 3y agoWe haven’t tried materialize - IIUC materialized is pure kappa. Since we need to correct upstream data errors and forget selective data(GDPR) automatically - we need a lambda system. Tecton, we evaluated, but decided that the time-travel strategy wasn’t scalable for our needs at the time. A philosophical difference with tecton is that, we believe the compute primitives (aggregation and enrichment) need to be composable. We don’t have a FeatureSet or a TrainingSet for that reason - we instead have GroupBy and Join. This enables chaining or composition to handle normalization (think 3NF) / star-schema in the warehouse. Side benefit is that, non ml use-cases are able to leverage functionality within Chronon.
- jamesblonde 3y agoFeatureSets are mutable data and TrainingSets are consistent snapshots of feature data (from FeatureSets). I fail to see what that has to do with composability. Join is still available for FeatureSets to enable composable feature views - join is resuse of feature data. GroupBy is just an aggregation in a feature pipeline, not sure your point here. You can still do star schema (and even snowflake schema if you have the right abstractions).
- jamesblonde 3y agoNormalization is a model-dependent transformation and happens after the feature store - needs to be consistent between training and inference pipelines.
- csmpltn 3y agoThere is none. The industry is being flooded with DS and "AI" majors (and other generally non-technical people) that have zero historical context on storage and database systems - and so everything needs to be reinvented (but in Python this time) and rebranded. At the end of the day you're simply looking at different mixtures of relational databases, key-value stores, graph databases, caches, time-series databases, column stores, etc. The same stuff we've had for 50+ years.
- nikhilsimha 3y agoTwo main differences - ability to time travel for training data generation and the ability to push compute to the write side of the view rather than the read side for low latency feature serving.
- ShamelessC 3y ago> ability to time travel for training data generation What now?
- nikhilsimha 3y agoPardon the jargon. But it is a necessary addition to the vocabulary. To evaluate if a feature is valuable, you could attach the value of the feature to past inferences and retrain a new model to check for improvement in performance. But this “attach”-ing needs the feature value to be as of the time of the past inference.