4 ms·
You can see the Hopsworks feature store as a repository of curated features ready to be used in ML models. Or, a middle layer between data engineers and data sc
by SirOibaf 6y ago
You can see the Hopsworks feature store as a repository of curated features ready to be used in ML models. Or, a middle layer between data engineers and data scientists:
- Data engineers write the data pipelines with the transformations and publish the features on the feature store.
- Data scientists browse the feature store, pick which features they need and build the model
In the Hopsworks Feature Store we group features together in feature groups. Feature groups can be then joined to create training datasets. (You can also select a subset of features from a feature group) Training datasets are stored in a ML Framework friendly format (e.g. TFRecords if you are using TensorFlow) and you can feed them directly to your model.
If you are interested, we have a longer blog post explaining the core concepts of the Hopsworks feature store: https://www.logicalclocks.com/blog/feature-store-the-missing-data-layer-in-ml-pipelines https://www.logicalclocks.com/blog/feature-store-the-missing...
- overfitted 6y agoThanks! Also the first 10-15 minutes in this recording explains the Feature Store concept and how it can integrate with other ML tools (in this case Sagemaker) incl. slides, examples and demo if you are more of a watch & listen type. https://www.youtube.com/watch?v=3DaTA7o0FHY&list=PLgN6fhzkSui-NWA7_tPFhBACUaIzFjj9C&index=5 https://www.youtube.com/watch?v=3DaTA7o0FHY&list=PLgN6fhzkSu...