4 ms·
what is exactly a "feature store"? do you store "features" by themselves? do you store whole data? Can you give me a bit of insight on this?
by kuu 6y ago
what is exactly a "feature store"? do you store "features" by themselves? do you store whole data?
Can you give me a bit of insight on this?
- SirOibaf 6y agoYou can see the Hopsworks feature store as a repository of curated features ready to be used in ML models. Or, a middle layer between data engineers and data scientists: - Data engineers write the data pipelines with the transformations and publish the features on the feature store. - Data scientists browse the feature store, pick which features they need and build the model In the Hopsworks Feature Store we group features together in feature groups. Feature groups can be then joined to create training datasets. (You can also select a subset of features from a feature group) Training datasets are stored in a ML Framework friendly format (e.g. TFRecords if you are using TensorFlow) and you can feed them directly to your model. If you are interested, we have a longer blog post explaining the core concepts of the Hopsworks feature store: https://www.logicalclocks.com/blog/feature-store-the-missing-data-layer-in-ml-pipelines https://www.logicalclocks.com/blog/feature-store-the-missing...
- overfitted 6y agoThanks! Also the first 10-15 minutes in this recording explains the Feature Store concept and how it can integrate with other ML tools (in this case Sagemaker) incl. slides, examples and demo if you are more of a watch & listen type. https://www.youtube.com/watch?v=3DaTA7o0FHY&list=PLgN6fhzkSui-NWA7_tPFhBACUaIzFjj9C&index=5 https://www.youtube.com/watch?v=3DaTA7o0FHY&list=PLgN6fhzkSu...