5 ms·
The Big Dictionary of MLOps
- vatj2 4y agoHelpful for the serverless MLOps community, big thanks!
- mari_lc 4y agoWas this generated by GPT-4?
- jamesblonde 4y agoAuthor here. I started it prior to Chat-GPT. At the end, i used it for help on some of the definitions. GPT-4 is better than Chat-GPT for definitions, but there are still large amounts of hallucinations there. I think it's because it's a new field.
- garryindiana 4y agoThe “AI hallucinations is bad” is a wacky take. Feels like it presumes humans are 100% accurate, concrete thinkers. I mean; fiat currency value, nation states; human hallucinations. The list of ephemeral “truth” we hallucinate is extensive. Superman, Star Wars, video games. AI cannot escape replicating the training material and characteristics of its creators. Is it not a hallucination we are creating “artificial intelligence”? Human consciousness took millions of years of galactic churn. Is convincing our senses via disembodied metal and plastic we can unplug creating intelligence? Seems pretty dumb if it can’t avoid it’s own death from unplugging.
- antman 4y agoData leakage definition is not accurate I think. Data leakage is when your training data contains information about the target, but similar data will not be available when the model is used. It can be either internal or external data.
- jamesblonde 4y agoYour training data always contains information about the target - the features are the information about the target. You will need to provide values for those features when the model is used. If you are training the model with features that cannot be used at inference time, then it is data leakage. The definition can be improved, i agree. Is there anything incorrect in the definition from your point of view?
- antman 4y ago"Data leakage occurs when data from outside of the training dataset is explicitly or implicitly used to train a model, and it can result in the incorrect estimation of the trained model’s performance" Data leakage is I think when data IN the training dataset regardless how they ended up there contained information which were part of the output and will not be available at inference time, thus enhancing the model's performance. Example: Estimating crop yield earnings in $ from sat photos and getting a historical dataset that included crop type. Crop type should be unavailable at inference time so it should also be detected from the sat photo although it is not the target variable. Whether it is externally or internally provided does not capture the essence of the problem.
- jamesblonde 4y agoYou're correct that the leaked data has to be in the training dataset, sure. Now, for your example - if the crop was a feature used when training, your model inference input should also require that you input the crop type. When you write your inference pipeline, you will realize that - oh, that feature is not available at inference time. I should retrain my model without the feature. It's not really data leakage - it's incorrect use of features when training a model - that model should never make it to production. So, maybe it's an edge case of data leakage.
- jamesblonde 4y agoI added this to the definition to cover your example: Feature data leakage is when you include a feature when training a model, but that feature is not available at inference time. You should, hopefully, discover feature data leakage when you try to write your inference pipeline and realize the feature is not available. The use of a feature store, that informs you whether your feature will be available for online models or only offline models, should help prevent you from training models with feature data leakage.
- momothereal 4y ago> ETL stands for Extract, Load, and Transform of data.
- jamesblonde 4y agoThanks! Will fix this typo.
- rbirkby 4y agoDistillation. Corpora. Zero-shot. One-shot/few-shot. Labelled and unlabelled. Self-supervised.
- LexSiga 4y agoSo I guess this opens the question of which part really covers MLOps; I would love to see those but some strike me heavily as being part of the model development and training. I somewhat, in my simple mind always got stuck on the Ops in a “how to keep the system rolling” kind of way.