3 ms·

https://github.com/huggingface/transformers https://github.com/huggingface/transformers (was) based on PyTorch, so they originally had lot of the models like gp

https://github.com/huggingface/transformers https://github.com/huggingface/transformers (was) based on PyTorch, so they originally had lot of the models like gpt2-small. Because of that influence, PyTorch was probably going to win there. The one con about TF Serve (TFX) is packaging the entire model into the container (so that ends up being 3gb?+). This was a couple of months ago, so I might be wrong by now ... It was an area I wasn't very confident I could do, TF Serve is really new, so there aren't many guides (and many were already out of date).

solidasparagus 7y ago
TF Serving has been around several years and is pretty mature. (It predates TFX). You can stick the model in the container or have the container pull the model from blob storage (or have no container at all if you really want). It's too bad you didn't find a good guide - if you have the training dump a SavedModelBundle at the end, you can have a production-quality serving microservice up and running in about two lines of code - https://www.tensorflow.org/tfx/serving/docker https://www.tensorflow.org/tfx/serving/docker. But it doesn't really matter since you got it working.