5 ms·
Show HN: Deploy Scikit and Keras Models with a Simple Drag and Drop
- ebalit 6y agoI see that you accept models up to 1 GB. It seems the inference time might be high for models of this size on CPUs. Do you use GPUs to speed up inference for deep learning models ?
- theo31 6y agoWe don’t offer GPU accelerated inference for now. However it’s on our roadmap, sign up for inferrd to get updates!
- ishcheklein 6y agoLooks interesting! How about models that require dictionaries - e.g. tf-idf to convert text into a feature vector? Does it allow for some preprocessing?
- theo31 6y agoHi! If don’t need any pre processing we have built in support for all the major librairies. In addition we support custom pre and post processor via custom environments! Simply write your inference code into a predict.py and we take care of the rest. (Btw I am really into CML.dev, great idea)
- bpg_92 6y agoIt seems very interesting! What about support for pytorch models or .onnx? I usually use the pytorch->onnx->tensorrt to deploy models.
- theo31 6y agoHi! We do not currently have PyTorch model but it's one of the next items in our roadmap. If you have a more custom pipeline, we have a custom environment where you can deploy any custom code with specific package versions!
- zara_arias 6y agoIs this able to support more than 50 requests per second? Are there any benchmarks on performance overhead of the underlying web server/routing that is handling the requests?
- pplonski86 6y agoIs there any reason that you select 50 requests per second? Anyway, I'm also interested in performance metrics.
- theo31 6y agoTheoretically we can support up to thousands of request per second under the entreprise plan. We stopped at 50 request/s in the pricing table because that seemed like a reasonable number for most use cases.
- rubatuga 6y agoYou might want to change 100 Mb --> 100 MB
- theo31 6y agoThank you for spotting that, it's 100MB.