Have you thought about deploying the ML model on the edge, using something like TensorflowJS or ONNX-runtime? Haven't done it myself (looking into it right now

Have you thought about deploying the ML model on the edge, using something like TensorflowJS or ONNX-runtime? Haven't done it myself (looking into it right now!), but my impression is that model quantization (and possibly prunning) can give you a palatable model size that doesn't affect performance too much.