We use big data and ML for our project. The training happens on a separate machine running Scala, spark and deeplearning4j. This app reads from our database cre

We use big data and ML for our project. The training happens on a separate machine running Scala, spark and deeplearning4j. This app reads from our database creates the model and then saves it to a network drive. This happens every Sunday night. The web application reloads the ml model and then makes predictions based on real time data. There are also plenty of predictions that do not require real time prediction for example inventory levels, sales for next few days etc. In this case we train on the training machine and write the results back to the DB using spark jdbc. The web just displays this data. We have scaled up pretty well however in the future we will move to Apache Kafka to real time data transfer and prediction