Current state of the art in embedded/IoT ML is to train ML algos in the cloud on large datasets, then run it on gateway class devices (usually Linux/MSFT boxes,

Current state of the art in embedded/IoT ML is to train ML algos in the cloud on large datasets, then run it on gateway class devices (usually Linux/MSFT boxes, but can get down to RPi levels of mem and compute). Most companies today use docker to package and deploy the models, hence the need for a larger footprint box. Check out AWS Greengrass, Azure Edge, and Foghorn for examples.