Pretty sure they are talking about inference in the post you are responding to. Training the model obviously needs far more compute, but running them locally is

Pretty sure they are talking about inference in the post you are responding to. Training the model obviously needs far more compute, but running them locally is what most devs are interested in.