Most "production" ML models of the last 10 years were only a few millions of parameters owing to limited training data and simplistic tasks. While BERT and simi

Most "production" ML models of the last 10 years were only a few millions of parameters owing to limited training data and simplistic tasks. While BERT and similar models raised the parameter count substantially, my anecdotal experience observing hundreds of ML projects was that BERT based models provided only a marginal (to the business) gain in performance over smaller models. We are now seeing mass consumer and enterprise adoption of models in the hundreds of billions of parameters (GPT-4). These models use relatively standard, building blocks - and are even chained together in a relatively standard manner. If everyone's laptop was running a 7B parameter LLM, or actively using a cloud based derivative - then LLM inference would have a chance at being the dominant compute workload in the industry. This will definitely change the way that inference is done, even if training remains the same.