I'm curious if these low-code models matter. I understand that small codebases can be cached effectively speeding up computations, but isn't it the data load th

I'm curious if these low-code models matter. I understand that small codebases can be cached effectively speeding up computations, but isn't it the data load the bottleneck in training? Furthermore, how important is the breadth of data in the dataset to getting the desired results? I was under the impression that the main reason these LLM work is based on massive data sets. As such, is there data-breadth metrics to validate whether training on a given dataset is even worthwhile? (ie: avoid sunk cost on a dataset that will yield a poorly performing LLM)