Models like GPT-4 are highly functional, so they are not snake oil in that sense, but they are certainly not scientifically reproducible (or understood for that

Models like GPT-4 are highly functional, so they are not snake oil in that sense, but they are certainly not scientifically reproducible (or understood for that matter). In fact there's generally a reproducibility crisis in machine learning, since things like models, datasets, training techniques, hyper-parameters are often only documented in the vaguest of terms. Reproducibility doesn't mean I have a black box and when I push the button (after swiping my credit card) the light comes on as expected. Reproducibility in science means I can reproduce the experiment from scratch and get the same results. Practically nothing is known about GPT-4. OpenAI's "GPT-4 Technical Report" doesn't say much more than the fact that it's "Transformer-based". https://arxiv.org/pdf/2303.08774.pdf https://arxiv.org/pdf/2303.08774.pdf