Especially given that the buffer is often short, i.e. far shorter than what the sampling rate implies, in order to reduce latency. The buffer size of 32--1024 s

Especially given that the buffer is often short, i.e. far shorter than what the sampling rate implies, in order to reduce latency. The buffer size of 32--1024 samples is common and, assuming 44.1 kHz, translates to 0.7--23 ms of audio to repeat in such cases. The limited buffer size would act as a crude high-pass filter and almost always result in a high-pitched noise. It is possible to have a larger buffer that can be opportunistically filled for such cases, say, you may fill 1024 future samples while only the first 256 will be processed soon and you don't have to touch the remainder unless something else happened, but AFAIK this is not how most audio APIs work because such opportunistic buffering can be done outside of the API anyway.