2 ms·

I guess the biggest limitation of this approach is that the max output length is fixed before generation starts. Unlike autoregressive LLM, which can keep gener

I guess the biggest limitation of this approach is that the max output length is fixed before generation starts. Unlike autoregressive LLM, which can keep generating forever.

gdiamos 2y ago
max output size is always limited by the inference framework in autoregressive LLMs eventually they run out of memory or patience