3 ms·
So it's a price concern? Because you could run for 4k output 8 times to get 32K? Or does the RLHF stuff prevent you from feeding the output back in as more inpu
by seeknotfind 2y ago
So it's a price concern? Because you could run for 4k output 8 times to get 32K? Or does the RLHF stuff prevent you from feeding the output back in as more input and still get a decent result? The underlying transformers shouldn't care because they'll be doing that already effectively.
- xena 2y agoI'd say it's less a price concern and more a consistency of output concern. It doesn't make much sense to continue incomplete JSON like that I don't think. I need to do some more research.