"World state" is the text people see on script. It can reside in the browser. So a server can be completely stateless, i.e. it receives text so far (say, 5 KB)

"World state" is the text people see on script. It can reside in the browser. So a server can be completely stateless, i.e. it receives text so far (say, 5 KB), applies GPT-2 generate and returns. The problem is that GPT-2 generate seems to be very computationally intensive. As I understand, it actually does number crunching with all these 6 GB of data, so it takes 5-10 seconds even on GPU (K80, at least, is that slow). Is GPU capable of running multiple GPT-2 generate in parallel? No idea. Assuming that a high-end GPU would be able to produce response in 2 seconds, you can only run maybe 10 concurrent sessions per server if you want fast response time.