> Does that mean that a single user has an exclusive lock on 8 of these things for the duration of the request? I'm surely missing something. You're not missin

> Does that mean that a single user has an exclusive lock on 8 of these things for the duration of the request? I'm surely missing something. You're not missing anything, that's exactly what it means. There is a reason OpenAI severely ratelimits the number of GPT-4 prompts users can make. Every token (approximately 4 letters) requires many trillions of floating point operations. Large language models are only possible because of decades of exponential transistor density growth.