It depends. For batching to be viable each prompt has to share some similarities of context/intention, which quite often is the case in specific applications (a

It depends. For batching to be viable each prompt has to share some similarities of context/intention, which quite often is the case in specific applications (as opposed to say general chats)