It depends. LLMs are text completion bots. If you want it to chat with you, you prompt it with 'The following is a chat log between user and bot: User: Hi\n\nBo

It depends. LLMs are text completion bots. If you want it to chat with you, you prompt it with 'The following is a chat log between user and bot: User: Hi\n\nBot:' and it will keep writing as if it got to the end of an actual chat log and is completing what it thinks could be the next part of it, starting with Bot:. In this case it will take cues from the writing before, so any stylistic or misspelling issues could be imitated. However, instruct finetunes (like GPT-4) will generally not do this. They are trained on a set of data with 'Instruction' and 'Ideal completion', and are filling the 'ideal completion' part, and thus any errors on the instruction should not be detrimental, as long as it can understand what has been written. Of course this also depends on the 'unseen' parts of the prompting. An example of this is given above -- you don't write the initial part of the prompt as a user, nor do you add the start and end sequences. If you don't use the type of prompting that a model was trained on, you can get strange results.