Its what "interprets" the prompt to give to the unet (which does the actual diffusion). SDXL's better text encoder is partially why it can understand sentence

Its what "interprets" the prompt to give to the unet (which does the actual diffusion). SDXL's better text encoder is partially why it can understand sentences better than SD 1.5, and you can swap it out for, say, BLIP for combined image + text prompting.