Simon, sorry I didn't get around to answering your question on post-t5 encoder-decoders from the Markdown Lethal Trifecta prompt injection post. (https://news.y

Simon, sorry I didn't get around to answering your question on post-t5 encoder-decoders from the Markdown Lethal Trifecta prompt injection post. (https://news.ycombinator.com/item?id=45724941 https://news.ycombinator.com/item?id=45724941) Since the plain decoder models stole the show, Google DeepMind demonstrated a way to adapt LLMs,adding a T5 encoder to an existing normal Gemma model to get the benefits of the more grounded text-to-text tasks WITHOUT instruction tuning (and the increased risk of prompt injection). They also have a few different kinds they shared on HuggingFace. I didn't get around to fine-tuning the weights of one for summarisation yet but it could well be a good way for more reliable summarisation. I did try out some models for inference though and made a Gist here, which is useful since I found the HF default code example a bit broken: https://gist.github.com/lukestanley/ee89758ea315b68fd66ba52cd2123c35 https://gist.github.com/lukestanley/ee89758ea315b68fd66ba52c... Google's minisite: https://deepmind.google/models/gemma/t5gemma/ https://deepmind.google/models/gemma/t5gemma/ Paper: https://arxiv.org/abs/2504.06225 https://arxiv.org/abs/2504.06225 Here is one such model that didn't hallucinate and actually did summarise on HF: https://huggingface.co/google/t5gemma-l-l-prefixlm https://huggingface.co/google/t5gemma-l-l-prefixlm