3 ms·
Each text token already represents the activation of certain neurons. There is nothing "more direct." And you cannot fully separate data and metadata if you wan
by yorwba 7mo ago
Each text token already represents the activation of certain neurons. There is nothing "more direct." And you cannot fully separate data and metadata if you want them to influence the output. At best you can clearly distinguish them and hope that this is enough for the model to learn to treat them differently.
- perlgeek 7mo agoAre there tokens reserved for tool calls? If yes, I can see the equivalence. If not, not so much.
- yorwba 7mo agoYes, typically the tags used for tool calls get their own special tokens, e.g. https://huggingface.co/google/gemma-4-E4B-it/blob/main/tokenizer_config.json https://huggingface.co/google/gemma-4-E4B-it/blob/main/token...