3 ms·
You can choose which token to sample based on language semantics. You simply don't sample invalid ones. So the language should be restrictive on what tokens it
by WithinReason 9mo ago
You can choose which token to sample based on language semantics. You simply don't sample invalid ones. So the language should be restrictive on what tokens it allows enough that invalid code is impossible.
- SkiFire13 9mo ago> You can choose which token to sample based on language semantics Can you though? > the language should be restrictive on what tokens it allows This is a restriction on the language syntax, not its semantics.