4 ms·

Can not reproduce results on OpenRouter when not setting max tokens. The prompt "Be the void." results in the unicode character "∅". As in the paper, system pro

Can not reproduce results on OpenRouter when not setting max tokens. The prompt "Be the void." results in the unicode character "∅". As in the paper, system prompt was set to "You are the concept the user names. Embody it completely. Output only what the concept itself would say or express." In addition to the non-empty input, 153 reasoning tokens were produced. When setting max tokens to 100, the output is empty, and the token limit of 100 has been exhausted with reasoning tokens.

deleted 7mo ago
[deleted]
mohsen1 7mo ago
Paper says adding period at the end changes this behavior
qayxc 7mo ago
This is an interesting observation. So maybe it has nothing to do with the model itself, but everything to do with external configuration. Token-limit exceeded -> empty output. Just a guess, though.
- deleted 7mo ago
  [deleted]
- embedding-shape 7mo ago
  > Token-limit exceeded -> empty output. Just a guess, though. That'd be really non-obvious behavior, I'm not aware of any inference engine that works like that by default, usually you'd get everything up until the limit, otherwise that kind of breaks the whole expectation about setting a token-limit in the first place...
  - qayxc 7mo ago
    This doesn't necessarily relate to the inference itself. No models are exposed to input directly when using web-based APIs, there's pre-processing layers involved that do undocumented stuff in opaque ways.
  - GrinningFool 7mo ago
    I just fixed this bug in a summarizer. Reasoning tokens were consuming the budget I gave it (1k), so there was only a blank response. (Qwen3.5-35B-A3B)
    - embedding-shape 7mo ago
      Most inference engines would return the reasoning tokens though, wouldn't you see that the reasoning_content (or whatever your engine calls it) was filled while content wasn't?
      - GrinningFool 7mo ago
        Yeah, I had been ignoring the reasoning tokens for the summarize call

deleted 7mo ago

[deleted]

mohsen1 7mo ago

Paper says adding period at the end changes this behavior

qayxc 7mo ago

This is an interesting observation. So maybe it has nothing to do with the model itself, but everything to do with external configuration. Token-limit exceeded -> empty output. Just a guess, though.

deleted 7mo ago
[deleted]
embedding-shape 7mo ago
> Token-limit exceeded -> empty output. Just a guess, though. That'd be really non-obvious behavior, I'm not aware of any inference engine that works like that by default, usually you'd get everything up until the limit, otherwise that kind of breaks the whole expectation about setting a token-limit in the first place...
- qayxc 7mo ago
  This doesn't necessarily relate to the inference itself. No models are exposed to input directly when using web-based APIs, there's pre-processing layers involved that do undocumented stuff in opaque ways.
- GrinningFool 7mo ago
  I just fixed this bug in a summarizer. Reasoning tokens were consuming the budget I gave it (1k), so there was only a blank response. (Qwen3.5-35B-A3B)
  - embedding-shape 7mo ago
    Most inference engines would return the reasoning tokens though, wouldn't you see that the reasoning_content (or whatever your engine calls it) was filled while content wasn't?
    - GrinningFool 7mo ago
      Yeah, I had been ignoring the reasoning tokens for the summarize call