4 ms·

I think I saw a better overall composition out of Flash 0731 Effort on this one?

by wolttam 2mo ago

simonw 2mo ago
Default effort for OpenRouter. I'll try a grid of efforts... Wow, the low, medium, and high pelicans came out in surprisingly different styles: https://tools.simonwillison.net/markdown-svg-renderer#url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Fc1108a380593547c2def5863bca63160 https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
- segmondy 2mo ago
  exciting. it's almost like 3 models in one. that variety would matter when trying to solve a creative problem.
- throwaway9af11m 2mo ago
  [flagged]
  - simonw 2mo ago
    If I live my life on the basis that some people don't share my sense of humor, and hence I should avoid doing anything funny that might be misunderstood, my life will be a lot less fun.
    - throw10920 2mo ago
      You're doing great. Don't let insanely low-effort (negative-effort, as in making others dumber rather than having no effect?) comments like from the above throwaway affect your actions.
    - Barbing 2mo ago
      I’m always excited to see “simonw” on my screen. More likely than not to be interesting & accessible for those of us outside e.g. compsci. Hope you stay happy so you stay nerdy and keep sharing both with us!
- p1necone 2mo ago
  It's interesting that all three of those used roughly the same amount of tokens, and almost entirely output. Feels like the thinking level lever didn't alter cost at all for this specific task, even though it did change the output.
  - ComputerGuru 2mo ago
    I never trust OpenRouter to forward parameters correctly and would only ever conduct benchmarks with the official api, personally.
    - ljlolel 2mo ago
      should use an open source one you can audit like my site TrustedRouter
  - Lerc 2mo ago
    That raises the question of what is it actually doing? If it isn't spending tokens on quality, is it the assumptions about the task difficulty that cause it to perform better? Or are their broader differences in the model being run.

I think I saw a better overall composition out of Flash 0731 Effort on this one?

by wolttam 2mo ago

I think I saw a better overall composition out of Flash 0731 Effort on this one?

Default effort for OpenRouter. I'll try a grid of efforts... Wow, the low, medium, and high pelicans came out in surprisingly different styles: https://tools.simonwillison.net/markdown-svg-renderer#url=https%3A%2F%2Fgist.github.com%2Fsimonw%2Fc1108a380593547c2def5863bca63160 https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

segmondy 2mo ago
exciting. it's almost like 3 models in one. that variety would matter when trying to solve a creative problem.
throwaway9af11m 2mo ago
[flagged]
- simonw 2mo ago
  If I live my life on the basis that some people don't share my sense of humor, and hence I should avoid doing anything funny that might be misunderstood, my life will be a lot less fun.
  - throw10920 2mo ago
    You're doing great. Don't let insanely low-effort (negative-effort, as in making others dumber rather than having no effect?) comments like from the above throwaway affect your actions.
  - Barbing 2mo ago
    I’m always excited to see “simonw” on my screen. More likely than not to be interesting & accessible for those of us outside e.g. compsci. Hope you stay happy so you stay nerdy and keep sharing both with us!
p1necone 2mo ago
It's interesting that all three of those used roughly the same amount of tokens, and almost entirely output. Feels like the thinking level lever didn't alter cost at all for this specific task, even though it did change the output.
- ComputerGuru 2mo ago
  I never trust OpenRouter to forward parameters correctly and would only ever conduct benchmarks with the official api, personally.
  - ljlolel 2mo ago
    should use an open source one you can audit like my site TrustedRouter
- Lerc 2mo ago
  That raises the question of what is it actually doing? If it isn't spending tokens on quality, is it the assumptions about the task difficulty that cause it to perform better? Or are their broader differences in the model being run.