XCSme

15 ms·

301.
by XCSme 6mo ago
I actually meant it in a different way. I would get it for local AI stuff, but being able to game on it would be a huge plus, otherwise I would need two different machines.
302.
by XCSme 6mo ago
But with new hardware comping out, and maybe models being smart enough to help with optimizing them and reducing inference costs even more, I think we should still expect the costs to go down.
303.
by XCSme 6mo ago
Can you use those AI cards for gaming too? Or the makers intentionally nerf them, in order to better segment the markets/product lines?
304.
by XCSme 6mo ago
I am a bit confused by the separation between VSCode and Copilot. If I cancel my Pro+ subscription, can I still use Copilot with my own OpenRouter key?
305.
by XCSme 6mo ago
Yeah, but you get the benefit of using any model of your choice.
306.
by XCSme 6mo ago
That's so cool. I would have expected for the domain to go for hundreds of thousands or millions, or, more likely, not not be purchasable for some reason. I can see a future where google.com is purchased for fun by some robot in 200 ye
307.
by XCSme 6mo ago
Do we also have to pay for the API usage? Then they will actually be profitable, lol
308.
by XCSme 6mo ago
Their API issues seemed to have been resolved, now it does[0] as expected, similar to GLM 5 level. [0]: https://aibenchy.com/compare/deepseek-deepseek-v4-flash-high...
309.
by XCSme 6mo ago
Oh, this makes sense, so instead of "the app is rotating its keys" is more like "the keys in our app are being rotated by an external service".
310.
by XCSme 6mo ago
But how do you do that without also having a long-lived key or access token to those services?
311.
by XCSme 6mo ago
Some API errors are actually not about reliability, are because that specific API doesn't support some common features (e.g. specific structured_output formats).
312.
by XCSme 6mo ago
I agree that it's confusing, I have already implemented a reliability score, but it will only apply for new tests from now on. I have already re-tested DeepSeek v4, so it doesn't have any API error issues. API errors are quite rar
313.
by XCSme 6mo ago
GPT 5.5 is close to Opus 4.7, but at 7x the cost[0]... Either Opus 4.7 miscounts reasoning tokens, or it's A LOT more efficient than GPT 5.5 I thought they made GPT 5.5 more token efficient than 5.4, but it uses 2x the reasoning tokens
314.
by XCSme 6mo ago
But how would you test a closed model independent of their API? For example, the speed score (tokens/s) is also variable and changes over time.
315.
by XCSme 6mo ago
Doubt it, DeepSeek v4 is quite underwhelming.
316.
by XCSme 6mo ago
I feel like the last 2-3 generations of models (after gpt-5.3-codex) didn't really improve much, just changed stuff around and making different tradeoffs.
317.
by XCSme 6mo ago
> You've essentially sampled the infrastructure's reliability at only a few points, which doesn't provide a very meaningful signal Well, sampling is still somewhat meaningful, but I agree with you, I am considering making
318.
by XCSme 6mo ago
It depends... If the conclusion is: "DeepSeek v4 is this good, if you use it from DeepSeek" (which is how most people would use it anyway), then it makes sense to count API errors as failures. But, if the conclusion must be "
319.
by XCSme 6mo ago
@seanw265 Yes, that's a problem. This can be solved for open-source models by running them on my own, but again the TPS will be dependent on the hardware used. All models are tested through OpenRouter. The providers on OpenRouter vary
320.
by XCSme 6mo ago
@danyw, we reached max comment thread depth Yes, I would. Currently I don't have that many tests (~20), and by default a test "run" includes 3 executions of each test. So, "bad luck" is already sort of solved in eac
321.
by XCSme 6mo ago
Because the idea of those benchmarks is to see how well a model performs in real-world scenarios, as most models are served via APIs, not self-hosted. So, for example, hypothetically if GPT-5.5 was super intelligent, but using it via API wo
322.
by XCSme 6mo ago
Yes, Flash doesn't seem to have the same rate limits as Pro. I expect once the API issues are fixed, for v4-pro to be around the same level as GLM-5.
323.
by XCSme 6mo ago
Yup, can't really be used in production atm.
324.
by XCSme 6mo ago
Yup, I tried to benchmark it, but harder questions time out or get rate-limited...
325.
by XCSme 6mo ago
Something is odd with this model, their blog posts shows REALLY good results, but in most other third-party benchmarks, people realize it's not really SOTA, even bellow Kimi K2.6 and GLM-5/5.1 In my tests too[0], it doesn't r
326.
by XCSme 6mo ago
They mentioned in their release page, that the Claude team noticed memorization of the SWE-bench test, so the test is actually in the training data. Here: https://www.anthropic.com/news/claude-opus-4-7#:~:text=memor...
327.
by XCSme 6mo ago
Is this direct API usage allowed by their terms? I remember Anthropic really not liking such usage.
328.
by XCSme 6mo ago
2x the price for 1-5% performance gain
329.
by XCSme 6mo ago
So, an amateur player
330.
by XCSme 6mo ago
Not to take anything away from the robot, but I would have liked to see the match be against a male player, as they can impart higher speed an spin on the ball, which would give the robot less reaction time.

More ›