qwesr123

4 ms·

1.
A tycoon game where you run an AI company(store.steampowered.com)
10 points by qwesr123 2mo ago|1 comments
2.
Claude Code Degraded Before Opus 4.8 Release(marginlab.ai)
8 points by qwesr123 4mo ago|0 comments
3.
by qwesr123 8mo ago
How do you plan to be harness agnostic if you design requires using the agent SDKs (Claude Agent SDK, etc.)?
4.
Claude Code daily benchmarks for degradation tracking(marginlab.ai)
760 points by qwesr123 8mo ago|354 comments
5.
by qwesr123 8mo ago
FYI the MarginLab Claude Code degradation tracker is showing a statistically significant ~4% drop in SWE-Bench-Pro accuracy over the past month
6.
No one is evaluating AI coding agents in the way they are used(marginlab.ai)
1 points by qwesr123 9mo ago|0 comments
7.
by qwesr123 9mo ago
Thanks! Daily confidence intervals are quite large and not super useful at the moment. Weekly aggregation is more sensitive. Hoping to increase sample sizes but it is quite expensive! Would be about $100-$150/day in API costs. We are u
8.
Claude Code Daily Degradation Tracker(marginlab.ai)
3 points by qwesr123 9mo ago|3 comments
9.
Anatomy of a Coding Agent: A step-by-step illustration(marginlab.ai)
3 points by qwesr123 10mo ago|0 comments
10.
How are coding assistants evaluated? SWE-Bench Pro Explorer(marginlab.ai)
2 points by qwesr123 10mo ago|0 comments
11.
by qwesr123 10mo ago
Where are you getting SWE-Bench Verified scores for 5.2-Codex? AFAIK those have not been published. And I don't think your Terminal-Bench 2.0 scores are accurate. Per the latest benchmarks: Opus 4.5 is at 59% GPT-5.2-Codex is at 64% S
12.
by qwesr123 10mo ago
Not sure why they don't compare with others, but they are actually leading on the benchmarks they published. See here (bottom) for a chart comparing to other models: https://marginlab.ai/blog/swe-bench-deep-dive&#x
13.
SWE-Bench: The $500B Benchmark(marginlab.ai)
5 points by qwesr123 10mo ago|0 comments
14.
Show HN: Database of foods likely to trigger IBS(bloaty.io)
4 points by qwesr123 1y ago|0 comments