steer_dev

7 ms·

1.
by steer_dev 9mo ago
Hi HN, I'm a senior MLE. I've been watching the hype around agent swarms and tools like Gas Town clash with reality of productizing them. I wrote this because I see companies burning millions in ''reasoning tax" (LL
2.
Engineering is no longer construction: it is verification(steerlabs.substack.com)
3 points by steer_dev 9mo ago|1 comments
3.
by steer_dev 10mo ago
The Data: Green Line (Capability): MMLU State-of-the-Art (48% → 94%). Red Line (Control): Organizations with effective mitigation for inaccuracy/hallucination (lagging at ~52%). Sources: MMLU: Official Repository (hendrycks/test)
4.
The "Confident Idiot" Problem: Intelligence vs. Control (2021-2026)(steerlabs.substack.com)
2 points by steer_dev 10mo ago|1 comments
5.
Show HN: I got tired of "Reliability Spaghetti," so I monkeypatched PydanticAI
2 points by steer_dev 10mo ago|0 comments
6.
by steer_dev 10mo ago
AI Slop (apologies, filler) isn't just annoying; it is In-Band Signaling Noise. The model mixes control signals (persona) with data (payload). Prompting ("Be concise") is brittle. I switched to Shannon Entropy. The Hypothesis
7.
Detecting "AI Slop" with Shannon Entropy (Python)(steerlabs.substack.com)
2 points by steer_dev 10mo ago|1 comments
8.
by steer_dev 10mo ago
Doing post-mortems on my agent's failures over the holidays made me realize the problem isn't the model. It is the lack of a deterministic inference-time verification layer. I spent the break reading the recent Stanford/Harva
9.
The Agent Lobotomy: Inference-time verification for autonomous systems(steerlabs.substack.com)
1 points by steer_dev 10mo ago|1 comments
10.
by steer_dev 11mo ago
OP here. I’ve realized that a 3,000-token system prompt isn't "logic", it's legacy code that no one wants to touch. It’s brittle, hard to test, and expensive to run. It is Technical Debt. My thesis is that we need to sto
11.
Prompt Engineering Is Technical Debt(steerlabs.substack.com)
1 points by steer_dev 11mo ago|1 comments
12.
by steer_dev 11mo ago
OP here. Last week I posted a discussion ("The Confident Idiot Problem") about why we need deterministic checks instead of just "LLM vibes" for reliability. That thread [1] blew up, so I’m sharing the open-source impleme
13.
Show HN: Steer (v0.2) – Active reliability layer for AI agents (Python)(github.com)
1 points by steer_dev 11mo ago|1 comments
14.
by steer_dev 11mo ago
You're 100% right. For a "judgment" task like "Does this patient have cancer?", the final acceptance criteria must be a human expert. A purely deterministic verifier is impossible. My thesis is that even in those &q
15.
by steer_dev 11mo ago
Exactly. We treat them like databases, but they are hallucination machines. My thesis isn't that we can stop the hallucinating (non-determinism), but that we can bound it. If we wrap the generation in hard assertions (e.g., assert resp
16.
by steer_dev 11mo ago
OP here. I wrote this because I got tired of agents confidently guessing answers when they should have asked for clarification (e.g. guessing "Springfield, IL" instead of asking "Which state?" when asked "weather in
17.
The "confident idiot" problem: Why AI needs hard rules, not vibe checks(steerlabs.substack.com)
324 points by steer_dev 11mo ago|380 comments
18.
Show HN: Steer – Stop debugging agents, start teaching them (Open Source)(github.com)
2 points by steer_dev 11mo ago|0 comments