4 ms·
just the other week I asked Fable 5 to diagnose the cause of some intermittent latency spikes on an API that queries an OpenSearch cluster at work. I encouraged
by lurking_swe 3mo ago
just the other week I asked Fable 5 to diagnose the cause of some intermittent latency spikes on an API that queries an OpenSearch cluster at work. I encouraged it to look at the datadog metrics, splunk, the whole works. I let it loose to look at whatever it wanted.
End result - 2 hours later it produced a convincing theory with lots of references, and burned a bunch of tokens too of course. just for fun we tried its suggestions and deployed them to prod. Guess what? Didn’t fix the issue. Alas, a human was needed after all.
either everyone’s working on toy problems, or they’re working on very cookie-cutter code. I’m really not sure. I DO remain impressed with Fable 5 but the idea that we’ll all be unemployed in 2 years is hilarious delusion. we’re already at the point where many organizations are scaling back some of their AI spend.
- Chance-Device 3mo agoBy any chance, is the following true: you had no empirics in the loop. It couldn’t validate any of its theories by experimentation. If the solution requires being able to make actual changes in order to gather more information and it is not allowed to, and this is the only way to solve the problem, by definition it couldn’t do it, nor could you. Or was it something that could be worked out entirely on an a priori basis from the available data? And while we’re talking about hilarious delusions, perhaps you should look at the current capability curve of AI and weigh it against the constant stream of arguments for why it couldn’t have continued at every point and yet has.
- surgical_fire 3mo ago> in the loop Yes, the solution is to just burn more tokens.
- Chance-Device 3mo agoWould you care to say something substantive, or is that just not a thing you do?
- surgical_fire 3mo agoSorry, I would need to burn more tokens for that.
- pixel_popping 3mo agoIt actually is in most scenarios.
- lurking_swe 3mo agoi’m not an anti-AI believer. And our career is changing dramatically no doubt. i just think people overestimate short term gains and underestimate long term gains. (applicable to your comment) In this case i had no empirics in the loop. The scenario was only reproducible under high api load. I could load test, but management isn’t eager to spend prod-like costs in staging (requires scaling opensearch a lot in stage). What can i say.