3 ms·
Over the last week I tried to use a combination of Claude and OpenAI o3-mini to do a direct conversion of about 500 lines of uncommented academic modeling code
by bglazer 2y ago
Over the last week I tried to use a combination of Claude and OpenAI o3-mini to do a direct conversion of about 500 lines of uncommented academic modeling code from Matlab to Python. I can’t stress enough how badly these models performed. Nearly every consequential line had some variety of off by one or logic error, often very subtle. I didn’t try cursor or the more agentic systems, but I would be astounded if they properly rigged up a test harness, inspected the output and were able to respond to the runtime errors. I’d be happy to share the code if anyone wants to surprise me.
This is exactly the kind of semi-mechanical, low added value work that would greatly benefit from automation, and they really fell on their faces. I really benefit from these models on greenfield tasks where I can delegate minor drudge work, but in this case I honestly think they actually increased the difficulty.
- ikiris 2y agoAs a counter to this, I had grok build an entire set of micro services and all I had to clean up was some format strings. It blew me away. Did in an hour what should have taken a week.
- hu3 2y agonice. did you use grok web interface or some ide + grok?
- ikiris 2y agoI use cursor, and if there’s a nice interface for cursor to use grok I don’t know about it. Which leads to pain if I try to have the built in options try to fix things because they’re nowhere near as competent currently.
- raffael_de 2y agoi'm not sure if you can compare run of the mill micro services with "uncommented academic modeling code from Matlab".
- TeMPOraL 2y agoExactly. "uncommented academic modeling code from Matlab" is something very few people ever do, and the importance of that work is negligible in isolation. Meanwhile, "run of the mill micro services" are like most of what the software industry is doing these days; in aggregate, it's also how the industry delivers most of its value and how it makes most of its money. It's the CRUD thing all over again.
- samsin 2y agoCan you share the code it produced?