4 ms·

The "upside" description: On the other you have a non-technical executive who's got his head round Claude Code and can run e.g. Python locally. I helped o

by defrost 9mo ago

The "upside" description: On the other you have a non-technical executive who's got his head round Claude Code and can run e.g. Python locally. I helped one recently almost one-shot converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. Once the model is in Python, you effectively have a data science team in your pocket with Claude Code. You can easily run Monte Carlo simulations, pull external data sources as inputs, build web dashboards and have Claude Code work with you to really integrate weaknesses in your model (or business). It's a pretty magical experience watching someone realise they have so much power at their fingertips, without having to grind away for hours/days in Excel. almost makes me physically sick. I've a reasonably intense math background corrupted by application to geophysics and implementing real world numerical applications. To be fair, this statement alone: * 30 sheet mind numbingly complicated Excel financial model makes my skin crawl and invokes a flight reflex. Still, I'll concede that a Claude Code conversion to Python of a 30 sheet Excel financial model is unlikely to be significantly worse than the original.

decimalenough 9mo ago
I'm almost certain it will be significantly worse. The Excel sheet will have been tuned over the years by people who knew exactly what it was doing and fixed countless bugs along the way. The Claude Code copy will be a simulacrum that may behave the same way with some inputs, but is likely to get many of edge cases wrong, and, when you're talking about 30 sheets of Excel, there will be many, many of these sharp edges.
- defrost 9mo ago
  I won't disagree - I suffered from insufficient damning praise in my last sentence above. IMHO, earned through years of bleeding eyeballs, the first will be riddled with subtle edge cases curiously patched and fettled such that it'll limp through to the desired goal .. mostly. The automated AI assisted transcoding will be ... interesting.
- holoduke 9mo ago
  My assumption is that with the right approach you can create a much much better and reliable program using only Claude code. You are referring to yolo coding results
majormajor 9mo ago
One of the dirty secrets of a lot of these "code adjacent" areas is that they have very little testing. If a data science team modeled something incorrectly in their simulation, who's gonna catch it? Usually nobody. At least not until it's too late. Will you say "this doesn't look plausible" about the output? Or maybe you'll be too worried about getting chided for "not being data driven" enough. If an exec tells an intern or temp to vibecode that thing instead, then you definitely won't have any checkpoints in the process to make sure the human-language prompt describing process was properly turned into the right simulation. But unlike in coding, you don't have a user-facing product that someone can click around in, or send requests to, and verify. Is there a test suite for the giant excel doc? I'm assuming no, maybe I'm wrong. It feels like it's going to be very hard for anyone working in areas with less black-and-white verifiability or correctness like that sort of financial modeling.
- tharkun__ 9mo ago
  This is a pet peeve of mine at work. Any and I mean any statistic someone throws at me I will try and dig in. And if I'm able to, I will usually find that something is very wrong somewhere. As in, the underlying data is usually just wrong, invalidating the whole thing or the data is reasonably sound but the person doing the analysis is making incorrect assumptions about parts of the data and then drawing incorrect conclusions.
  - aschla 9mo ago
    It seems to be an ever-present trait of modern business. There is no rigor, probably partly because most business professionals have never learned how to properly approach and analyze data. Can't tell you how many times I've seen product managers making decisions based on a few hundred analytics events, trying to glean insight where there is none.
    - gyomu 9mo ago
      If what you're saying 1) is true and 2) does matter in the success of a business, then wouldn't anyone be able to displace an incumbent trivially by applying a bit of rigor? I think 1) holds (as my experience matches your cynicism :), but I have a feeling that data minded people tend to overestimate the importance of 2)...
ChrisMarshallNY 9mo ago
Obligatory xkcd: https://xkcd.com/1667/ https://xkcd.com/1667/
bitwize 9mo ago
The thing is, when you use AI, you're not really doing things, you're having things done. AI isn't a tool, it's a service. Now, back in the day, IBM designed and built an "executive data terminal". It wasn't really a computer terminal in the sense that you and I understand it. Rather, it was a video and two-way-audio feed to a room with a team of underlings, which an executive could ask for business data and analyses, which could be called up on a computer display (also routed to the executive's office). This allowed the executive to ask questions so he (it was the 1960s, it was almost invariably a he) could make informed decisions, and the team of underlings to call up data or crunch numbers on the computer and show the results on the display. So because executives are used to having things done for them, I can totally see AI being used by executives to replace the "team of underlings" in this setup—in principle. The fact is that were I in that CEO's chair, I'd be thinking twice before trusting anything an LLM tells me, and double-checking those results—perhaps with my team of underlings. Discussed on Hackernews: https://news.ycombinator.com/item?id=42405462 https://news.ycombinator.com/item?id=42405462 IEEE article: https://spectrum.ieee.org/ibm-demo https://spectrum.ieee.org/ibm-demo
- chrisjj 9mo ago
  > be thinking twice before trusting anything an LLM tells me You're too modest. You'd be thinking once. However when the parrot is hidden in a shiny box made up to look like a regular, relatively trustworthy program...
PunchyHamster 9mo ago
we're going from "bad excel sheet caused recession" to "bad vibe-coded financial thing caused recession"
- ruszki 9mo ago
  I heard already a story like that. My home country’s government made already some decisions based on some vibe code verified by nobody. It was made by one of my friends. Nobody cares.
simonebrunozzi 9mo ago
> Still, I'll concede that a Claude Code conversion to Python of a 30 sheet Excel financial model is unlikely to be significantly worse than the original. You made me laugh hard. :)
xmcqdpt2 9mo ago
I work in finance and we have prod excel spreadsheets. Those spreadsheets are versioned like code artifacts, with automated testing and everything. Converting them to real applications is a major part of the work the technology division does. They usually happen because some new and exciting line of business is started by a small team as a POC. Those teams don't get full technology backing, it would slow down the early iteration and cost a lot of money for an idea that may not be lucrative. Eventually they make a lot of money and by then risk controls are basically requiring them to document every single change they make in excel. This eventually sucks enough that they complain and get a tech team to convert the spreadsheet.
- defrost 9mo ago
  I too have seen such things. My experience being they are an exception rather than the rule and many more businesses have sheets that tend further toward Heath Robinson than would be admitted in public. * https://en.wikipedia.org/wiki/W._Heath_Robinson https://en.wikipedia.org/wiki/W._Heath_Robinson
biophysboy 9mo ago
Ha! I also have a physics background and had the same gag reflex.
theshrike79 9mo ago
At least the Python version can be verified with existing static analysis tooling and can have comprehensive unit tests. The Excel never had any tests and people just trusted it.

by defrost 9mo ago

decimalenough 9mo ago

I'm almost certain it will be significantly worse. The Excel sheet will have been tuned over the years by people who knew exactly what it was doing and fixed countless bugs along the way. The Claude Code copy will be a simulacrum that may behave the same way with some inputs, but is likely to get many of edge cases wrong, and, when you're talking about 30 sheets of Excel, there will be many, many of these sharp edges.

defrost 9mo ago
I won't disagree - I suffered from insufficient damning praise in my last sentence above. IMHO, earned through years of bleeding eyeballs, the first will be riddled with subtle edge cases curiously patched and fettled such that it'll limp through to the desired goal .. mostly. The automated AI assisted transcoding will be ... interesting.
holoduke 9mo ago
My assumption is that with the right approach you can create a much much better and reliable program using only Claude code. You are referring to yolo coding results

majormajor 9mo ago

One of the dirty secrets of a lot of these "code adjacent" areas is that they have very little testing. If a data science team modeled something incorrectly in their simulation, who's gonna catch it? Usually nobody. At least not until it's too late. Will you say "this doesn't look plausible" about the output? Or maybe you'll be too worried about getting chided for "not being data driven" enough. If an exec tells an intern or temp to vibecode that thing instead, then you definitely won't have any checkpoints in the process to make sure the human-language prompt describing process was properly turned into the right simulation. But unlike in coding, you don't have a user-facing product that someone can click around in, or send requests to, and verify. Is there a test suite for the giant excel doc? I'm assuming no, maybe I'm wrong. It feels like it's going to be very hard for anyone working in areas with less black-and-white verifiability or correctness like that sort of financial modeling.

tharkun__ 9mo ago
This is a pet peeve of mine at work. Any and I mean any statistic someone throws at me I will try and dig in. And if I'm able to, I will usually find that something is very wrong somewhere. As in, the underlying data is usually just wrong, invalidating the whole thing or the data is reasonably sound but the person doing the analysis is making incorrect assumptions about parts of the data and then drawing incorrect conclusions.
- aschla 9mo ago
  It seems to be an ever-present trait of modern business. There is no rigor, probably partly because most business professionals have never learned how to properly approach and analyze data. Can't tell you how many times I've seen product managers making decisions based on a few hundred analytics events, trying to glean insight where there is none.
  - gyomu 9mo ago
    If what you're saying 1) is true and 2) does matter in the success of a business, then wouldn't anyone be able to displace an incumbent trivially by applying a bit of rigor? I think 1) holds (as my experience matches your cynicism :), but I have a feeling that data minded people tend to overestimate the importance of 2)...

ChrisMarshallNY 9mo ago

Obligatory xkcd: https://xkcd.com/1667/ https://xkcd.com/1667/

bitwize 9mo ago

The thing is, when you use AI, you're not really doing things, you're having things done. AI isn't a tool, it's a service. Now, back in the day, IBM designed and built an "executive data terminal". It wasn't really a computer terminal in the sense that you and I understand it. Rather, it was a video and two-way-audio feed to a room with a team of underlings, which an executive could ask for business data and analyses, which could be called up on a computer display (also routed to the executive's office). This allowed the executive to ask questions so he (it was the 1960s, it was almost invariably a he) could make informed decisions, and the team of underlings to call up data or crunch numbers on the computer and show the results on the display. So because executives are used to having things done for them, I can totally see AI being used by executives to replace the "team of underlings" in this setup—in principle. The fact is that were I in that CEO's chair, I'd be thinking twice before trusting anything an LLM tells me, and double-checking those results—perhaps with my team of underlings. Discussed on Hackernews: https://news.ycombinator.com/item?id=42405462 https://news.ycombinator.com/item?id=42405462 IEEE article: https://spectrum.ieee.org/ibm-demo https://spectrum.ieee.org/ibm-demo

chrisjj 9mo ago
> be thinking twice before trusting anything an LLM tells me You're too modest. You'd be thinking once. However when the parrot is hidden in a shiny box made up to look like a regular, relatively trustworthy program...

PunchyHamster 9mo ago

we're going from "bad excel sheet caused recession" to "bad vibe-coded financial thing caused recession"

ruszki 9mo ago
I heard already a story like that. My home country’s government made already some decisions based on some vibe code verified by nobody. It was made by one of my friends. Nobody cares.

simonebrunozzi 9mo ago

> Still, I'll concede that a Claude Code conversion to Python of a 30 sheet Excel financial model is unlikely to be significantly worse than the original. You made me laugh hard. :)

xmcqdpt2 9mo ago

I work in finance and we have prod excel spreadsheets. Those spreadsheets are versioned like code artifacts, with automated testing and everything. Converting them to real applications is a major part of the work the technology division does. They usually happen because some new and exciting line of business is started by a small team as a POC. Those teams don't get full technology backing, it would slow down the early iteration and cost a lot of money for an idea that may not be lucrative. Eventually they make a lot of money and by then risk controls are basically requiring them to document every single change they make in excel. This eventually sucks enough that they complain and get a tech team to convert the spreadsheet.

defrost 9mo ago
I too have seen such things. My experience being they are an exception rather than the rule and many more businesses have sheets that tend further toward Heath Robinson than would be admitted in public. * https://en.wikipedia.org/wiki/W._Heath_Robinson https://en.wikipedia.org/wiki/W._Heath_Robinson

biophysboy 9mo ago

Ha! I also have a physics background and had the same gag reflex.

theshrike79 9mo ago

At least the Python version can be verified with existing static analysis tooling and can have comprehensive unit tests. The Excel never had any tests and people just trusted it.