13 ms·

I’m worried that they put co-pilot in Excel

494 points by isaacfrond 1y ago

cjs_ac 1y ago
At some point, a publicly-listed company will go bankrupt due to some catastrophic AI-induced fuck-up. This is a massive reputational risk for AI platforms, because ego-defensive behaviour guarantees that the people involved will make as much noise as they can about how it's all the AI's fault.
- ramon156 1y ago
  Do you really want these kind of companies to succeed? Let them burn tbh
  - cjs_ac 1y ago
    I don't find comments along the lines of 'those people over there are bad' to be interesting, especially when I agree with them. My comment is about why it'll go wrong for them.
  - mcphage 1y ago
    Make sure you’re not part of the kindling, then.
- meibo 1y ago
  That will never happen, AI cannot be allowed to fail, so we'll be paying for that AI bail-out.
- gosub100 1y ago
  I see the inverse of that happening: every critical decision will incorporate AI somehow. If the decision was good, the leadership takes credit. If something terrible happens, blame it on the AI. I think it's the part no one is saying out loud. That AI may not do a damn useful thing, but it can be a free insurance policy or surrogate to throw under the bus when SHTF.
  - halfcat 1y ago
    This works at most one time. If you show up to every board meeting and blame AI, you’re going to get fired. This is true if you blame a bad vendor, or something you don’t even control like the weather. Your job is to deliver. If bad weather is the new norm, you better figure out how to build circus tents so you can do construction in the rain. If your AI call center is failing, you better hire 20 people to answer phones.
AmbroseBierce 1y ago
Brenda has been getting slower over the years -as we all have-, but soon the boss will learn that it was a small price to pay for knowing well how to keep such house of cards from collapsing.
- Simulacra 1y ago
  And then the boss will make the decision to outsource her job, to a company that promises the use of AI to make finance better, and faster, and while Brenda is in the unemployment line, someone else thousands of miles away is celebrating a new job
  - gadflyinyoureye 1y ago
    We are setting AI deployed in the US, but actually Indians. They are not better, but they are cheaper. They are probably worse, but they are cheaper.
Traster 1y ago
I'm actually not that worried about this, because again I would classify this as a problem that already exists. There are already idiots in senior management who pass off bullshit and screw things up. There are natural mechanisms to cope with this, primarily in business reputation - if you're one of those idiots who does this people very quickly start just discounting what you're saying, they might not know how you're wrong, but they learn very quickly to discount what you're saying because they know you can't be trusted to self-check. I'm not saying that this can't happen and it's not bad. Take a look at nudge theory - the UK government created an entire department and spent enormous amounts of time and money on what they thought was a free lunch - that they could just "nudge" people into doing the things they wanted. So rather than actually solving difficult problems the uk government embarked on decades of pseudo-intellectual self agrandizement. The entire basis of that decades long debacle was based on bullshit data and fake studies. We didn't need AI to fuck it up, we managed it perfectly well by ourselves.
- gmac 1y ago
  Nudge theory isn't useless, it's just not anything like as powerful as money or regulation. It was taken up by the UK government at that time because the government was, unusually, a coalition of two quite different parties, and thus found it hard to agree to actually use the normal levers of power. This NY Times opinion piece by Loewenstein and Ubel makes some good arguments along these lines: https://web.archive.org/web/20250906130827/https://www.nytimes.com/2010/07/15/opinion/15loewenstein.html https://web.archive.org/web/20250906130827/https://www.nytim...
simonw 1y ago
This quote is pulled from a TikTok, I recommend watching the whole thing here: https://www.tiktok.com/@belligerentbarbies/video/7568380008633257271 https://www.tiktok.com/@belligerentbarbies/video/75683800086... (I pulled the quote by using yt-dlp to grab the MP4 and then running that through MacWhisper to generate a transcript.)
- donatj 1y ago
  It's a little over two paragraphs. Seems like it would have been simpler just to... type it out?
  - adlpz 1y ago
    Where's the fun in that? :D
    - mavhc 1y ago
      We choose to automate these things, not because they are easy, but because they are an interesting problem to solve
  - daliusd 1y ago
    Well if you do it once then yes, but if you automate this process it is different. E.g. I do this with YouTube videos, because watching 14 minutes video or reading 30 seconds summary is time saver. I still watch some videos fully, but many of them are not worth it. So in summary I think it was just part of automated process (maybe) or it will become one in the future.
  - rererereferred 1y ago
    But then you would need a Brenda. Ai can write the automation script for you.
  - simonw 1y ago
    Why spend two minutes typing (and realistically longer than that, if I want to capture the exact transcript I would need to keep hitting pause and play and correcting myself) when I can spend ten seconds pasting a URL into my terminal and then dragging and dropping the resulting file onto the MacWhisper window? I actually transcribed the whole TikTok which was about 50% longer than what I quoted, then edited it down to the best illustrative quote.
HeavyStorm 1y ago
Nay-sayers need to decide whether they fear AI because AI is dumb and will fuckup or because AI is smart and will take over.
- tossandthrow 1y ago
  Simon willson is definitely not a nay sayer.
- 9dev 1y ago
  Both are valid concerns, no need to decide. Take the USA: They are currently lead by a patently dumb president who fucks up the global economy, and at the same time they are powerful enough to do so! For a more serious example, consider the Paperclip Problem[0] for a very smart system that destroys the world due to very dumb behaviour. [0]: https://cepr.org/voxeu/columns/ai-and-paperclip-problem https://cepr.org/voxeu/columns/ai-and-paperclip-problem
  - RajT88 1y ago
    The paperclip problem is a bit hand-wavey about intelligence. It is taken as a given than unlimited intelligence would automatically win presumably because it could figure out how to do literally anything. But let's consider real life intelligence: - Our super geniuses do not take over the world. It is the generationally wealthy who do. - Super geniuses also have a tendency to be terribly neurotic, if not downright mentally ill. They can have trouble functioning in society. - There is no thought here about different kinds of intelligence and the roles they play. It is assumed there is only one kind, and AI will have it in the extreme.
    - 9dev 1y ago
      To be clear, I don't think the paperclip scenario is a realistic one. The point was that it's fairly easy to conceive an AI system that's simultaneously extremely savant and therefore dangerous in a single domain, yet entirely incapable of grasping the consequences or wider implications of its actions. None of us knows what an actual, artificial intelligence really looks like. I find it hard to draw conclusions from observing human super geniuses, when their minds may have next to nothing in common with the AI. Entirely different constraints might apply to them—or none at all. Having said all that, I'm pretty sceptical of an AI takeover doomsday scenario, especially if we're talking about LLMs. They may turn out to be good text generators, but not the road to AGI. But it's very hard to make accurate predictions in either direction.
glimshe 1y ago
This reminds me of a friend whose company ran a daily perl script that committed every financial transaction of the day to a database. Without the script, the company could literally make no money irrespectively of sales because this database was one piece in a complex system for payment processor interoperability. The script ran in a machine located at the corner of a cubicle and only one employee had the admin password. Nobody but a handful of people knew of the machine's existence, certainly not anyone in middle management and above. The script could only be updated by an admin. Copilot may be good, but sure as hell doesn't know that admin password.
- danielbln 1y ago
  If your mission critical process sits on some on-site box that no-one knows about, copilot being good or not is the least of your problems.
  - maccard 1y ago
    Everywhere I’ve ever worked has had that mission critical box. At one of my jobs we had a server rack with UPS, etc, all the usual business. On the floor next to it was a dell desktop with a piece of paper on it that said “do not turn off”. It had our source control server in it, and the power button didn’t work. We did eventually move it to something more sensible but we had that for a long time
    - victorbjorklund 1y ago
      with only one person on earth being able to access it? so if that person is hit by a car everything goes down?
      - glimshe 1y ago
        Pretty much
      - maccard 11mo ago
        yeah. I mean, someone else would _eventually_ figure it out. There wasn't full disk encryption or anyhting on it, so if the guy got hit by a bus, and the machien turned off we probably would have just imaged the disk and got it running in a VM. But we didn't (and nobody was hit by a bus)
teekert 1y ago
Don't worry, in Teams it bothers me just one time a day, and with the click of a button it's gone... For another whole day.
mikert89 1y ago
10 billion dollars is probably going to be spent on automating excel, it’s going to happen
- harryf 1y ago
  There needs to a financial equivalent to the Mythical Man Month.
  - graemep 1y ago
    There are plenty of things that play the role. The problem is that people ignore them.
eithed 1y ago
Let it all crash and burn
Dumblydorr 1y ago
Co-pilot and AI has been shoved at the Microsoft Stack in my org for months. Most of the features were disabled or hopelessly bad. It’s cheaper for Microsoft to push this junk and claim they’re doing something, it’s going to improve their stock far more than not doing it, even though it’s basically useless currently. Another issue is that my org disallows AI transcription bots. It’s a legit security risk if you have some random process recording confidential info because the person was too busy to attend the meeting and take notes themselves. Or possibly they just shirk off the meetings and have AI sit in.
- aDyslecticCrow 1y ago
  Transcription is arguably one of the must useful enterprise AI tools avaliable. But i sure as heck wouldn't trust the cloud with it.
  - 2dvisio 1y ago
    Still find the Copilot transcripts orders of magnitude worse than something like Wispr Flow and they tend to allucinate constantly and do not adapt to a company's context (that Copilot has access too...). I am talking about acronyms of products / teams, names of people (even when they are in the call), etc.
    - srean 1y ago
      Can anyone familiar with the technical details shed light on why this is so. Is it because of a globally trained model (as opposed to trained[tweaked on] on context specific data) or because of using different classes of models.
      - aDyslecticCrow 1y ago
        Neither copilot nor flow can natively handle audio to my understanding, so there is already a transcription model converting it to text that then GPT tries to summarise. It could be they simply use a mediocre transcription model. Wispr is amazing but would hurt their pride to use a competitor. But i feel it's more likley the experience is; GPT didn't actually improve on the raw transcription, just made it worse. Especially as any miss-transcipted words may trip it up and make it misunderstand while making the summary. if i can choose between a potentially confused and misunderstood summary, and a badly spellchecked (flipped words) raw transcription, i would trust the latter.
Esophagus4 1y ago
Hmmm the Brendas I know look a little different. “There are two Brendas - their job is to make spreadsheets in the Finance department. Well, not quite - they add the months and categories to empty spreadsheets, then they ask the other departments to fill in their sales numbers every month so it can be presented to management. “The two Brendas don’t seem to talk, otherwise they would realize that they’re both asking everyone for the same information, twice. And they’re so focused on their little spreadsheet worlds that neither sees enough of the bigger picture to say, ‘Wait… couldn’t we just automate this so we don’t need to do this song and dance every month? Then we wouldn’t need two people in different parts of the company compiling the same data manually.’ “But that’s not what Brenda was hired for. She’s a spreadsheet person, not a process fixer. She just makes the spreadsheets.” We need fewer Brendas, and more people who can automate away the need for them.
- SirFatty 1y ago
  "We need fewer Brendas, and more people who can automate away the need for them." True... I have an on-staff data engineer for the purpose. But not all companies (especially in the SMB space) have that luxury.
- 7thaccount 1y ago
  That's a pretty specific example when there are a lot of good "spreadsheet people" out there who do a lot more than spreadsheets (maybe they had to write SQL queries or scripts to get those numbers), but commonly need to simplify things down to a spreadsheet or power point for upper management. I'm not saying you should have multiple people doing redundant work, but this style isn't entirely dumb. What would this be replaced by? Some kind of large SAP like system that costs millions of dollars and requires a dozen IT staff to maintain?
  - Esophagus4 1y ago
    Fair - I was creating a straw man mostly to make a point. The people I’m thinking aren’t running SQL queries or scripts, they’re merely collection points for data. So one good BI developer who knows Tableau and Salesforce and Excel and SQL can replace those pure collection points with a better process, but they can also generate insight into the data because they have some business understanding from being close to the teams, which is what my hypothetical Brenda can’t do. In my example, Brenda would be asking sales leaders to enter in their data instead of going into Salesforce herself because she doesn’t know that tool / side of the company well enough. I was making the point that, contrary to the article, the Brendas I know aren’t touched by the Excel angels, they’re just maintaining spreadsheets that we probably shouldn’t have anyway.
motoboi 1y ago
Excel is the “beast that drives the ENTIRE economy” and he’s worried about Brenda from the finance department losing her job because then her boss will get bad financial reports I suppose the person that wrote that have not ideia Excel is just an app builder where you embed data together with code. You know that we have excel because computers didn’t understand column names in databases and so data extraction needed to be made by humans. Humans then design those little apps in excel to massage the data. Well, now an agent can read the boss saying gimme the sales from last month and the agent don’t need excel for that, because it can query the database itself, massage the data itself using python and present the data itself with html or PNGs. So, we are in the process of automating Brenda AND excel away. Also, finance departments are a very small part of excel users. Just think everywhere were people need small programs, excel is there.
- evolve2k 1y ago
  You missed this bit “.. and then the AI is gonna fuck it up real bad and he won't be able to recognize it because he doesn't understand because AI hallucinates.”
  - brazukadev 1y ago
    Brendas have fucked it up multiple times, by themselves or because their boss demanded
    - Forgeties79 1y ago
      The underlying assumption is that Brenda generally does her job pretty well. Human errors exist but usually peers/managers (or the person who did it) can identify and correct them reliably. If we have to compare LLM’s against people who are bad at their jobs in order to highlight their utility we’re going the wrong direction.
      - Telemakhos 1y ago
        There are a lot of underlying assumptions: Brenda, the woman, is accurate and trustworthy and has mastered an accurate and trustworthy technology; the upper manager, the male, will introduce error by not understanding that the technology he brings to bear on the situation is hallucinatory. The woman is lower in status and pay than the male. The woman is necessary to the functioning of "the economy" and "capitalism," while the man threatens those. There are a lot of unsubtle political undertones on TikTok.
fancyfredbot 1y ago
This is transparent nonsense. People are very very happy to introduce errors into excel spreadsheets without any help from AI. Financial statements are correct because of auditors who check the numbers. If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit).
- tl 1y ago
  > If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit). Volume matters. The single largest problem I run into: AI can generate slop faster than anyone can evaluate it.
  - fancyfredbot 11mo ago
    If nobody can evaluate it then nobody will sign it off.
- svnt 1y ago
  It’s like calling out the county to inspect the home you built but when they arrive it’s a bouncy castle.
runako 1y ago
"the sweat from Brenda's brow is what allows us to do capitalism." The CEO has been itching to fire this person and nuke her department forever. She hasn't gotten the hint with the low pay or long hours, but now Copilot creates exactly the opening the CEO has been looking for.
quixoticaxolotl 1y ago
I find the contrast between two narratives around technology use so fascinating: 1. We advocate automation because people like Brenda are error-prone and machines are perfect. 2. We disavow AI because people like Brenda are perfect and the machine is error-prone. These aren't contradictions because we only advocate for automation in limited contexts: when the task is understandable, the execution is reliable, the process is observable, and the endeavour tedious. The complexity of the task isn't a factor - it's complex to generate correct machine code, but we trust compilers to do it all the time. In a nutshell, we seem to be fine with automation if we can have a mental model of what it does and how it does it in a way that saves humans effort. So, then - why don't people embrace AI with thinking mode as an acceptable form of automation? Can't the C-suite in this case follow its thought process and step in when it messes up? I think people still find AI repugnant in that case. There's still a sense of "I don't know why you did this and it scares me", despite the debuggability, and it comes from the autonomy without guardrails. People want to be able to stop bad things before they happen, but with AI you often only seem to do so after the fact. Narrow AI, AI with guardrails, AI with multiple safety redundancies - these don't elicit the same reaction. They seem to be valid, acceptable forms of automation. Perhaps that's what the ecosystem will eventually tend to, hopefully.
- deleted 1y ago
  [deleted]
- Aeolun 1y ago
  > We disavow AI because people like Brenda are perfect and the machine is error-prone. No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda.
  - misnome 1y ago
    “Let’s deploy something as or more error prone as Brad at infinite scale across our organisation”
  - candiddevmike 1y ago
    I don't understand why generative AI gets a pass at constantly being wrong, but an average worker would be fired if they performed the same way. If a manager needed to constantly correct you or double check your work, you'd be out. Why are we lowering the bar for generative AI?
Havoc 1y ago
That mirrors my experience as well. LLMs get instantly confused in real world scenarios in Excel and confidently hallucinate millions in errors If you look at the demos for these it’s always something that is clean and abundantly available in training data. Like an income statement. Or a textbook example DCF. Or my personal fav „here is some data show me insights“. Real world excel use looks nothing like that. I’m getting some utility out of them for some corporate tasks but zilch in excel space.
- lxgr 1y ago
  As somebody with non-existent experience with Excel, I could totally see myself getting a lot of value out of LLMs, if nothing else then simply for telling me what's possible, what functions and patterns exist at all etc.
  - Havoc 11mo ago
    Yeah definitely has some value in that sense. That in itself isn't enough to make a dent in the work though. Think of it this way - an IDE can tell you what functions an object has or autocomplete something is useful to a beginner & learning. But that's not what puts food on the programmers table - writing code that solves real problems does. Same in excel business use cases - the numbers and formulas don't matter directly - their meaning in a business context does. And that connection can be very tenuous. With code the compiler is the ultimate arbiter - it has to make sense on that level. Excel files it's all freestyle - it could be anything from your grandmas shopping list to a model that runs half a bank.
diego_sandoval 1y ago
I'm more shocked that someone is using TikTok to speak things that actually make sense instead of mindless memes.
d--b 1y ago
It looks like the OP is thinking that AI causing errors in spreadsheets is going to make the whole economy collapse. When tools break, people stop using them before they sink the ship down. If AI is that terrible at spreadsheet, people will just revert to Brenda. And it's not like spreadsheets have no errors right now.
133361096 1y ago
320784788
jwsteigerwalt 1y ago
Many fears of “AI mucking it up” could be mitigated with an ability to connect a workbook to a git repository. Not for data, but for VBA, cell formulas, and cell metadata. When you can encapsulate the changes a contributor (in this case co-pilot) makes into a commit, you can more easily understand what changes it/they made.
- smoothbran 11mo ago
  Even if "teaching the rest of the workforce how to use Git" wasn't a massive obstacle, many (most?) corporate workflows involve editing files that weren't designed to be human-readable. There are a couple of approaches out there for comparing diffs to excel spreadsheets specifically, but it's not exactly pleasant.
  - jwsteigerwalt 11mo ago
    Considering that Microsoft perfected change control in the legal industry (ms word’s track changes), something can be done to add a degree of change control to spreadsheets.
alienbaby 1y ago
Using ai does not absolve you from the responsibility of doing it correctly. If you use ai, then you better have the skills to have done the job yourself, and so have the ability to check the AI did things correctly. You can save time still, but perhaps not as much as you think, because you need to check the ai's work thoroughly.
b3lvedere 1y ago
"You know who's not hallucinating? Brenda" I don't know about that. There could be lots of interesting ways Brenda can (be convinced to) hallucinate.
- giarc 1y ago
  I agree - having watched many people use Excel over the years, I'd say people often overestimate their skills. I see three categories of Excel users. First there are the people that are intimidated by it and stay away from any task involving Excel. Second are the people that know a little bit (a few basic formulas) and overestimate their skills because they only compare themselves to the first group. And the third group are the actual power users but know to keep that quiet because otherwise they become the "excel person" and have to fix every sheet that has issues. I don't know if AI is going to make any of the above better or worse. I expect the only group to really use it will be that second group.
  - b3lvedere 1y ago
    I have seen lots and lots of different uses for Excel in my line of work: - password database - script to automatically rename jpeg files - game - grocery lists - Book keeping (and try and not get caught for fraud several years, because the monthly spending limit is $5000 and $4999 a month is below that...) - embed/collect lots of Word documents - coloring book - Minecraft processes - Resume database - ID scans
    - recursive 11mo ago
      I've seen it used for laser testing. Not tracking test results. Testing lasers.
Zigurd 1y ago
It's verifier law. Coding agents are useful and good and real products because when they screw up, things stop working almost always before they can do damage. Coding agents are flawed in ways that existing tools are good at catching, never mind the more obvious build and runtime errors. Letting AI write your emails and create your P&L and cash flow projections doesn't have to run the gauntlet of tools that were created to stop flawed humans from creating bad code.
- phyzome 1y ago
  Nah, I've seen them screw in all sorts of ways that would fail in some conditions and not others. You're way too optimistic about this.
  - Zigurd 1y ago
    Fair. I've been using the coding agent in Android Studio Canary to do exploratory code in Dart/Flutter and using ATProto. Low stakes, but higher productivity is a significant benefit. It's a daily surprise how brilliant it is it's some things and how abysmal at others.
- insane_dreamer 11mo ago
  Another problem is that the AI may try to fudge the numbers to mask its mistakes so they look like they all add up while the rot is hidden away. Just like it tries to manipulate the unit tests so they pass without fixing the _actual_ bug. I've seen it happen.
intended 1y ago
Everything is now about verification. AI may be able to spit out ann excel sheet or formula - But if it can’t be verified, so what ? And here’s my analogy to think about the debugging of an excel sheet - you can debug most corporate excel sheets with a calculator. But when AI is spitting out excel sheets - when the program is making smaller programs - what is the calculator in this analogy ? Are we going to be using excel sheets to debug the output of AI? I think this is the inherent limiter to the uptake of AI. There’s only so much intellectual / experiential / training depth present. And now we’re going to be training even fewer people. At the end of the day I /customers need something to work. But failing that - I will settle for someone to blame. Brenda handles a lot of blame. Is OpenAI going to step into that gap ?
byyoung3 1y ago
Brendas hallucinate all the time.
skeptrune 1y ago
Simon posting tiktok quotes on his blog was not on my 2025 bingo card.
- simonw 1y ago
  This isn't the first: https://simonwillison.net/2025/Aug/8/pearlmania500/ https://simonwillison.net/2025/Aug/8/pearlmania500/ and https://simonwillison.net/2024/Jul/29/dealing-with-your-ai-obsessed-co-worker-tiktok/ https://simonwillison.net/2024/Jul/29/dealing-with-your-ai-o... Also this fun diversion into Occlupanids: https://simonwillison.net/2024/Dec/8/holotypic-occlupanid-research-group/ https://simonwillison.net/2024/Dec/8/holotypic-occlupanid-re... A lot of people complain that the internet isn't as weird and funny as it used to be. The weird and funny stuff is all on TikTok!
  - andybak 1y ago
    The mismatch between what people not on it think TikTok is like and what it's actually like (once you get the algo tuned to your taste) is pretty crazy. But then the "new user" experience is so horrific in terms of the tacky default content it serves you that I'm not surprised so many people don't get past it.
    - hunterpayne 11mo ago
      This is something I think all short form video platforms struggle with. I think I know why. Its because the difference in the UI. Basically, a user doesn't choose what to see next. On YouTube, you get dozens of next videos recommended on each view. On short form video, you get one. This causes clickbait to work a lot better for short form video. The problem with platforms like TikTok is more about UI than algorithm or the length of the videos.
thisisit 1y ago
Don't be like that. I work at a Fortune 500 and Brenda wants that co-pilot in Excel because it can help her achieve so much more. What is so much more you ask? Brenda and her C-Suits can not define it but they know for sure Copilot in excel will lead to enormous time saving.
sherinjosephroy 1y ago
[dead]
like_any_other 1y ago
Excel doesn't need AI to ruin your work: https://www.science.org/content/article/one-five-genetics-papers-contains-errors-thanks-microsoft-excel https://www.science.org/content/article/one-five-genetics-pa...
slipperybeluga 1y ago
[dead]
__mharrison__ 1y ago
Excel is the most popular programming environment in the universe. It has optimized the five minute out of the box experience so well that grade schoolers can use it. Other than that, it is pretty horrible for coding.
cssinate 11mo ago
I've partied with Brenda on the weekends, and let me tell you... SOMETIMES Brenda hallucinates. But never during work hours. The woman's a saint M-F.
danjl 11mo ago
Excel is programming. Spreadsheets have been full of bugs for decades. How is Brenda any different from a developer? Why are people scared when the LLM might affect their dollar calculations, and less bothered when it affects their product?
- simonw 11mo ago
  +100 this. Programmers who work in Excel (and never even dream of calling themselves programmers) are still programmers.
  - bdangubic 11mo ago
    I know couple of them that get paid C-level money too :)
blitzar 11mo ago
another cheese that will affect the outcome of major tournaments, not a good look for microsoft its like the xlookup situation all over again, yet another move aimed at the casual audience, designed to bring in the party gamers and make the program an absolute mess competitively
NumberCruncher 11mo ago
Is this not the guy who is on the payroll of Anthropic? Not because he is wrong, but because there is so much marketing going on in this space nowadays.
- simonw 11mo ago
  Who, me? I'm still independent - I have a disclosures section on my blog here: https://simonwillison.net/about/#disclosures https://simonwillison.net/about/#disclosures Anthropic sometimes give me free credits (so I can try out preview features) and gave me a ticket to their conference a few months ago.
manuelz 11mo ago
I actually know Brenda.
newscracker 11mo ago
I feel what this article says based on some recent (non-catastrophic) experiences. I think I’m probably an above average user when it comes to Excel skills. I love spreadsheets. But I struggle with formulas like index, match, vlookup/xlookup and many others, and even more so when it requires nesting one within another and coming up with the underlying logic that leads to some complex nested formulas. Over the past couple of months, I’ve tried some smaller models on duck.ai and also ChatGPT directly to create some columns and formulas for a specific purpose. I found that ChatGPT is a lot better than the “mini” models on duck.ai. But in all these cases, though these platforms seemed more capable than me and could make attempts to explain their formulas, they were many a times creating junk and “looping” back with formulas that didn’t really work. I had to point out the result (blank or some #REF or other error) multiple times and they would acknowledge that there’s an issue and provide a working formula. That wouldn’t work either! I really love that these LLMs can sort of “understand” what I’m asking, break it down in English, and provide answers. But the end result has been an exercise in frustration and waste of time. Initially I really thought and believed that LLMs could make Excel more approachable and easier to use — like you tell it what you want and it’ll figure it out and give the magic incantations (formulas). Now I don’t think we’re anywhere close to that if ChatGPT (which I presume powers Copilot as well) struggles and hallucinates so much. I personally don’t have much hope with the (comparatively) smaller and older models.
deburo 11mo ago
Luckily this is a capitalist society and usually mistakes in the private market resolve themselves because losing money is not a winning strategy.
throwmeaway307 11mo ago
I'm worried Excel will go "enterprise only". and only LLM based interfaces will be enabled on the "office+windows" for consumers tier. e.g. MS Access is well on its way. as soon as x86 gets fully overtaken by ARM, and LLMs overtake "compilers" (also taken enterprise only).. then things like sqlite-browsers (FOSS "access") will be an arcane tool of binary incompatible ("obsolete") formats (edits: this worry has not been easy to type out)
CheeseFromLidl 11mo ago
As an aside - isn’t it remarkable that we’ve introduced uncertainty and doubt into the knowledge processing layer? We have decentralised networks that run on Bayesian symbols for server-client models, CPUs that crunch Markov chains and now AI that hallucinates. On Deterministic Turing Machines.
more_corn 11mo ago
Me too. Theres no tool more trusted for accessible numerical precision than Excel. Lets sell all that goodwill for a shiny new magic bean.
not_a_bot_4sho 11mo ago
I get it but I'm also tired of this meme
wodenokoto 11mo ago
Isn't it already there? https://support.microsoft.com/en-gb/office/get-started-with-copilot-in-excel-d7110502-0334-4b4f-a175-a73abdfc118a https://support.microsoft.com/en-gb/office/get-started-with-...
snowwrestler 11mo ago
Some companies may have only one Brenda, but what really makes accounting work is multiple Brendas. In the case of my employer, we have a pretty small finance dept but we pay a big firm to come in once a year to check the books. “Brenda as a service.” (BaaS) Brendas actually aren’t totally perfect so the tools between Brendas need to deterministically store and show their work so they can reliably check each other. If the tool itself has a Brenda baked into it—even if it’s a very good Brenda simulation!—it seems like a company could run the risk of losing that deterministic basis for double-checking. And therefore lose track of their accounting reality. Some people will for sure over-trust the AI in the spreadsheet and make some dumb mistakes. Let’s all remember though that dumb mistakes in business are not illegal, and the whole point of a private market is to enable “natural consequences” for businesses that make them. Some people will need to touch the hot stove of AI to understand how it could hurt them. I’m not sure there is any way to stop that, or even if we should.
tempodox 11mo ago
Deterministic formula evaluation is so boring. You can tell beforehand that the same input will produce the same result every time. With Copilot’s probabilistic formula evaluation you can never tell what the result will be, which makes things much more interesting and entertaining! Who wouldn’t want to have Copilot in Excel?
sakesun 11mo ago
Somebody will soon name their AI agent Brenda.
Fairburn 11mo ago
Dont use Excel? Unless you cant remove it or disable it assume this is the new normal. Act accordingly.
musicandpiss 11mo ago
Yes.-)
andreww_young 11mo ago
Being a copilot for Excel is an extremely challenging task. I've read many papers on this subject and feel that current technology is still far from achieving 90% or higher accuracy. If you frequently use ChatGPT to analyze Excel, I believe you also often encounter situations where it fails or makes mistakes, and this feature of ChatGPT (Code Interpreter) has been out for over two years now. Of course, ChatGPT is quite naive when handling Excel; it doesn't even bother to analyze how many tables are actually in a sheet, always foolishly treating each sheet as a single table.

494 points by isaacfrond 1y ago

At some point, a publicly-listed company will go bankrupt due to some catastrophic AI-induced fuck-up. This is a massive reputational risk for AI platforms, because ego-defensive behaviour guarantees that the people involved will make as much noise as they can about how it's all the AI's fault.

ramon156 1y ago
Do you really want these kind of companies to succeed? Let them burn tbh
- cjs_ac 1y ago
  I don't find comments along the lines of 'those people over there are bad' to be interesting, especially when I agree with them. My comment is about why it'll go wrong for them.
- mcphage 1y ago
  Make sure you’re not part of the kindling, then.
meibo 1y ago
That will never happen, AI cannot be allowed to fail, so we'll be paying for that AI bail-out.
gosub100 1y ago
I see the inverse of that happening: every critical decision will incorporate AI somehow. If the decision was good, the leadership takes credit. If something terrible happens, blame it on the AI. I think it's the part no one is saying out loud. That AI may not do a damn useful thing, but it can be a free insurance policy or surrogate to throw under the bus when SHTF.
- halfcat 1y ago
  This works at most one time. If you show up to every board meeting and blame AI, you’re going to get fired. This is true if you blame a bad vendor, or something you don’t even control like the weather. Your job is to deliver. If bad weather is the new norm, you better figure out how to build circus tents so you can do construction in the rain. If your AI call center is failing, you better hire 20 people to answer phones.

AmbroseBierce 1y ago

Brenda has been getting slower over the years -as we all have-, but soon the boss will learn that it was a small price to pay for knowing well how to keep such house of cards from collapsing.

Simulacra 1y ago
And then the boss will make the decision to outsource her job, to a company that promises the use of AI to make finance better, and faster, and while Brenda is in the unemployment line, someone else thousands of miles away is celebrating a new job
- gadflyinyoureye 1y ago
  We are setting AI deployed in the US, but actually Indians. They are not better, but they are cheaper. They are probably worse, but they are cheaper.

Traster 1y ago

I'm actually not that worried about this, because again I would classify this as a problem that already exists. There are already idiots in senior management who pass off bullshit and screw things up. There are natural mechanisms to cope with this, primarily in business reputation - if you're one of those idiots who does this people very quickly start just discounting what you're saying, they might not know how you're wrong, but they learn very quickly to discount what you're saying because they know you can't be trusted to self-check. I'm not saying that this can't happen and it's not bad. Take a look at nudge theory - the UK government created an entire department and spent enormous amounts of time and money on what they thought was a free lunch - that they could just "nudge" people into doing the things they wanted. So rather than actually solving difficult problems the uk government embarked on decades of pseudo-intellectual self agrandizement. The entire basis of that decades long debacle was based on bullshit data and fake studies. We didn't need AI to fuck it up, we managed it perfectly well by ourselves.

gmac 1y ago
Nudge theory isn't useless, it's just not anything like as powerful as money or regulation. It was taken up by the UK government at that time because the government was, unusually, a coalition of two quite different parties, and thus found it hard to agree to actually use the normal levers of power. This NY Times opinion piece by Loewenstein and Ubel makes some good arguments along these lines: https://web.archive.org/web/20250906130827/https://www.nytimes.com/2010/07/15/opinion/15loewenstein.html https://web.archive.org/web/20250906130827/https://www.nytim...

simonw 1y ago

This quote is pulled from a TikTok, I recommend watching the whole thing here: https://www.tiktok.com/@belligerentbarbies/video/7568380008633257271 https://www.tiktok.com/@belligerentbarbies/video/75683800086... (I pulled the quote by using yt-dlp to grab the MP4 and then running that through MacWhisper to generate a transcript.)

donatj 1y ago
It's a little over two paragraphs. Seems like it would have been simpler just to... type it out?
- adlpz 1y ago
  Where's the fun in that? :D
  - mavhc 1y ago
    We choose to automate these things, not because they are easy, but because they are an interesting problem to solve
- daliusd 1y ago
  Well if you do it once then yes, but if you automate this process it is different. E.g. I do this with YouTube videos, because watching 14 minutes video or reading 30 seconds summary is time saver. I still watch some videos fully, but many of them are not worth it. So in summary I think it was just part of automated process (maybe) or it will become one in the future.
- rererereferred 1y ago
  But then you would need a Brenda. Ai can write the automation script for you.
- simonw 1y ago
  Why spend two minutes typing (and realistically longer than that, if I want to capture the exact transcript I would need to keep hitting pause and play and correcting myself) when I can spend ten seconds pasting a URL into my terminal and then dragging and dropping the resulting file onto the MacWhisper window? I actually transcribed the whole TikTok which was about 50% longer than what I quoted, then edited it down to the best illustrative quote.

HeavyStorm 1y ago

Nay-sayers need to decide whether they fear AI because AI is dumb and will fuckup or because AI is smart and will take over.

tossandthrow 1y ago
Simon willson is definitely not a nay sayer.
9dev 1y ago
Both are valid concerns, no need to decide. Take the USA: They are currently lead by a patently dumb president who fucks up the global economy, and at the same time they are powerful enough to do so! For a more serious example, consider the Paperclip Problem[0] for a very smart system that destroys the world due to very dumb behaviour. [0]: https://cepr.org/voxeu/columns/ai-and-paperclip-problem https://cepr.org/voxeu/columns/ai-and-paperclip-problem
- RajT88 1y ago
  The paperclip problem is a bit hand-wavey about intelligence. It is taken as a given than unlimited intelligence would automatically win presumably because it could figure out how to do literally anything. But let's consider real life intelligence: - Our super geniuses do not take over the world. It is the generationally wealthy who do. - Super geniuses also have a tendency to be terribly neurotic, if not downright mentally ill. They can have trouble functioning in society. - There is no thought here about different kinds of intelligence and the roles they play. It is assumed there is only one kind, and AI will have it in the extreme.
  - 9dev 1y ago
    To be clear, I don't think the paperclip scenario is a realistic one. The point was that it's fairly easy to conceive an AI system that's simultaneously extremely savant and therefore dangerous in a single domain, yet entirely incapable of grasping the consequences or wider implications of its actions. None of us knows what an actual, artificial intelligence really looks like. I find it hard to draw conclusions from observing human super geniuses, when their minds may have next to nothing in common with the AI. Entirely different constraints might apply to them—or none at all. Having said all that, I'm pretty sceptical of an AI takeover doomsday scenario, especially if we're talking about LLMs. They may turn out to be good text generators, but not the road to AGI. But it's very hard to make accurate predictions in either direction.

glimshe 1y ago

This reminds me of a friend whose company ran a daily perl script that committed every financial transaction of the day to a database. Without the script, the company could literally make no money irrespectively of sales because this database was one piece in a complex system for payment processor interoperability. The script ran in a machine located at the corner of a cubicle and only one employee had the admin password. Nobody but a handful of people knew of the machine's existence, certainly not anyone in middle management and above. The script could only be updated by an admin. Copilot may be good, but sure as hell doesn't know that admin password.

danielbln 1y ago
If your mission critical process sits on some on-site box that no-one knows about, copilot being good or not is the least of your problems.
- maccard 1y ago
  Everywhere I’ve ever worked has had that mission critical box. At one of my jobs we had a server rack with UPS, etc, all the usual business. On the floor next to it was a dell desktop with a piece of paper on it that said “do not turn off”. It had our source control server in it, and the power button didn’t work. We did eventually move it to something more sensible but we had that for a long time
  - victorbjorklund 1y ago
    with only one person on earth being able to access it? so if that person is hit by a car everything goes down?
    - glimshe 1y ago
      Pretty much
    - maccard 11mo ago
      yeah. I mean, someone else would _eventually_ figure it out. There wasn't full disk encryption or anyhting on it, so if the guy got hit by a bus, and the machien turned off we probably would have just imaged the disk and got it running in a VM. But we didn't (and nobody was hit by a bus)

teekert 1y ago

Don't worry, in Teams it bothers me just one time a day, and with the click of a button it's gone... For another whole day.

mikert89 1y ago

10 billion dollars is probably going to be spent on automating excel, it’s going to happen

harryf 1y ago
There needs to a financial equivalent to the Mythical Man Month.
- graemep 1y ago
  There are plenty of things that play the role. The problem is that people ignore them.

eithed 1y ago

Let it all crash and burn

Dumblydorr 1y ago

Co-pilot and AI has been shoved at the Microsoft Stack in my org for months. Most of the features were disabled or hopelessly bad. It’s cheaper for Microsoft to push this junk and claim they’re doing something, it’s going to improve their stock far more than not doing it, even though it’s basically useless currently. Another issue is that my org disallows AI transcription bots. It’s a legit security risk if you have some random process recording confidential info because the person was too busy to attend the meeting and take notes themselves. Or possibly they just shirk off the meetings and have AI sit in.

aDyslecticCrow 1y ago
Transcription is arguably one of the must useful enterprise AI tools avaliable. But i sure as heck wouldn't trust the cloud with it.
- 2dvisio 1y ago
  Still find the Copilot transcripts orders of magnitude worse than something like Wispr Flow and they tend to allucinate constantly and do not adapt to a company's context (that Copilot has access too...). I am talking about acronyms of products / teams, names of people (even when they are in the call), etc.
  - srean 1y ago
    Can anyone familiar with the technical details shed light on why this is so. Is it because of a globally trained model (as opposed to trained[tweaked on] on context specific data) or because of using different classes of models.
    - aDyslecticCrow 1y ago
      Neither copilot nor flow can natively handle audio to my understanding, so there is already a transcription model converting it to text that then GPT tries to summarise. It could be they simply use a mediocre transcription model. Wispr is amazing but would hurt their pride to use a competitor. But i feel it's more likley the experience is; GPT didn't actually improve on the raw transcription, just made it worse. Especially as any miss-transcipted words may trip it up and make it misunderstand while making the summary. if i can choose between a potentially confused and misunderstood summary, and a badly spellchecked (flipped words) raw transcription, i would trust the latter.

Esophagus4 1y ago

Hmmm the Brendas I know look a little different. “There are two Brendas - their job is to make spreadsheets in the Finance department. Well, not quite - they add the months and categories to empty spreadsheets, then they ask the other departments to fill in their sales numbers every month so it can be presented to management. “The two Brendas don’t seem to talk, otherwise they would realize that they’re both asking everyone for the same information, twice. And they’re so focused on their little spreadsheet worlds that neither sees enough of the bigger picture to say, ‘Wait… couldn’t we just automate this so we don’t need to do this song and dance every month? Then we wouldn’t need two people in different parts of the company compiling the same data manually.’ “But that’s not what Brenda was hired for. She’s a spreadsheet person, not a process fixer. She just makes the spreadsheets.” We need fewer Brendas, and more people who can automate away the need for them.

SirFatty 1y ago
"We need fewer Brendas, and more people who can automate away the need for them." True... I have an on-staff data engineer for the purpose. But not all companies (especially in the SMB space) have that luxury.
7thaccount 1y ago
That's a pretty specific example when there are a lot of good "spreadsheet people" out there who do a lot more than spreadsheets (maybe they had to write SQL queries or scripts to get those numbers), but commonly need to simplify things down to a spreadsheet or power point for upper management. I'm not saying you should have multiple people doing redundant work, but this style isn't entirely dumb. What would this be replaced by? Some kind of large SAP like system that costs millions of dollars and requires a dozen IT staff to maintain?
- Esophagus4 1y ago
  Fair - I was creating a straw man mostly to make a point. The people I’m thinking aren’t running SQL queries or scripts, they’re merely collection points for data. So one good BI developer who knows Tableau and Salesforce and Excel and SQL can replace those pure collection points with a better process, but they can also generate insight into the data because they have some business understanding from being close to the teams, which is what my hypothetical Brenda can’t do. In my example, Brenda would be asking sales leaders to enter in their data instead of going into Salesforce herself because she doesn’t know that tool / side of the company well enough. I was making the point that, contrary to the article, the Brendas I know aren’t touched by the Excel angels, they’re just maintaining spreadsheets that we probably shouldn’t have anyway.

motoboi 1y ago

Excel is the “beast that drives the ENTIRE economy” and he’s worried about Brenda from the finance department losing her job because then her boss will get bad financial reports I suppose the person that wrote that have not ideia Excel is just an app builder where you embed data together with code. You know that we have excel because computers didn’t understand column names in databases and so data extraction needed to be made by humans. Humans then design those little apps in excel to massage the data. Well, now an agent can read the boss saying gimme the sales from last month and the agent don’t need excel for that, because it can query the database itself, massage the data itself using python and present the data itself with html or PNGs. So, we are in the process of automating Brenda AND excel away. Also, finance departments are a very small part of excel users. Just think everywhere were people need small programs, excel is there.

evolve2k 1y ago
You missed this bit “.. and then the AI is gonna fuck it up real bad and he won't be able to recognize it because he doesn't understand because AI hallucinates.”
- brazukadev 1y ago
  Brendas have fucked it up multiple times, by themselves or because their boss demanded
  - Forgeties79 1y ago
    The underlying assumption is that Brenda generally does her job pretty well. Human errors exist but usually peers/managers (or the person who did it) can identify and correct them reliably. If we have to compare LLM’s against people who are bad at their jobs in order to highlight their utility we’re going the wrong direction.
    - Telemakhos 1y ago
      There are a lot of underlying assumptions: Brenda, the woman, is accurate and trustworthy and has mastered an accurate and trustworthy technology; the upper manager, the male, will introduce error by not understanding that the technology he brings to bear on the situation is hallucinatory. The woman is lower in status and pay than the male. The woman is necessary to the functioning of "the economy" and "capitalism," while the man threatens those. There are a lot of unsubtle political undertones on TikTok.

fancyfredbot 1y ago

This is transparent nonsense. People are very very happy to introduce errors into excel spreadsheets without any help from AI. Financial statements are correct because of auditors who check the numbers. If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit).

tl 1y ago
> If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit). Volume matters. The single largest problem I run into: AI can generate slop faster than anyone can evaluate it.
- fancyfredbot 11mo ago
  If nobody can evaluate it then nobody will sign it off.
svnt 1y ago
It’s like calling out the county to inspect the home you built but when they arrive it’s a bouncy castle.

runako 1y ago

"the sweat from Brenda's brow is what allows us to do capitalism." The CEO has been itching to fire this person and nuke her department forever. She hasn't gotten the hint with the low pay or long hours, but now Copilot creates exactly the opening the CEO has been looking for.

quixoticaxolotl 1y ago

I find the contrast between two narratives around technology use so fascinating: 1. We advocate automation because people like Brenda are error-prone and machines are perfect. 2. We disavow AI because people like Brenda are perfect and the machine is error-prone. These aren't contradictions because we only advocate for automation in limited contexts: when the task is understandable, the execution is reliable, the process is observable, and the endeavour tedious. The complexity of the task isn't a factor - it's complex to generate correct machine code, but we trust compilers to do it all the time. In a nutshell, we seem to be fine with automation if we can have a mental model of what it does and how it does it in a way that saves humans effort. So, then - why don't people embrace AI with thinking mode as an acceptable form of automation? Can't the C-suite in this case follow its thought process and step in when it messes up? I think people still find AI repugnant in that case. There's still a sense of "I don't know why you did this and it scares me", despite the debuggability, and it comes from the autonomy without guardrails. People want to be able to stop bad things before they happen, but with AI you often only seem to do so after the fact. Narrow AI, AI with guardrails, AI with multiple safety redundancies - these don't elicit the same reaction. They seem to be valid, acceptable forms of automation. Perhaps that's what the ecosystem will eventually tend to, hopefully.

deleted 1y ago
[deleted]
Aeolun 1y ago
> We disavow AI because people like Brenda are perfect and the machine is error-prone. No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda.
- misnome 1y ago
  “Let’s deploy something as or more error prone as Brad at infinite scale across our organisation”
- candiddevmike 1y ago
  I don't understand why generative AI gets a pass at constantly being wrong, but an average worker would be fired if they performed the same way. If a manager needed to constantly correct you or double check your work, you'd be out. Why are we lowering the bar for generative AI?

Havoc 1y ago

That mirrors my experience as well. LLMs get instantly confused in real world scenarios in Excel and confidently hallucinate millions in errors If you look at the demos for these it’s always something that is clean and abundantly available in training data. Like an income statement. Or a textbook example DCF. Or my personal fav „here is some data show me insights“. Real world excel use looks nothing like that. I’m getting some utility out of them for some corporate tasks but zilch in excel space.

lxgr 1y ago
As somebody with non-existent experience with Excel, I could totally see myself getting a lot of value out of LLMs, if nothing else then simply for telling me what's possible, what functions and patterns exist at all etc.
- Havoc 11mo ago
  Yeah definitely has some value in that sense. That in itself isn't enough to make a dent in the work though. Think of it this way - an IDE can tell you what functions an object has or autocomplete something is useful to a beginner & learning. But that's not what puts food on the programmers table - writing code that solves real problems does. Same in excel business use cases - the numbers and formulas don't matter directly - their meaning in a business context does. And that connection can be very tenuous. With code the compiler is the ultimate arbiter - it has to make sense on that level. Excel files it's all freestyle - it could be anything from your grandmas shopping list to a model that runs half a bank.

diego_sandoval 1y ago

I'm more shocked that someone is using TikTok to speak things that actually make sense instead of mindless memes.

d--b 1y ago

It looks like the OP is thinking that AI causing errors in spreadsheets is going to make the whole economy collapse. When tools break, people stop using them before they sink the ship down. If AI is that terrible at spreadsheet, people will just revert to Brenda. And it's not like spreadsheets have no errors right now.

133361096 1y ago

320784788

jwsteigerwalt 1y ago

Many fears of “AI mucking it up” could be mitigated with an ability to connect a workbook to a git repository. Not for data, but for VBA, cell formulas, and cell metadata. When you can encapsulate the changes a contributor (in this case co-pilot) makes into a commit, you can more easily understand what changes it/they made.

smoothbran 11mo ago
Even if "teaching the rest of the workforce how to use Git" wasn't a massive obstacle, many (most?) corporate workflows involve editing files that weren't designed to be human-readable. There are a couple of approaches out there for comparing diffs to excel spreadsheets specifically, but it's not exactly pleasant.
- jwsteigerwalt 11mo ago
  Considering that Microsoft perfected change control in the legal industry (ms word’s track changes), something can be done to add a degree of change control to spreadsheets.

alienbaby 1y ago

Using ai does not absolve you from the responsibility of doing it correctly. If you use ai, then you better have the skills to have done the job yourself, and so have the ability to check the AI did things correctly. You can save time still, but perhaps not as much as you think, because you need to check the ai's work thoroughly.

b3lvedere 1y ago

"You know who's not hallucinating? Brenda" I don't know about that. There could be lots of interesting ways Brenda can (be convinced to) hallucinate.

giarc 1y ago
I agree - having watched many people use Excel over the years, I'd say people often overestimate their skills. I see three categories of Excel users. First there are the people that are intimidated by it and stay away from any task involving Excel. Second are the people that know a little bit (a few basic formulas) and overestimate their skills because they only compare themselves to the first group. And the third group are the actual power users but know to keep that quiet because otherwise they become the "excel person" and have to fix every sheet that has issues. I don't know if AI is going to make any of the above better or worse. I expect the only group to really use it will be that second group.
- b3lvedere 1y ago
  I have seen lots and lots of different uses for Excel in my line of work: - password database - script to automatically rename jpeg files - game - grocery lists - Book keeping (and try and not get caught for fraud several years, because the monthly spending limit is $5000 and $4999 a month is below that...) - embed/collect lots of Word documents - coloring book - Minecraft processes - Resume database - ID scans
  - recursive 11mo ago
    I've seen it used for laser testing. Not tracking test results. Testing lasers.

Zigurd 1y ago

It's verifier law. Coding agents are useful and good and real products because when they screw up, things stop working almost always before they can do damage. Coding agents are flawed in ways that existing tools are good at catching, never mind the more obvious build and runtime errors. Letting AI write your emails and create your P&L and cash flow projections doesn't have to run the gauntlet of tools that were created to stop flawed humans from creating bad code.

phyzome 1y ago
Nah, I've seen them screw in all sorts of ways that would fail in some conditions and not others. You're way too optimistic about this.
- Zigurd 1y ago
  Fair. I've been using the coding agent in Android Studio Canary to do exploratory code in Dart/Flutter and using ATProto. Low stakes, but higher productivity is a significant benefit. It's a daily surprise how brilliant it is it's some things and how abysmal at others.
insane_dreamer 11mo ago
Another problem is that the AI may try to fudge the numbers to mask its mistakes so they look like they all add up while the rot is hidden away. Just like it tries to manipulate the unit tests so they pass without fixing the _actual_ bug. I've seen it happen.

intended 1y ago

Everything is now about verification. AI may be able to spit out ann excel sheet or formula - But if it can’t be verified, so what ? And here’s my analogy to think about the debugging of an excel sheet - you can debug most corporate excel sheets with a calculator. But when AI is spitting out excel sheets - when the program is making smaller programs - what is the calculator in this analogy ? Are we going to be using excel sheets to debug the output of AI? I think this is the inherent limiter to the uptake of AI. There’s only so much intellectual / experiential / training depth present. And now we’re going to be training even fewer people. At the end of the day I /customers need something to work. But failing that - I will settle for someone to blame. Brenda handles a lot of blame. Is OpenAI going to step into that gap ?

byyoung3 1y ago

Brendas hallucinate all the time.

skeptrune 1y ago

Simon posting tiktok quotes on his blog was not on my 2025 bingo card.

simonw 1y ago
This isn't the first: https://simonwillison.net/2025/Aug/8/pearlmania500/ https://simonwillison.net/2025/Aug/8/pearlmania500/ and https://simonwillison.net/2024/Jul/29/dealing-with-your-ai-obsessed-co-worker-tiktok/ https://simonwillison.net/2024/Jul/29/dealing-with-your-ai-o... Also this fun diversion into Occlupanids: https://simonwillison.net/2024/Dec/8/holotypic-occlupanid-research-group/ https://simonwillison.net/2024/Dec/8/holotypic-occlupanid-re... A lot of people complain that the internet isn't as weird and funny as it used to be. The weird and funny stuff is all on TikTok!
- andybak 1y ago
  The mismatch between what people not on it think TikTok is like and what it's actually like (once you get the algo tuned to your taste) is pretty crazy. But then the "new user" experience is so horrific in terms of the tacky default content it serves you that I'm not surprised so many people don't get past it.
  - hunterpayne 11mo ago
    This is something I think all short form video platforms struggle with. I think I know why. Its because the difference in the UI. Basically, a user doesn't choose what to see next. On YouTube, you get dozens of next videos recommended on each view. On short form video, you get one. This causes clickbait to work a lot better for short form video. The problem with platforms like TikTok is more about UI than algorithm or the length of the videos.

thisisit 1y ago

Don't be like that. I work at a Fortune 500 and Brenda wants that co-pilot in Excel because it can help her achieve so much more. What is so much more you ask? Brenda and her C-Suits can not define it but they know for sure Copilot in excel will lead to enormous time saving.

sherinjosephroy 1y ago

[dead]

like_any_other 1y ago

Excel doesn't need AI to ruin your work: https://www.science.org/content/article/one-five-genetics-papers-contains-errors-thanks-microsoft-excel https://www.science.org/content/article/one-five-genetics-pa...

slipperybeluga 1y ago

[dead]

__mharrison__ 1y ago

Excel is the most popular programming environment in the universe. It has optimized the five minute out of the box experience so well that grade schoolers can use it. Other than that, it is pretty horrible for coding.

cssinate 11mo ago

I've partied with Brenda on the weekends, and let me tell you... SOMETIMES Brenda hallucinates. But never during work hours. The woman's a saint M-F.

danjl 11mo ago

Excel is programming. Spreadsheets have been full of bugs for decades. How is Brenda any different from a developer? Why are people scared when the LLM might affect their dollar calculations, and less bothered when it affects their product?

simonw 11mo ago
+100 this. Programmers who work in Excel (and never even dream of calling themselves programmers) are still programmers.
- bdangubic 11mo ago
  I know couple of them that get paid C-level money too :)

blitzar 11mo ago

another cheese that will affect the outcome of major tournaments, not a good look for microsoft its like the xlookup situation all over again, yet another move aimed at the casual audience, designed to bring in the party gamers and make the program an absolute mess competitively

NumberCruncher 11mo ago

Is this not the guy who is on the payroll of Anthropic? Not because he is wrong, but because there is so much marketing going on in this space nowadays.

simonw 11mo ago
Who, me? I'm still independent - I have a disclosures section on my blog here: https://simonwillison.net/about/#disclosures https://simonwillison.net/about/#disclosures Anthropic sometimes give me free credits (so I can try out preview features) and gave me a ticket to their conference a few months ago.

manuelz 11mo ago

I actually know Brenda.

newscracker 11mo ago

I feel what this article says based on some recent (non-catastrophic) experiences. I think I’m probably an above average user when it comes to Excel skills. I love spreadsheets. But I struggle with formulas like index, match, vlookup/xlookup and many others, and even more so when it requires nesting one within another and coming up with the underlying logic that leads to some complex nested formulas. Over the past couple of months, I’ve tried some smaller models on duck.ai and also ChatGPT directly to create some columns and formulas for a specific purpose. I found that ChatGPT is a lot better than the “mini” models on duck.ai. But in all these cases, though these platforms seemed more capable than me and could make attempts to explain their formulas, they were many a times creating junk and “looping” back with formulas that didn’t really work. I had to point out the result (blank or some #REF or other error) multiple times and they would acknowledge that there’s an issue and provide a working formula. That wouldn’t work either! I really love that these LLMs can sort of “understand” what I’m asking, break it down in English, and provide answers. But the end result has been an exercise in frustration and waste of time. Initially I really thought and believed that LLMs could make Excel more approachable and easier to use — like you tell it what you want and it’ll figure it out and give the magic incantations (formulas). Now I don’t think we’re anywhere close to that if ChatGPT (which I presume powers Copilot as well) struggles and hallucinates so much. I personally don’t have much hope with the (comparatively) smaller and older models.

deburo 11mo ago

Luckily this is a capitalist society and usually mistakes in the private market resolve themselves because losing money is not a winning strategy.

throwmeaway307 11mo ago

I'm worried Excel will go "enterprise only". and only LLM based interfaces will be enabled on the "office+windows" for consumers tier. e.g. MS Access is well on its way. as soon as x86 gets fully overtaken by ARM, and LLMs overtake "compilers" (also taken enterprise only).. then things like sqlite-browsers (FOSS "access") will be an arcane tool of binary incompatible ("obsolete") formats (edits: this worry has not been easy to type out)

CheeseFromLidl 11mo ago

As an aside - isn’t it remarkable that we’ve introduced uncertainty and doubt into the knowledge processing layer? We have decentralised networks that run on Bayesian symbols for server-client models, CPUs that crunch Markov chains and now AI that hallucinates. On Deterministic Turing Machines.

more_corn 11mo ago

Me too. Theres no tool more trusted for accessible numerical precision than Excel. Lets sell all that goodwill for a shiny new magic bean.

not_a_bot_4sho 11mo ago

I get it but I'm also tired of this meme

wodenokoto 11mo ago

Isn't it already there? https://support.microsoft.com/en-gb/office/get-started-with-copilot-in-excel-d7110502-0334-4b4f-a175-a73abdfc118a https://support.microsoft.com/en-gb/office/get-started-with-...

snowwrestler 11mo ago

Some companies may have only one Brenda, but what really makes accounting work is multiple Brendas. In the case of my employer, we have a pretty small finance dept but we pay a big firm to come in once a year to check the books. “Brenda as a service.” (BaaS) Brendas actually aren’t totally perfect so the tools between Brendas need to deterministically store and show their work so they can reliably check each other. If the tool itself has a Brenda baked into it—even if it’s a very good Brenda simulation!—it seems like a company could run the risk of losing that deterministic basis for double-checking. And therefore lose track of their accounting reality. Some people will for sure over-trust the AI in the spreadsheet and make some dumb mistakes. Let’s all remember though that dumb mistakes in business are not illegal, and the whole point of a private market is to enable “natural consequences” for businesses that make them. Some people will need to touch the hot stove of AI to understand how it could hurt them. I’m not sure there is any way to stop that, or even if we should.

tempodox 11mo ago

Deterministic formula evaluation is so boring. You can tell beforehand that the same input will produce the same result every time. With Copilot’s probabilistic formula evaluation you can never tell what the result will be, which makes things much more interesting and entertaining! Who wouldn’t want to have Copilot in Excel?

sakesun 11mo ago

Somebody will soon name their AI agent Brenda.

Fairburn 11mo ago

Dont use Excel? Unless you cant remove it or disable it assume this is the new normal. Act accordingly.

musicandpiss 11mo ago

Yes.-)

andreww_young 11mo ago

Being a copilot for Excel is an extremely challenging task. I've read many papers on this subject and feel that current technology is still far from achieving 90% or higher accuracy. If you frequently use ChatGPT to analyze Excel, I believe you also often encounter situations where it fails or makes mistakes, and this feature of ChatGPT (Code Interpreter) has been out for over two years now. Of course, ChatGPT is quite naive when handling Excel; it doesn't even bother to analyze how many tables are actually in a sheet, always foolishly treating each sheet as a single table.