7 ms·

Having wrangled many spreadsheets personally, and worked with CFOs who use them to run small-ish businesses, and all the way up to one of top 3 brokerage houses

Having wrangled many spreadsheets personally, and worked with CFOs who use them to run small-ish businesses, and all the way up to one of top 3 brokerage houses world-wide using them to model complex fixed income instruments... this is a disaster waiting to happen. Spreadsheet UI is already a nightmare. The formula editing and relationship visioning is not there at all. Mistakes are rampant in spreadsheets, even my own carefully curated ones. Claude is not going to improve this. It is going to make it far, far worse with subtle and not so subtle hallucinations happening left and right. The key is really this - all LLMs that I know of rely on entropy and randomness to emulate human creativity. This works pretty well for pretty pictures and creating fan fiction or emulating someone's voice. It is not a basis for getting correct spreadsheets that show what you want to show. I don't want my spreadsheet correctness to start from a random seed. I want it to spring from first principles.

noosphr 1y ago
My first job out of uni was building a spreadsheet infra as code version control system after a Windows update made an eight year old spreadsheet go haywire and lose $10m in a afternoon. Spreadsheets are already a disaster.
- daveguy 1y ago
  > Spreadsheets are already a disaster. Yeah, that's what OP said. Now add a bunch of random hallucinations hidden inside formulas inside cells. If they really have a good spreadsheet solution they've either fixed the spreadsheet UI issues or the LLM hallucination issues or both. My guess is neither.
- sally_glance 1y ago
  Compared to what? Granted, Excel incidents are probably underreported and might produce "silent" consequential losses. But compared to that, for enterprise or custom software in general we have pretty scary estimates of the damages. Like Y2K (between 300-600bn) and the UK Postal Office thing (~1bn).
  - array_key_first 1y ago
    Excel spreadsheets ARE custom software, with custom requirements, calculations, and algorithms. They're just not typically written by programmers, have no version control or rollback abilities, are not audited, are not debuggable, and are typically not run through QA or QC.
    - iambateman 1y ago
      If I could teach managers one lesson, it would be this one.
    - jackcviers3 1y ago
      I'll add to this - if you work on a software project to port an excel spreadsheet to real software that has all those properties, if the spreadsheet is sophisticated enough to warrant the process, the creators won't be able to remember enough details abut how they created it to tell you the requirements necessary to produce the software. You may do all the calculations right, and because they've always had a rounding error that they've worked around somewhere else, your software shows calculations that have driven business decisions for decades were always wrong, and the business will insist that the new software is wrong instead of owning some mistake. It's never pretty, and it always governs something extremely important.
      - calgoo 1y ago
        Now, if we could give that excel file to an llm and it creates a design document that explains everything is does, then that would be a great use of an LLM.
    - pjmlp 1y ago
      Thing is, they are also the common workaround solution for savy office workers that don't want to wait for the IT department if it exists, or some outsourced consultancy, to finally deliver something that only does half the job they need. So far no one has managed to deliver an alternative to spreedsheets that fix this issue, doesn't matter if we can do much better in Python, Java, C# whatever, if it is always over budget and only covers half of the work. I know, I have taken part in such project, and it run over budget because there was always that little workflow super easy to do in Excel and they would refuse to adopt the tool if it didn't cover that workflow as well.
- p4ul 1y ago
  It's interesting that you mention disaster; there is at least one annual conference dedicated to "spreadsheet risk management".[1] [1] https://eusprig.org/ https://eusprig.org/
- anitil 1y ago
  I know you probably can't share the details, but if you can I (and I'm sure all of us) would love to hear them
MattGaiser 1y ago
> Mistakes are rampant in spreadsheets To me, the case for LLMs is strongest not because LLMs are so unusually accurate and awesome, but because if human performance were put on trial in aggregate, it would be found wanting. Humans already do a mediocre job of spreadsheets, so I don't think it is a given that Claude will make more mistakes than humans do.
- lionkor 1y ago
  But isn't this only fine as long someone who knows what they are doing has oversight and can fix issues when they arise and Claude gets stuck? Once we all forget how to write SUM(A:A), will we just invent a new kind of spreadsheet once Claude gets stuck? Or in other words; what's the end game here? LLMs clearly cannot be left alone to do anything properly, so what's the end game of making people not learn anything anymore?
  - solumunus 1y ago
    Well the end game with AI is AGI of course. But realistically the best case scenario with LLM’s is having fewer people with the required knowledge, leveraging LLM’s to massively enhance productivity. We’re already there to some degree. It is hard to put a number on my productivity gain, but as a small business owner with a growing software company it’s clear to me already that I can reduce developer hiring going forward. When I read the skeptics I just have to conclude that they’re either poor at context building and/or work on messy, inconsistent and poorly documented projects. My sense is that many weaker developers who can’t learn these tools simply won’t compete in the new environment. Those who can build well designed and documented projects with deep context easy for LLM’s to digest will thrive. I assume all of this applies to spreadsheets.
    - dns_snek 1y ago
      Why isn't there a single study that would back up your observations? The only study with a representative experimental design that I know about is the METR study and it showed the opposite. Every study citing significant productivity improvements that I've seen is either: - relying on self-assessments from developers about how much time they think they saved, or - using useless metrics like lines of code produced or PRs opened, or - timing developers on toy programming assignments like implementing a basic HTTP server that aren't representative of the real world. Why is it that any time I ask people to provide examples of high quality software projects that were predominantly LLM-generated (with video evidence to document the process and allow us to judge the velocity), nobody ever answers the call? Would you like to change that? My sense is that weaker developers and especially weaker leaders are easily impressed and fascinated by substandard results :)
extr 1y ago
Is this just a feeling you have or is this downstream of actual use cases you've applied AI to observed and measured reliability on?
- lionkor 1y ago
  Not OP but using LLMs in any professional setting, like programming, editing or writing technical specifications, OP is correct. Without extensive promoting and injectimg my own knowledge and experience, LLMs generate absolute unusable garbage (on average). Anyone who disagrees very likely is not someone who would produce good quality work by themselves (on average). That's not a clever quip; that's a very sad reality. SO MANY people cannot be bothered to learn anything if they can help it.
  - extr 1y ago
    I would completely disagree. I use LLMs daily for coding. They are quite far from AGI and it does not appear they are replacing Senior or Staff Engineers any time soon. But they are incredible machines that are perfectly capable of performing some economically valuable tasks in a fraction of the time it would have taken a human. If you deny this your head is in the sand.
    - lionkor 1y ago
      Capable, yeah, but not reliable, that's my point. They can one shot fantastic code, or they can one shot the code I then have to review and pull my hair out over for a week, because it's such crap (and the person who pushed it is my boss, for example, so I can't just tell him to try again). That's not consistent.
      - wahnfrieden 1y ago
        You can ask your boss to submit PRs using Codex’s “try 5 variations of the same task and select the one you like most though
        1y ago
sothatsit 1y ago
I don't think tools like Claude are there yet, but I already trust GPT-5 Pro to be more diligent about catching bugs in software than me, even when I am trying to be very careful. I expect even just using these tools to help review existing Excel spreadsheets could lead to a significant boost in quality if software is any guide (and Excel spreadsheets seem even worse than software when it comes to errors). That said, Claude is still quite behind GPT-5 in its ability to review code, and so I'm not sure how much to expect from Sonnet 4.5 in this new domain. OpenAI could probably do better.
- deleted 1y ago
  [deleted]
- admdly 1y ago
  > That said, Claude is still quite behind GPT-5 in its ability to review code, and so I'm not sure how much to expect from Sonnet 4.5 in this new domain. OpenAI could probably do better. It’s always interesting to see others opinions as it’s still so variable and “vibe” based. Personally, for my use, the idea that any GPT-5 model is superior to Claude just doesn’t resonate - and I use both regularly for similar tasks.
  - sothatsit 1y ago
    I also find the subjective nature of these models interesting, but in this case the difference in my experiences between Sonnet 4.5 and GPT-5 Codex, and especially GPT-5 Pro, for code review is pretty stark. GPT-5 is consistently much better at hard logic problems, which code review often involves. I have had GPT-5 point out dozens of complex bugs to me. Often in these cases I will try to see if other models can spot the same problems, and Gemini has occasionally but the Claude models never have (using Opus 4, 4.1, and Sonnet 4.5). These are bugs like complex race conditions or deadlocks that involve complex interactions between different parts of the codebase. GPT-5 and Gemini can spot these types of bugs with a decent accuracy, while I’ve never had Claude point out a bug like this. If you haven’t tried it, I would try the codex /review feature and compare its results to asking Sonnet to do a review. For me, the difference is very clear for code review. For actual coding tasks, both models are much more varied, but for code review I’ve never had an instance where Claude pointed out a serious bug that GPT-5 missed. And I use these tools for code review all the time.
scoot 1y ago
Or you could, you know, read the article before commenting to see the limited scope of this integration? Anyway, Google has already integrated Gemini into Sheets, and recently added direct spreadsheet editing capability so your comment was disproven before you even wrote it
silenced_trope 1y ago
> The key is really this - all LLMs that I know of rely on entropy and randomness to emulate human creativity. This works pretty well for pretty pictures and creating fan fiction or emulating someone's voice. I think you need to turn down the temperature a little bit. This could be a beneficial change.
scosman 1y ago
> all LLMs that I know of rely on entropy and randomness to emulate human creativity Those are tuneable parameters. Turn down the temperature and top_p if you don't want the creativity. > Claude is not going to improve this. We can measure models vs humans and figure this out. To your own point, humans already make "rampant" mistakes. With models, we can scale inference time compute to catch and eliminate mistakes, for example: run 6x independent validators using different methodologies. One-shot financial models are a bad idea, but properly designed systems can probably match or beat humans pretty quickly.
- th0ma5 1y ago
  > Turn down the temperature and top_p if you don't want the creativity. This also reduces accuracy in real terms. The randomness is used to jump out of local minima.
  - scosman 1y ago
    That's at training time, not inference time. And temp/top_p aren't used to escape local minima, methods like SDG batch sampling, Adam, dropout, LR decay, and other techniques do that.
    - th0ma5 1y ago
      Ahh okay, so you really can't escape the indeterminacy?
      - scosman 1y ago
        You can zero out temperature and get determinism at inference time. Which is separate from training time where you need forms of randomness to learn. The point is for the quote "all LLMs that I know of rely on entropy and randomness to emulate human creativity" is a runtime parameter you can tweak down to zero, not a fundamental property of the technology.
        1y ago
sally_glance 1y ago
Having AI create the spreadsheet you want is totally possible, just like generating bash scripts works well. But to get good results, there needs to be some documentation describing all the hidden relationships and nasty workarounds first. Don't try to make LLMs generate results or numbers, that's bound to fail in any case. But they're okay to generate a starting point for automations (like Excel sheets with lots of formulas and macros), given they get access to the same context we have in our heads.
- bnug 1y ago
  I like this take. There seems to be an over-focus on 'one-shot' results, but I've found that even the free tools are a significant productivity booster when you focus on generating smaller pieces of code that you can verify. Maybe I'm behind the power curve since I'm not leveraging the full capability of the advanced LLM's, but if the argument is disaster is right around the corner due to potential hallucinations, I think we should consider that you still have to check your work for mission critical systems. That said, I don't really build mission critical systems - I just work in Aerospace Engineering and like building small time saving scripts / macros for other engineers to use. For this use, free LLMs even have been huge for me. Maybe I'm in a very small minority, but I do use Excel & Python nearly every day.
mountainriver 1y ago
You can do it cursor style
hoistbypetard 1y ago
IMO people tend to over-trust both AI and Excel. Maybe this will recalibrate that after it leads to a catastrophic business failure or two.
- phatfish 1y ago
  You would hope so. But how many companies have actually changed their IT policy of outsourcing everything to Tata Consultancy Services (or similar) where a sweaty office in Mumbai full of people who don't give a shit run critical infrastructure? Jaguar Landrover had production stopped for over a month I think, and 100+ million impact to their business (including a trail of smaller suppliers put near bankruptcy). I'd bet Tata are still there and embedded even further in 5 years. If AI provides some day-to-day running cost reduction that looks good on quarterly financial statements it will be fully embraced, despite the odd "act of god".
  - gpderetta 1y ago
    to be clear, tata owns JLR.
    - phatfish 1y ago
      Indeed, that slipped my mind. However the Marks and Spencer hack was also their fault. Just searching on it now it seems there is a ray of hope. Although i have a feeling the response won't be a well trained onshore/internal IT department. It will be another offshore outsourcing jaunt but with better compensation for incompetent staff on the outsourcers side. "Marks & Spencer Cuts Ties With Tata Consultancy Services Amid £300m Cyber Attack Fallout" (ibtimes.co.uk)
jbs789 1y ago
I tend to agree that dropping the tool as it is into untrained hands is going to be catastrophic. I’ve had similar professional experiences as you and have been experimenting with Claude Code. I’ve found I really need to know what I’m doing and the detail in order to make effective (safe) use out of it. And that’s been a learning curve. The one area I hope/think it’s closest to (given comments above) is potentially as a “checker” or validator. But even then I’d consider the extent to which it leaks data, steers me the wrong way, or misses something. The other case may be mocking up a simple financial model for a test / to bounce ideas around. But without very detailed manual review (as a mitigating check), I wouldn’t trust it. So yeah… that’s the experience of someone who maybe bridges these worlds somewhat… And I think many out there see the tough (detailed) road ahead, while these companies are racing to monetize.
stocksinsmocks 1y ago
My take is more optimistic. This could be an off ramp to stop putting critical business workflows in spreadsheets. If people start to learn that general purpose programming languages are actually easier than Excel (and with LLMs, there is no barrier), then maybe more robust workflows and automation will be the norm. I think the world would be a lot better off if excel weren’t in it. For example, I work at business with 50K+ employees where project management is done in a hellish spreadsheet literally one guy in Australia understands. Data entry errors can be anywhere and are incomprehensible. 3 or 4 versions are floating around to support old projects. A CRUD app with a web front end would solve it all. Yet it persists because Excel is erroneously seen as accessible whereas Rails, Django, or literally anything else is witchcraft.
- player1234 1y ago
  [flagged]
xbmcuser 1y ago
In my opinion the biggest use case for spread sheet with LLM is to ask them to build python scripts to do what ever manipulations you want to do with the data. Once people learn to do this workplace productivity would increase greatly I have been using LLM for years now to write python scripts that automate different repeatable tasks. Want a pdf of this data to be overlayed on this file create a python script with an LLM. Want the data exported out of this to be formated and tallied create a script for that.
- calgoo 1y ago
  Yesterday I had to pass a bunch of data to finance as the person that does so had left the company. But they wanted me to basically group by a few columns, so instead of spending an hour on this in excel, I created 3 rows of fake data, gave it to the llm, it created a Python script which I ran against the dataset. After manual verification of the results, it could be submitted to finance.
  - jb1991 1y ago
    Congrats? But you are not likely a typical user.
  - brabel 1y ago
    That’s exactly how it should be done if accuracy is important.
  - xbmcuser 1y ago
    Yeah I am not a programmer just more tech literate than most as I have always been fascinated by tech. I think people are missing the forest for the trees when it comes to LLMS. I have been using them to create simple bash, bat, python scripts. Which I would not have been able to put together before even with weeks of googling. I say that because I used to do that unsuccessfully but my success rate thorugh the roof with LLM's. Now I just ask an LLM to create the scripts and explain all the steps. If it is a complex script I would also ask it to add logging to the script so that I can feed the log back to the LLM and explain what is going wrong which allowed for a lot faster fixes. In the early days I and the LLM would be going around in circles till I hit the token limits. And to start from scratch again.
hansmayer 1y ago
Yeah, it's like that commercial for OpenAI (or was it Gemini?) where the guy says it lets the tool work on it's complex financial spreadsheets, goes for a walk with a dog, gets back and it is done with "like 98% accuracy". I cannot imagine what the 2% margin of error looks like for a company that moves around hundreds of billions of dollars...

noosphr 1y ago

My first job out of uni was building a spreadsheet infra as code version control system after a Windows update made an eight year old spreadsheet go haywire and lose $10m in a afternoon. Spreadsheets are already a disaster.

daveguy 1y ago
> Spreadsheets are already a disaster. Yeah, that's what OP said. Now add a bunch of random hallucinations hidden inside formulas inside cells. If they really have a good spreadsheet solution they've either fixed the spreadsheet UI issues or the LLM hallucination issues or both. My guess is neither.
sally_glance 1y ago
Compared to what? Granted, Excel incidents are probably underreported and might produce "silent" consequential losses. But compared to that, for enterprise or custom software in general we have pretty scary estimates of the damages. Like Y2K (between 300-600bn) and the UK Postal Office thing (~1bn).
- array_key_first 1y ago
  Excel spreadsheets ARE custom software, with custom requirements, calculations, and algorithms. They're just not typically written by programmers, have no version control or rollback abilities, are not audited, are not debuggable, and are typically not run through QA or QC.
  - iambateman 1y ago
    If I could teach managers one lesson, it would be this one.
  - jackcviers3 1y ago
    I'll add to this - if you work on a software project to port an excel spreadsheet to real software that has all those properties, if the spreadsheet is sophisticated enough to warrant the process, the creators won't be able to remember enough details abut how they created it to tell you the requirements necessary to produce the software. You may do all the calculations right, and because they've always had a rounding error that they've worked around somewhere else, your software shows calculations that have driven business decisions for decades were always wrong, and the business will insist that the new software is wrong instead of owning some mistake. It's never pretty, and it always governs something extremely important.
    - calgoo 1y ago
      Now, if we could give that excel file to an llm and it creates a design document that explains everything is does, then that would be a great use of an LLM.
  - pjmlp 1y ago
    Thing is, they are also the common workaround solution for savy office workers that don't want to wait for the IT department if it exists, or some outsourced consultancy, to finally deliver something that only does half the job they need. So far no one has managed to deliver an alternative to spreedsheets that fix this issue, doesn't matter if we can do much better in Python, Java, C# whatever, if it is always over budget and only covers half of the work. I know, I have taken part in such project, and it run over budget because there was always that little workflow super easy to do in Excel and they would refuse to adopt the tool if it didn't cover that workflow as well.
p4ul 1y ago
It's interesting that you mention disaster; there is at least one annual conference dedicated to "spreadsheet risk management".[1] [1] https://eusprig.org/ https://eusprig.org/
anitil 1y ago
I know you probably can't share the details, but if you can I (and I'm sure all of us) would love to hear them

MattGaiser 1y ago

> Mistakes are rampant in spreadsheets To me, the case for LLMs is strongest not because LLMs are so unusually accurate and awesome, but because if human performance were put on trial in aggregate, it would be found wanting. Humans already do a mediocre job of spreadsheets, so I don't think it is a given that Claude will make more mistakes than humans do.

lionkor 1y ago
But isn't this only fine as long someone who knows what they are doing has oversight and can fix issues when they arise and Claude gets stuck? Once we all forget how to write SUM(A:A), will we just invent a new kind of spreadsheet once Claude gets stuck? Or in other words; what's the end game here? LLMs clearly cannot be left alone to do anything properly, so what's the end game of making people not learn anything anymore?
- solumunus 1y ago
  Well the end game with AI is AGI of course. But realistically the best case scenario with LLM’s is having fewer people with the required knowledge, leveraging LLM’s to massively enhance productivity. We’re already there to some degree. It is hard to put a number on my productivity gain, but as a small business owner with a growing software company it’s clear to me already that I can reduce developer hiring going forward. When I read the skeptics I just have to conclude that they’re either poor at context building and/or work on messy, inconsistent and poorly documented projects. My sense is that many weaker developers who can’t learn these tools simply won’t compete in the new environment. Those who can build well designed and documented projects with deep context easy for LLM’s to digest will thrive. I assume all of this applies to spreadsheets.
  - dns_snek 1y ago
    Why isn't there a single study that would back up your observations? The only study with a representative experimental design that I know about is the METR study and it showed the opposite. Every study citing significant productivity improvements that I've seen is either: - relying on self-assessments from developers about how much time they think they saved, or - using useless metrics like lines of code produced or PRs opened, or - timing developers on toy programming assignments like implementing a basic HTTP server that aren't representative of the real world. Why is it that any time I ask people to provide examples of high quality software projects that were predominantly LLM-generated (with video evidence to document the process and allow us to judge the velocity), nobody ever answers the call? Would you like to change that? My sense is that weaker developers and especially weaker leaders are easily impressed and fascinated by substandard results :)

extr 1y ago

Is this just a feeling you have or is this downstream of actual use cases you've applied AI to observed and measured reliability on?

lionkor 1y ago
Not OP but using LLMs in any professional setting, like programming, editing or writing technical specifications, OP is correct. Without extensive promoting and injectimg my own knowledge and experience, LLMs generate absolute unusable garbage (on average). Anyone who disagrees very likely is not someone who would produce good quality work by themselves (on average). That's not a clever quip; that's a very sad reality. SO MANY people cannot be bothered to learn anything if they can help it.
- extr 1y ago
  I would completely disagree. I use LLMs daily for coding. They are quite far from AGI and it does not appear they are replacing Senior or Staff Engineers any time soon. But they are incredible machines that are perfectly capable of performing some economically valuable tasks in a fraction of the time it would have taken a human. If you deny this your head is in the sand.
  - lionkor 1y ago
    Capable, yeah, but not reliable, that's my point. They can one shot fantastic code, or they can one shot the code I then have to review and pull my hair out over for a week, because it's such crap (and the person who pushed it is my boss, for example, so I can't just tell him to try again). That's not consistent.
    - wahnfrieden 1y ago
      You can ask your boss to submit PRs using Codex’s “try 5 variations of the same task and select the one you like most though
      - 1y ago

sothatsit 1y ago

I don't think tools like Claude are there yet, but I already trust GPT-5 Pro to be more diligent about catching bugs in software than me, even when I am trying to be very careful. I expect even just using these tools to help review existing Excel spreadsheets could lead to a significant boost in quality if software is any guide (and Excel spreadsheets seem even worse than software when it comes to errors). That said, Claude is still quite behind GPT-5 in its ability to review code, and so I'm not sure how much to expect from Sonnet 4.5 in this new domain. OpenAI could probably do better.

deleted 1y ago
[deleted]
admdly 1y ago
> That said, Claude is still quite behind GPT-5 in its ability to review code, and so I'm not sure how much to expect from Sonnet 4.5 in this new domain. OpenAI could probably do better. It’s always interesting to see others opinions as it’s still so variable and “vibe” based. Personally, for my use, the idea that any GPT-5 model is superior to Claude just doesn’t resonate - and I use both regularly for similar tasks.
- sothatsit 1y ago
  I also find the subjective nature of these models interesting, but in this case the difference in my experiences between Sonnet 4.5 and GPT-5 Codex, and especially GPT-5 Pro, for code review is pretty stark. GPT-5 is consistently much better at hard logic problems, which code review often involves. I have had GPT-5 point out dozens of complex bugs to me. Often in these cases I will try to see if other models can spot the same problems, and Gemini has occasionally but the Claude models never have (using Opus 4, 4.1, and Sonnet 4.5). These are bugs like complex race conditions or deadlocks that involve complex interactions between different parts of the codebase. GPT-5 and Gemini can spot these types of bugs with a decent accuracy, while I’ve never had Claude point out a bug like this. If you haven’t tried it, I would try the codex /review feature and compare its results to asking Sonnet to do a review. For me, the difference is very clear for code review. For actual coding tasks, both models are much more varied, but for code review I’ve never had an instance where Claude pointed out a serious bug that GPT-5 missed. And I use these tools for code review all the time.

scoot 1y ago

Or you could, you know, read the article before commenting to see the limited scope of this integration? Anyway, Google has already integrated Gemini into Sheets, and recently added direct spreadsheet editing capability so your comment was disproven before you even wrote it

silenced_trope 1y ago

> The key is really this - all LLMs that I know of rely on entropy and randomness to emulate human creativity. This works pretty well for pretty pictures and creating fan fiction or emulating someone's voice. I think you need to turn down the temperature a little bit. This could be a beneficial change.

scosman 1y ago

> all LLMs that I know of rely on entropy and randomness to emulate human creativity Those are tuneable parameters. Turn down the temperature and top_p if you don't want the creativity. > Claude is not going to improve this. We can measure models vs humans and figure this out. To your own point, humans already make "rampant" mistakes. With models, we can scale inference time compute to catch and eliminate mistakes, for example: run 6x independent validators using different methodologies. One-shot financial models are a bad idea, but properly designed systems can probably match or beat humans pretty quickly.

th0ma5 1y ago
> Turn down the temperature and top_p if you don't want the creativity. This also reduces accuracy in real terms. The randomness is used to jump out of local minima.
- scosman 1y ago
  That's at training time, not inference time. And temp/top_p aren't used to escape local minima, methods like SDG batch sampling, Adam, dropout, LR decay, and other techniques do that.
  - th0ma5 1y ago
    Ahh okay, so you really can't escape the indeterminacy?
    - scosman 1y ago
      You can zero out temperature and get determinism at inference time. Which is separate from training time where you need forms of randomness to learn. The point is for the quote "all LLMs that I know of rely on entropy and randomness to emulate human creativity" is a runtime parameter you can tweak down to zero, not a fundamental property of the technology.
      - 1y ago

sally_glance 1y ago

Having AI create the spreadsheet you want is totally possible, just like generating bash scripts works well. But to get good results, there needs to be some documentation describing all the hidden relationships and nasty workarounds first. Don't try to make LLMs generate results or numbers, that's bound to fail in any case. But they're okay to generate a starting point for automations (like Excel sheets with lots of formulas and macros), given they get access to the same context we have in our heads.

bnug 1y ago
I like this take. There seems to be an over-focus on 'one-shot' results, but I've found that even the free tools are a significant productivity booster when you focus on generating smaller pieces of code that you can verify. Maybe I'm behind the power curve since I'm not leveraging the full capability of the advanced LLM's, but if the argument is disaster is right around the corner due to potential hallucinations, I think we should consider that you still have to check your work for mission critical systems. That said, I don't really build mission critical systems - I just work in Aerospace Engineering and like building small time saving scripts / macros for other engineers to use. For this use, free LLMs even have been huge for me. Maybe I'm in a very small minority, but I do use Excel & Python nearly every day.

mountainriver 1y ago

You can do it cursor style

hoistbypetard 1y ago

IMO people tend to over-trust both AI and Excel. Maybe this will recalibrate that after it leads to a catastrophic business failure or two.

phatfish 1y ago
You would hope so. But how many companies have actually changed their IT policy of outsourcing everything to Tata Consultancy Services (or similar) where a sweaty office in Mumbai full of people who don't give a shit run critical infrastructure? Jaguar Landrover had production stopped for over a month I think, and 100+ million impact to their business (including a trail of smaller suppliers put near bankruptcy). I'd bet Tata are still there and embedded even further in 5 years. If AI provides some day-to-day running cost reduction that looks good on quarterly financial statements it will be fully embraced, despite the odd "act of god".
- gpderetta 1y ago
  to be clear, tata owns JLR.
  - phatfish 1y ago
    Indeed, that slipped my mind. However the Marks and Spencer hack was also their fault. Just searching on it now it seems there is a ray of hope. Although i have a feeling the response won't be a well trained onshore/internal IT department. It will be another offshore outsourcing jaunt but with better compensation for incompetent staff on the outsourcers side. "Marks & Spencer Cuts Ties With Tata Consultancy Services Amid £300m Cyber Attack Fallout" (ibtimes.co.uk)

jbs789 1y ago

I tend to agree that dropping the tool as it is into untrained hands is going to be catastrophic. I’ve had similar professional experiences as you and have been experimenting with Claude Code. I’ve found I really need to know what I’m doing and the detail in order to make effective (safe) use out of it. And that’s been a learning curve. The one area I hope/think it’s closest to (given comments above) is potentially as a “checker” or validator. But even then I’d consider the extent to which it leaks data, steers me the wrong way, or misses something. The other case may be mocking up a simple financial model for a test / to bounce ideas around. But without very detailed manual review (as a mitigating check), I wouldn’t trust it. So yeah… that’s the experience of someone who maybe bridges these worlds somewhat… And I think many out there see the tough (detailed) road ahead, while these companies are racing to monetize.

stocksinsmocks 1y ago

My take is more optimistic. This could be an off ramp to stop putting critical business workflows in spreadsheets. If people start to learn that general purpose programming languages are actually easier than Excel (and with LLMs, there is no barrier), then maybe more robust workflows and automation will be the norm. I think the world would be a lot better off if excel weren’t in it. For example, I work at business with 50K+ employees where project management is done in a hellish spreadsheet literally one guy in Australia understands. Data entry errors can be anywhere and are incomprehensible. 3 or 4 versions are floating around to support old projects. A CRUD app with a web front end would solve it all. Yet it persists because Excel is erroneously seen as accessible whereas Rails, Django, or literally anything else is witchcraft.

player1234 1y ago
[flagged]

xbmcuser 1y ago

In my opinion the biggest use case for spread sheet with LLM is to ask them to build python scripts to do what ever manipulations you want to do with the data. Once people learn to do this workplace productivity would increase greatly I have been using LLM for years now to write python scripts that automate different repeatable tasks. Want a pdf of this data to be overlayed on this file create a python script with an LLM. Want the data exported out of this to be formated and tallied create a script for that.

calgoo 1y ago
Yesterday I had to pass a bunch of data to finance as the person that does so had left the company. But they wanted me to basically group by a few columns, so instead of spending an hour on this in excel, I created 3 rows of fake data, gave it to the llm, it created a Python script which I ran against the dataset. After manual verification of the results, it could be submitted to finance.
- jb1991 1y ago
  Congrats? But you are not likely a typical user.
- brabel 1y ago
  That’s exactly how it should be done if accuracy is important.
- xbmcuser 1y ago
  Yeah I am not a programmer just more tech literate than most as I have always been fascinated by tech. I think people are missing the forest for the trees when it comes to LLMS. I have been using them to create simple bash, bat, python scripts. Which I would not have been able to put together before even with weeks of googling. I say that because I used to do that unsuccessfully but my success rate thorugh the roof with LLM's. Now I just ask an LLM to create the scripts and explain all the steps. If it is a complex script I would also ask it to add logging to the script so that I can feed the log back to the LLM and explain what is going wrong which allowed for a lot faster fixes. In the early days I and the LLM would be going around in circles till I hit the token limits. And to start from scratch again.

hansmayer 1y ago

Yeah, it's like that commercial for OpenAI (or was it Gemini?) where the guy says it lets the tool work on it's complex financial spreadsheets, goes for a walk with a dog, gets back and it is done with "like 98% accuracy". I cannot imagine what the 2% margin of error looks like for a company that moves around hundreds of billions of dollars...