9 ms·

SpreadsheetLLM: Encoding Spreadsheets for Large Language Models

190 points by goplayoutside 2y ago

goplayoutside 2y ago
https://venturebeat.com/ai/microsofts-new-ai-system-spreadsheetllm-unlocks-insights-from-spreadsheets-boosting-enterprise-productivity/ https://venturebeat.com/ai/microsofts-new-ai-system-spreadsh...
pavel_lishin 2y ago
Ah yes, Excel, the piece of software that already famously mangles data, is now going to be glued to software that also famously mangles data. Honestly, though, I kind of kid - I love spreadsheets, and if this actually works, it could be interesting. God help whoever needs to troubleshoot the hallucinated results - it's already hard enough to figure out what byzantine knotwork someone created using existing Excel functions, but now we'll have to also guess and second-guess layers of prompts that were used to either generate those same functions, or just generate output that got mulched through some AI black-box.
- Obscurity4340 2y ago
  Spreadsheets have such a way of making chaotic things more clear. I wonder if there's any work on spreadsheets as a multidimensional thinking tool
  - laborcontract 2y ago
    Indeed. I accidentally taught myself linear algebra through spending a ridiculous amount of time in excel. I only realized that after taking a linear algebra class and feeling helpless.. until I mentally remapped the concepts into excel space, after which it all became easy.
    - mettamage 2y ago
      Could you give an example?
      - laborcontract 2y ago
        Sure. So back in an old finance job I was given a whole bunch of portfolio modeling spreadsheets that was a huge mess of ad hoc column so which drove me nuts so everything started with me learning how to use arrays, which significantly reduced the complexity of basic data transformations. But then I wanted to analyze all our portfolio data over time so i had to figure out then how handle multi dimensionality in my spreadsheets. Then I figured out how to integrate and transform and reduce portfolio characteristics into sensible components for risk management and portfolio optimizations across different asset classes. I figured out how to do some absolutely ridiculous stuff in excel, it’s tough for me to think of tools that scratch the surface me if l that is nearly as good at helping working through
  - deleted 2y ago
    [deleted]
surfingdino 2y ago
I am calmly waiting for the SEC to rip them a new hole the size of Manhattan when hallucinated spreadsheets inevitably make their way into listed companies' reports.
- bongodongobob 2y ago
  Well you'll be waiting a very very long time.
- rsynnott 2y ago
  That would be on the companies (or their auditors), not Microsoft, in general. Clearly, no-one should ever _use_ this, should it ever make it out of research-land, but there's not that much obvious risk to _making_ it as long as they're honest about the risks.
  - SkyBelow 2y ago
    If it could spit out the analysis as spreadsheets that used standard formulas and only use the LLM to generate the formulas, it could be verified. Errors would slip through, but no worse than people applying the wrong formula based on a quick internet search that calculates a close but incorrect answer.
    - KoolKat23 2y ago
      Agreed, The workflow could also include a person checking it before submitting, just skimming through for errors.
  - RodgerTheGreat 2y ago
    There may not be much risk from a legal culpability perspective if they make the appropriate disclosures somewhere in the depths of a EULA, but even so it is a failure of professional ethics to build tools which are "dangerous at any speed" and inflict them upon the world.
    - surfingdino 2y ago
bsenftner 2y ago
I've found that all the top foundation models already understand spreadsheets very well, as well as all the functions, as well as all the common spreadsheet problems people run into using them. The Internet is chock full spreadsheet support forums and tutorials, and the foundation models have all been trained on this data. With not very much effort, one can explain to an LLM "here is a spreadsheet, formatted as..." which takes about 150 word tokens, and then not much more mental effort in your favorite language to translate an arbitrary spreadsheet into that format, and one gets a very capable LLM interface that can help explain complex arbitrary spreadsheets as well as generate them on request. I've got finance professionals and attorneys using a tool I wrote doing this to help them understand and debug complex spreadsheets given to them by peers and clients.
- ec109685 2y ago
  The issue was that before, large spreadsheets would overflow the context so this “compression” technique helps the LLM do more from the same data.
  - bsenftner 2y ago
    Which strikes me as an ingenious method of locking in their customers with a proprietary compressed format only their finetuned LLMs can parse.
fimdomeio 2y ago
Congratulations everyone, we can now automate the next global financial crisis.
- lainga 2y ago
  Now imagine an LLM trained on LLM web content. We call that an AIslop-squared.
victor9000 2y ago
Why on earth would you task a non-deterministic technology with data persistence?
- bubblyworld 2y ago
  The universe is fundamentally a non-deterministic technology, friend. We do what we can =D
ffhhj 2y ago
Waiting for the hallucinate formula: =HAL(9000)
chatmasta 2y ago
At Databricks summit there was a nice presentation [0] by the CEO of V7 labs who showed a demo of their LLM + Spreadsheet product. The kneejerk reaction of “ugh, LLM and spreadsheet?!” is understandable, but I encourage you to watch that demo. It makes clear some obvious potentials of LLMs in spreadsheets. They can basically be an advanced autofill. If you’ve used CoPilot in VSCode, you understand the satisfaction of feeling like an LLM is thinking one step ahead of you. This should be achievable in spreadsheets as well. [0] https://youtube.com/watch?v=0SVilfbn-HY&t=1251 https://youtube.com/watch?v=0SVilfbn-HY&t=1251 (queued to demo at 20:51)
- Kiro 2y ago
  Thank you. Tired of the usual jokers in threads like this. Right now the majority of comments are all sarcastic snark.
  - nforgerit 2y ago
    We jokers are tired as well
  - grape_surgeon 2y ago
    I'm new here; Hacker News is supposed to avoid the modern Reddit trap but feel it often falls into it. The topics are more relevant to me but the comments are often unbearably cynical and excessively dismissive
    - bubblyworld 2y ago
      Yeah, I feel like there's a real culture problem on HN right now, especially for topics that have received any degree of hype (AI and crypto, mainly). People can be excessively rude for no reason if you express an outsider view. Gets to the point where I can't trust anyone here to engage with me in good faith (the exceptions are a welcome blessing). I've come close to blocking the site on my network many times but it's an absolute goldmine of interesting info too... I'm not really sure if there's a solution, other than to practice emotionally disengaging from internet discussions.
christianqchung 2y ago
Goodness, I've been making a joke that AI companies are going to spend 500 billion dollars to make spreadsheet generators since 2023, and now it's becoming real. Gemini has a limited form of this too.
galaxyLogic 2y ago
How will it work? I open an Excel spreadsheet and also the AI Copilot. Then whenever I want to do something with Excel like "Show me which cells have formulas" CoPilot will interact with Excel and issue some command I cannot remember to do that for me? Menus are good but often hard to navigate and find. So the CoPilot can give me a whole new (prompt-based) user-interface to any MS-application? Is that how it works?
skywhopper 2y ago
Uhh, this is a paper about how to compress spreadsheet data to fit inside an LLM’s token limits, including such novel approaches as ignoring exact values of numbers, meaning of data types, and any context outside of a detected table of values. The paper doesn’t speak at all to actual uses of this approach, but that doesn’t stop the article writer from assuming this is probably a big step towards automated tools that analyze spreadsheet data for non-numerically inclined users. This is not that.
- dang 2y ago
  "The article writer" refers to https://news.ycombinator.com/item?id=40996429 https://news.ycombinator.com/item?id=40996429 - we merged that thread hither. supporting correct context for comments everywhere
awesomeMilou 2y ago
@ludicity 's head is going to explode.
cyanydeez 2y ago
DNA rEsearchers HAD TO STOP USING their preferrdd letter sequences because excel would autocorrect its typs. This.will not end. Well
ilaksh 2y ago
Is there a github?
- janpmz 2y ago
  It should become a standard for papers to publish their code. After all it's a publication.
IanCal 2y ago
I love the deep technical discussions on HN, and I'm disappointed to see anything AI related start to just resemble Reddit threads of people with knee jerk reactions to the title. This is interesting, it's about how you can represent spreadsheets to llms.
- nunodonato 2y ago
  Yes, for some reason we really have an established hate club around here. And the comments are usually the same thing everytime
mitjam 2y ago
Spreadsheets can fill the gap between ad-hoc prompting/prompt workbooks and custom software for special business tasks. By using a prompt function like LABS.GENERATIVEAI in Excel you can create solutions that combine calculations, data, and Generative AI. In my experience, transforming data to and from CSV works best for prompting in spreadsheets. Getting data to and from CSV format can be done with other spreadsheet functions. I've created a book and course (https://mitjamartini.com/resources/ai-engineering/ebooks/hands-on-chatgpt-in-excel/ https://mitjamartini.com/resources/ai-engineering/ebooks/han...) that teaches how to do this (both more beginner level). Just working through the examples or the examples provided by Anthropic for Claude for Sheets should be enough to get going.
blueyes 2y ago
The real trick would be for LLMs, which currently do math very poorly, to simply send "math to be done" into a spreadsheet, and retrieve the results... (If anyone is aware of an LLM that's great at math and physics, pls lmk!!)
nickpinkston 2y ago
I'm now so reliant on ChatGPT for gSheets, that I'd be almost unable to maintain my sheets' absurd formulas without it. It's also really accelerated my knowledge / skills of the specifics of the excel language. Having an LLM being able to directly read/write at the sheet level, instead of just generating formulas for one cell, would be amazing.
oslis 2y ago
[flagged]

190 points by goplayoutside 2y ago

https://venturebeat.com/ai/microsofts-new-ai-system-spreadsheetllm-unlocks-insights-from-spreadsheets-boosting-enterprise-productivity/ https://venturebeat.com/ai/microsofts-new-ai-system-spreadsh...

pavel_lishin 2y ago

Ah yes, Excel, the piece of software that already famously mangles data, is now going to be glued to software that also famously mangles data. Honestly, though, I kind of kid - I love spreadsheets, and if this actually works, it could be interesting. God help whoever needs to troubleshoot the hallucinated results - it's already hard enough to figure out what byzantine knotwork someone created using existing Excel functions, but now we'll have to also guess and second-guess layers of prompts that were used to either generate those same functions, or just generate output that got mulched through some AI black-box.

Obscurity4340 2y ago
Spreadsheets have such a way of making chaotic things more clear. I wonder if there's any work on spreadsheets as a multidimensional thinking tool
- laborcontract 2y ago
  Indeed. I accidentally taught myself linear algebra through spending a ridiculous amount of time in excel. I only realized that after taking a linear algebra class and feeling helpless.. until I mentally remapped the concepts into excel space, after which it all became easy.
  - mettamage 2y ago
    Could you give an example?
    - laborcontract 2y ago
      Sure. So back in an old finance job I was given a whole bunch of portfolio modeling spreadsheets that was a huge mess of ad hoc column so which drove me nuts so everything started with me learning how to use arrays, which significantly reduced the complexity of basic data transformations. But then I wanted to analyze all our portfolio data over time so i had to figure out then how handle multi dimensionality in my spreadsheets. Then I figured out how to integrate and transform and reduce portfolio characteristics into sensible components for risk management and portfolio optimizations across different asset classes. I figured out how to do some absolutely ridiculous stuff in excel, it’s tough for me to think of tools that scratch the surface me if l that is nearly as good at helping working through
- deleted 2y ago
  [deleted]

surfingdino 2y ago

I am calmly waiting for the SEC to rip them a new hole the size of Manhattan when hallucinated spreadsheets inevitably make their way into listed companies' reports.

bongodongobob 2y ago
Well you'll be waiting a very very long time.
rsynnott 2y ago
That would be on the companies (or their auditors), not Microsoft, in general. Clearly, no-one should ever _use_ this, should it ever make it out of research-land, but there's not that much obvious risk to _making_ it as long as they're honest about the risks.
- SkyBelow 2y ago
  If it could spit out the analysis as spreadsheets that used standard formulas and only use the LLM to generate the formulas, it could be verified. Errors would slip through, but no worse than people applying the wrong formula based on a quick internet search that calculates a close but incorrect answer.
  - KoolKat23 2y ago
    Agreed, The workflow could also include a person checking it before submitting, just skimming through for errors.
- RodgerTheGreat 2y ago
  There may not be much risk from a legal culpability perspective if they make the appropriate disclosures somewhere in the depths of a EULA, but even so it is a failure of professional ethics to build tools which are "dangerous at any speed" and inflict them upon the world.
  - surfingdino 2y ago

bsenftner 2y ago

I've found that all the top foundation models already understand spreadsheets very well, as well as all the functions, as well as all the common spreadsheet problems people run into using them. The Internet is chock full spreadsheet support forums and tutorials, and the foundation models have all been trained on this data. With not very much effort, one can explain to an LLM "here is a spreadsheet, formatted as..." which takes about 150 word tokens, and then not much more mental effort in your favorite language to translate an arbitrary spreadsheet into that format, and one gets a very capable LLM interface that can help explain complex arbitrary spreadsheets as well as generate them on request. I've got finance professionals and attorneys using a tool I wrote doing this to help them understand and debug complex spreadsheets given to them by peers and clients.

ec109685 2y ago
The issue was that before, large spreadsheets would overflow the context so this “compression” technique helps the LLM do more from the same data.
- bsenftner 2y ago
  Which strikes me as an ingenious method of locking in their customers with a proprietary compressed format only their finetuned LLMs can parse.

fimdomeio 2y ago

Congratulations everyone, we can now automate the next global financial crisis.

lainga 2y ago
Now imagine an LLM trained on LLM web content. We call that an AIslop-squared.

victor9000 2y ago

Why on earth would you task a non-deterministic technology with data persistence?

bubblyworld 2y ago
The universe is fundamentally a non-deterministic technology, friend. We do what we can =D

ffhhj 2y ago

Waiting for the hallucinate formula: =HAL(9000)

chatmasta 2y ago

At Databricks summit there was a nice presentation [0] by the CEO of V7 labs who showed a demo of their LLM + Spreadsheet product. The kneejerk reaction of “ugh, LLM and spreadsheet?!” is understandable, but I encourage you to watch that demo. It makes clear some obvious potentials of LLMs in spreadsheets. They can basically be an advanced autofill. If you’ve used CoPilot in VSCode, you understand the satisfaction of feeling like an LLM is thinking one step ahead of you. This should be achievable in spreadsheets as well. [0] https://youtube.com/watch?v=0SVilfbn-HY&t=1251 https://youtube.com/watch?v=0SVilfbn-HY&t=1251 (queued to demo at 20:51)

Kiro 2y ago
Thank you. Tired of the usual jokers in threads like this. Right now the majority of comments are all sarcastic snark.
- nforgerit 2y ago
  We jokers are tired as well
- grape_surgeon 2y ago
  I'm new here; Hacker News is supposed to avoid the modern Reddit trap but feel it often falls into it. The topics are more relevant to me but the comments are often unbearably cynical and excessively dismissive
  - bubblyworld 2y ago
    Yeah, I feel like there's a real culture problem on HN right now, especially for topics that have received any degree of hype (AI and crypto, mainly). People can be excessively rude for no reason if you express an outsider view. Gets to the point where I can't trust anyone here to engage with me in good faith (the exceptions are a welcome blessing). I've come close to blocking the site on my network many times but it's an absolute goldmine of interesting info too... I'm not really sure if there's a solution, other than to practice emotionally disengaging from internet discussions.

christianqchung 2y ago

Goodness, I've been making a joke that AI companies are going to spend 500 billion dollars to make spreadsheet generators since 2023, and now it's becoming real. Gemini has a limited form of this too.

galaxyLogic 2y ago

How will it work? I open an Excel spreadsheet and also the AI Copilot. Then whenever I want to do something with Excel like "Show me which cells have formulas" CoPilot will interact with Excel and issue some command I cannot remember to do that for me? Menus are good but often hard to navigate and find. So the CoPilot can give me a whole new (prompt-based) user-interface to any MS-application? Is that how it works?

skywhopper 2y ago

Uhh, this is a paper about how to compress spreadsheet data to fit inside an LLM’s token limits, including such novel approaches as ignoring exact values of numbers, meaning of data types, and any context outside of a detected table of values. The paper doesn’t speak at all to actual uses of this approach, but that doesn’t stop the article writer from assuming this is probably a big step towards automated tools that analyze spreadsheet data for non-numerically inclined users. This is not that.

dang 2y ago
"The article writer" refers to https://news.ycombinator.com/item?id=40996429 https://news.ycombinator.com/item?id=40996429 - we merged that thread hither. supporting correct context for comments everywhere

awesomeMilou 2y ago

@ludicity 's head is going to explode.

cyanydeez 2y ago

DNA rEsearchers HAD TO STOP USING their preferrdd letter sequences because excel would autocorrect its typs. This.will not end. Well

ilaksh 2y ago

Is there a github?

janpmz 2y ago
It should become a standard for papers to publish their code. After all it's a publication.

IanCal 2y ago

I love the deep technical discussions on HN, and I'm disappointed to see anything AI related start to just resemble Reddit threads of people with knee jerk reactions to the title. This is interesting, it's about how you can represent spreadsheets to llms.

nunodonato 2y ago
Yes, for some reason we really have an established hate club around here. And the comments are usually the same thing everytime

mitjam 2y ago

Spreadsheets can fill the gap between ad-hoc prompting/prompt workbooks and custom software for special business tasks. By using a prompt function like LABS.GENERATIVEAI in Excel you can create solutions that combine calculations, data, and Generative AI. In my experience, transforming data to and from CSV works best for prompting in spreadsheets. Getting data to and from CSV format can be done with other spreadsheet functions. I've created a book and course (https://mitjamartini.com/resources/ai-engineering/ebooks/hands-on-chatgpt-in-excel/ https://mitjamartini.com/resources/ai-engineering/ebooks/han...) that teaches how to do this (both more beginner level). Just working through the examples or the examples provided by Anthropic for Claude for Sheets should be enough to get going.

blueyes 2y ago

The real trick would be for LLMs, which currently do math very poorly, to simply send "math to be done" into a spreadsheet, and retrieve the results... (If anyone is aware of an LLM that's great at math and physics, pls lmk!!)

nickpinkston 2y ago

I'm now so reliant on ChatGPT for gSheets, that I'd be almost unable to maintain my sheets' absurd formulas without it. It's also really accelerated my knowledge / skills of the specifics of the excel language. Having an LLM being able to directly read/write at the sheet level, instead of just generating formulas for one cell, would be amazing.

oslis 2y ago

[flagged]