5 ms·

Show HN: Open-Source Windows AI assistant that uses Word, Excel through COM

75 points by edmgood 2y ago

This started off as a project to understand how to get LLMs to interface with more traditional desktop softwares. We were especially interested in tools related to schematic drafting and molecular simulation. Decided to explore COM automation for more traditional Windows softwares as a starting point! Been using it to help some friends automate simple excel workflows. Thought we'd share!

ExxKA 2y ago
This is wild! If you want, this could become a great contender to the RPA products out there. I know people who are trying to build AI for RPA, but this seems like a much more solid approach. Let me know if you are interested in turning this into a startup, happy to direct you to some relevant people.
- edmgood 2y ago
  Very cool! Actively exploring some ideas in the computer-use space, happy to connect! Email in my bio :)
p_ing 2y ago
FYI the Office product group considers COM deprecated. The "new" version of Outlook has no COM support, for example. While Word/Excel may be many years off, it's something to consider if you do move this forward. I would also suggest moving to a compiled binary at some point. Very few Windows users are going to want to figure out Python in any capacity.
- Jayakumark 2y ago
  Does Webdriver selenium work for new outlook ? any idea on how to automate , currently using win32com to read emails https://github.com/hornlaszlomark/python_outlook https://github.com/hornlaszlomark/python_outlook as graph api is disabled.
  - nfredericks 2y ago
    Check out DavMail if you want programmatic access to your mailboxes. https://davmail.sourceforge.net/ https://davmail.sourceforge.net/
- victor106 2y ago
  What is the alternative to com?
  - jeroenhd 2y ago
    I think a combination of Office Scripts with maybe Power Automate and perhaps some kind of browser extension? I don't think all of COM will be available in the web applications. They haven't bothered implementing COM in Outlook, so Excel and Word will probably end up becoming web applications too.
    - andylynch 2y ago
      I am really curious about the conversations that will be had if or when they seriously propose to any of their banking and finance customers that they Webify excel for windows. (Though, going on recollection from past Q&As the excel devs have done, a more fundamental issue is that few people working they fully understand how COM works, and they are getting fewer)
      - p_ing 2y ago
        The only issue Microsoft has to worry about is legacy COM add-ins for Office products. The 'modern' method of creating add-ins has been around for awhile and has the benefit of also working on macOS and the web, plus not having an add-in that can crash the particular Office product, doesn't require an install on Windows, more secure, etc. etc... There are benefits to Microsoft and to end users. Migrating complex COM add-ins to modern add-ins is still a big lift. The app itself (Outlook) dropping COM support is simply given it no longer needs COM support for add-ins, it can drop COM itself.
      - internet101010 2y ago
        ERP add-ins for Excel are the main reason why I still use Windows at the office. Side note: the webified version of Outlook's new roomfinder is awful; it's slow, doesn't work correctly, and doesn't respect dark mode (but this is a larger failure on the Windows side in general).
  - 2y ago
- bitwize 2y ago
  What? COM is what made Windows great. Without it, it'll be just another shitty OS with a shitty DE, and you may as well just use Linux by that point.
  - TeMPOraL 2y ago
    COM doesn't work on the Web. They prefer to burn away the power features (like S/MIME in Outlook!) and release a strictly inferior, slower product, for the sake of... ... honestly I don't know what. It makes zero sense - it's pissing off corporate users, who are already captive. I'm guessing hidden inflation strikes again - they can't afford maintaining two versions, so they prefer to go with the worse one.
    - p_ing 2y ago
      Three version on the desktop. Web, macOS, and Windows. They're all unifying to the same interface, macOS mostly being there already.
    - tiahura 2y ago
      for the sake of monthly subs
      - TeMPOraL 2y ago
        That's the thing though: full-featured desktop software is already good enough at keeping people to their monthly subs, especially in enterprise contexts where there's no option for pirating it. Dragging the desktop versions down to the lowest common denominator of the web does not help drive more subscriptions; it's a miracle of vendor lock-in that it doesn't cause mass cancellations of existing ones.
      - bitwize 2y ago
        Pretty much. Satya's Microsoft has pivoted to a "cloud and services company", meaning that they're in a shortage of fucks to give about desktop operating systems and applications.
  - 2y ago
- edmgood 2y ago
  Ah interesting to hear about COM - interested where you heard or if you have other approaches for computer use! Yea, we're considering a .exe that people could download
abrichr 2y ago
Thanks for sharing! There is an open issue in OpenAdapt to implement COM support: https://github.com/OpenAdaptAI/OpenAdapt/issues/873 https://github.com/OpenAdaptAI/OpenAdapt/issues/873 This could be a valuable reference.
com2kid 2y ago
This is the type of innovations that AI should be used for. The fact that this type of product wasn't the very first thing Microsoft tried to create is unfortunate. Microsoft is kinda-sorta pushing for machines that can do AI stuff locally with their copilot branded PCs, but I don't see any evidence that they are willing to go all in on AI powered locally controlled everything with no SaSS model attached.
RebeVC 2y ago
[dead]

75 points by edmgood 2y ago

ExxKA 2y ago

This is wild! If you want, this could become a great contender to the RPA products out there. I know people who are trying to build AI for RPA, but this seems like a much more solid approach. Let me know if you are interested in turning this into a startup, happy to direct you to some relevant people.

edmgood 2y ago
Very cool! Actively exploring some ideas in the computer-use space, happy to connect! Email in my bio :)

p_ing 2y ago

FYI the Office product group considers COM deprecated. The "new" version of Outlook has no COM support, for example. While Word/Excel may be many years off, it's something to consider if you do move this forward. I would also suggest moving to a compiled binary at some point. Very few Windows users are going to want to figure out Python in any capacity.

Jayakumark 2y ago
Does Webdriver selenium work for new outlook ? any idea on how to automate , currently using win32com to read emails https://github.com/hornlaszlomark/python_outlook https://github.com/hornlaszlomark/python_outlook as graph api is disabled.
- nfredericks 2y ago
  Check out DavMail if you want programmatic access to your mailboxes. https://davmail.sourceforge.net/ https://davmail.sourceforge.net/
victor106 2y ago
What is the alternative to com?
- jeroenhd 2y ago
  I think a combination of Office Scripts with maybe Power Automate and perhaps some kind of browser extension? I don't think all of COM will be available in the web applications. They haven't bothered implementing COM in Outlook, so Excel and Word will probably end up becoming web applications too.
  - andylynch 2y ago
    I am really curious about the conversations that will be had if or when they seriously propose to any of their banking and finance customers that they Webify excel for windows. (Though, going on recollection from past Q&As the excel devs have done, a more fundamental issue is that few people working they fully understand how COM works, and they are getting fewer)
    - p_ing 2y ago
      The only issue Microsoft has to worry about is legacy COM add-ins for Office products. The 'modern' method of creating add-ins has been around for awhile and has the benefit of also working on macOS and the web, plus not having an add-in that can crash the particular Office product, doesn't require an install on Windows, more secure, etc. etc... There are benefits to Microsoft and to end users. Migrating complex COM add-ins to modern add-ins is still a big lift. The app itself (Outlook) dropping COM support is simply given it no longer needs COM support for add-ins, it can drop COM itself.
    - internet101010 2y ago
      ERP add-ins for Excel are the main reason why I still use Windows at the office. Side note: the webified version of Outlook's new roomfinder is awful; it's slow, doesn't work correctly, and doesn't respect dark mode (but this is a larger failure on the Windows side in general).
- 2y ago
bitwize 2y ago
What? COM is what made Windows great. Without it, it'll be just another shitty OS with a shitty DE, and you may as well just use Linux by that point.
- TeMPOraL 2y ago
  COM doesn't work on the Web. They prefer to burn away the power features (like S/MIME in Outlook!) and release a strictly inferior, slower product, for the sake of... ... honestly I don't know what. It makes zero sense - it's pissing off corporate users, who are already captive. I'm guessing hidden inflation strikes again - they can't afford maintaining two versions, so they prefer to go with the worse one.
  - p_ing 2y ago
    Three version on the desktop. Web, macOS, and Windows. They're all unifying to the same interface, macOS mostly being there already.
  - tiahura 2y ago
    for the sake of monthly subs
    - TeMPOraL 2y ago
      That's the thing though: full-featured desktop software is already good enough at keeping people to their monthly subs, especially in enterprise contexts where there's no option for pirating it. Dragging the desktop versions down to the lowest common denominator of the web does not help drive more subscriptions; it's a miracle of vendor lock-in that it doesn't cause mass cancellations of existing ones.
    - bitwize 2y ago
      Pretty much. Satya's Microsoft has pivoted to a "cloud and services company", meaning that they're in a shortage of fucks to give about desktop operating systems and applications.
- 2y ago
edmgood 2y ago
Ah interesting to hear about COM - interested where you heard or if you have other approaches for computer use! Yea, we're considering a .exe that people could download

abrichr 2y ago

Thanks for sharing! There is an open issue in OpenAdapt to implement COM support: https://github.com/OpenAdaptAI/OpenAdapt/issues/873 https://github.com/OpenAdaptAI/OpenAdapt/issues/873 This could be a valuable reference.

com2kid 2y ago

This is the type of innovations that AI should be used for. The fact that this type of product wasn't the very first thing Microsoft tried to create is unfortunate. Microsoft is kinda-sorta pushing for machines that can do AI stuff locally with their copilot branded PCs, but I don't see any evidence that they are willing to go all in on AI powered locally controlled everything with no SaSS model attached.

RebeVC 2y ago

[dead]