4 ms·

Assistants are cool, but I want better tools. Give me a timeline with all my emails, chats, documents, and pictures shown together. Full text search on my entir

Assistants are cool, but I want better tools. Give me a timeline with all my emails, chats, documents, and pictures shown together. Full text search on my entire browsing history. Exporting my Booking.com search results as a spreadsheet. Controlling my phone from my desktop, and vice versa. As a bonus, those capabilities don't require uploading your data to the cloud or risking LLM hallucinations.

vineyardmike 3y ago
> As a bonus, those capabilities don't require uploading your data to the cloud or risking LLM hallucinations. Ignoring that half those services are dependent on the cloud already, this is such a bold and hard problem. A solution would be magical but LLMs are our best chance. Without clearly defined protocols (eg SMTP, CalDav) and a ton of boring to write and use case specific glue code, you’re really dependent on LLMs to make those M*N connections from one service/format to another. An LLM can turn any current or future input format into any current or future output format. That’s really powerful and basically unsolvable at scale with the tools we have now.
- F7F7F7 3y ago
  And what’s to be said about every large orgs or serious companies TOS against screen scraping data?
  - vineyardmike 3y ago
    If they don't like screen scraping they probably also don't have an API and probably don't have 100 different ways to auto-export data.
oceanplexian 3y ago
> Exporting my Booking.com search results as a spreadsheet. Yeah except they actively don’t want you to have access to your data. They want you locked in to their shitty platform. And these companies will actually fight your ability to get data out, like Reddit or Twitter when they shut down their public APIs.
- davidd_ 3y ago
  They can want all they want, any human-readable information is now machine readable, just parse everything that's ever come across your screen.
  - bronco21016 3y ago
    In theory yes, has anyone built this tool yet? I’d love an LLM<->browser agent that can do something useful besides summarize Yelp reviews. I’m thinking something that, for example, accesses the company intranet and finds me information from my paystub without me doing anything.
niels_bom 3y ago
This so much. I want easily pluggable/connectable systems for everything. I want to see all my messages and all my contacts and all my stuff in interfaces I choose. The future is definitely not searching email, Whatsapp, Signal and Instagram to find the message my friend sent me the other day.

vineyardmike 3y ago

> As a bonus, those capabilities don't require uploading your data to the cloud or risking LLM hallucinations. Ignoring that half those services are dependent on the cloud already, this is such a bold and hard problem. A solution would be magical but LLMs are our best chance. Without clearly defined protocols (eg SMTP, CalDav) and a ton of boring to write and use case specific glue code, you’re really dependent on LLMs to make those M*N connections from one service/format to another. An LLM can turn any current or future input format into any current or future output format. That’s really powerful and basically unsolvable at scale with the tools we have now.

F7F7F7 3y ago
And what’s to be said about every large orgs or serious companies TOS against screen scraping data?
- vineyardmike 3y ago
  If they don't like screen scraping they probably also don't have an API and probably don't have 100 different ways to auto-export data.

oceanplexian 3y ago

> Exporting my Booking.com search results as a spreadsheet. Yeah except they actively don’t want you to have access to your data. They want you locked in to their shitty platform. And these companies will actually fight your ability to get data out, like Reddit or Twitter when they shut down their public APIs.

davidd_ 3y ago
They can want all they want, any human-readable information is now machine readable, just parse everything that's ever come across your screen.
- bronco21016 3y ago
  In theory yes, has anyone built this tool yet? I’d love an LLM<->browser agent that can do something useful besides summarize Yelp reviews. I’m thinking something that, for example, accesses the company intranet and finds me information from my paystub without me doing anything.

niels_bom 3y ago

This so much. I want easily pluggable/connectable systems for everything. I want to see all my messages and all my contacts and all my stuff in interfaces I choose. The future is definitely not searching email, Whatsapp, Signal and Instagram to find the message my friend sent me the other day.