5 ms·
I feel like the storage requirements are really going to be these issue for these apps/services that run on "take screenshots and OCR them" functionality with L
by crooked-v 3y ago
I feel like the storage requirements are really going to be these issue for these apps/services that run on "take screenshots and OCR them" functionality with LLMs. If you're using something like this a huge part of the value proposition is in the long term, but until something has a more efficient way to function, even a 1-year history is impractical for a lot of people.
For example, consider the classic situation of accidentally giving someone the same Christmas that you did a few years back. A sufficiently powerful personal LLM that 'remembers everything' could absolutely help with that (maybe even give you a nice table of the gifts you've purchased online, who they were for, and what categories of items would complement a previous gift), but only if it can practically store that memory for a multi-year time period.
- DariusKocar 3y agoIt's not that bad. With Perfect Memory AI I see ~9GB a month. That's 108 GB/year. HDD/SSDs are getting bigger than that every year. The storage also varies by what you do, your workflow and display resolution. Here's an article I wrote on my finding of storage requirements. https://www.perfectmemory.ai/support/storage-resources/storage-requirements https://www.perfectmemory.ai/support/storage-resources/stora... And if you want to use the data for LLM only, then you don't need to store the screenshots at all. Then it's ~ 15MB a month
- jascination 3y ago> That's 108 GB/year. HDD/SSDs are getting bigger than that every year. Cries in MacBook Pro
- lillecarl 3y agoPerfectMemory is only available on Windows at the moment.
- kristofferR 3y agohttps://Rewind.ai https://Rewind.ai is the macOS equivalent
- glenneroo 3y agoExcept that Rewind uses chatGPT whereas this runs entirely locally. I would like to note though that Anonymous Analytics are enabled as well as auto-updates, both of which I disabled for privacy reasons. Encryption is also disabled by default. I just blocked everything with my firewall for peace of mind :)
- technofiend 3y agoOutboard TB 3/4 storage only seems expensive until you price it against Apple's native storage. Is it slower? Of course! Is it fast enough? Probably.
- darreninthenet 3y agoI recently moved my macOS installation to an external Thunderbolt drive - it's faster than the internal SSD.
- technofiend 3y agoConsidering storage is a wasting asset and what Apple charges, this makes perfect sense to me.
- darreninthenet 3y agoThe funny thing is Apple even have a support article on how to do this (and actually say in it "may improve your performance") I literally followed it step by step and it was very easy and had no issues.
- ayewo 3y agoCan you share the Thunderbolt drive you got?
- darreninthenet 3y agohttps://glyphtech.com/products/atom-pro?variant=32121199919178 https://glyphtech.com/products/atom-pro?variant=321211999191... Shipped to the UK for me added a bit to the overall price with shipping and import duty but it was still better value for money and hugely reliable brand than anything I could have bought domestically.
- pauby 3y agoIt's Windows only so it won't run on your Mac anyway :-)
- pseudosavant 3y agoIs the 15mb basically embeddings from the video screenshots? What would it recall if there isn't the screenshots saved?
- rlt 3y agoI’m not sure if the above product does this, but you could use a multimodal model to extract descriptions of the screenshots and store those in a vector database with embeddings.
- dr_kiszonka 3y agoDoes storage use scale linearly with the number of connected monitors (assuming each monitor uses the same resolution)?
- DariusKocar 3y agoMost screenshots are of the application window in the foreground, so unless your application spans all monitors, there is no significant overhead with multiple monitors. DPI on the other hand has a significant impact. The text is finer, taking more pixels...
- xattt 3y agoThis reminds me of how Sherlock, Spotlight and its iterations came to be. It was very resource intensive to index everything and keep a live db, until it was not.
- rlt 3y agoI think ultimately you’d want it to summarize that down to something like: “Purchased socks from Amazon for $10 on 12/4/2024 at 5:04PM, shipped to Mom, 1600 Pennsylvania Av NW, Washington DC 20500, order number 1463355337 Probably stored in a vector DB for RAG.
- pennomi 3y agoMaybe. Until we find there’s a better way to encode the information and need the unfiltered, original context so it can be used with that new method.
- dav43 3y agoI set up two years ago a cron to screenshot every minute. Just did the second phase of using ocrmac (vision kit cli on GitHub) that extracts text and dumps it in a SQLite with FTS5. It’s simplistic but does the job for now. I looked at reducing storage requirements by using image magik to only store the difference between images - some 5 min sequence are essentially the same screen - but let that one go.
- sdenton4 3y ago/using image magik to only store the difference between images/ Well, that's basically how video codecs work... So might as well just find some codec params which work well with screen capture, and use an existing encoder.
- xhrpost 3y agoThanks for sharing. Curious, what main value adds have you gotten out of this data?
- dav43 3y agoI’m loose with my memory and I’d often recall reading or looking at something and could never find it in safari history etc. with info spread across WhatsApp emails files web history is helped nudge me in the right direction here and there. Saved me once when i made an online purchase, never got an email confirmation as well.
- jasonjayr 3y agoThis is where Microsoft (and Apple) has a leg up -- they can hook the UI at the draw level and parse the interface far more reliably + efficently than screenshot + OCR.
- deleted 3y ago[deleted]