16 ms·
I made an open source Windows app to rewind and search everything on screen
- djmips 2y agoThis will eventually be an AR app to do the same thing for you daily life.
- jstanley 2y agoWhoa, very cool project, I would be keen to try it out if it worked on Linux.
- haruharuha 2y agocheck out this similar project, it's made for Linux and has LLM feature: https://github.com/apirrone/Memento https://github.com/apirrone/Memento
- alchemist1e9 2y agoI’m surprised not more momentum on this. It would be neat to modify this to support local a LLM and maybe with txtai try to build semantic knowledge graphs on screenshots to auto detect “topics” of your desktop activity.
- INTPenis 2y agoI wonder if there is a sensible API where you can track everything without storing a video file. Something like seeing Window positions, classes, names, and maybe even GTK/QT cache?
- vitovito 2y agoYou can look into accessibility APIs, or see how Arbtt does its rule-based metadata tracking: https://arbtt.nomeata.de/doc/users_guide/effective-use.html#idp9 https://arbtt.nomeata.de/doc/users_guide/effective-use.html#...
- chabad360 2y agoSame, I've been looking for something like this ever since Rewind came out.
- petemir 2y agoSimilar (activity recording of each window), but not exactly the same (no screenshots): https://github.com/ActivityWatch/activitywatch https://github.com/ActivityWatch/activitywatch
- ksec 2y agoI wanted the same thing for Browser for a very long time. Since that is where 99% of all my information are consumed. But May be it does seems to fit in the OS in a broader prospective.
- aspenmayer 2y agoThat’s the feature I miss most about Google Desktop.
- asdefghyk 2y agoI use singlefile extension to save automatically a copy of every webpage I view. Also use BetterHistory extension on Chrome to record browsing history. Have been using both for a few years.
- alchemist1e9 2y agoI use singlefile also but not the auto-save. I’ve long thought that ideally an auto-archiving transparent proxy would be a more elegant approach.
- Cilvic 2y agoGreat, ever since rewind.ai came out I was longing for something similar (MacOS only for now)
- aspenmayer 2y agoCheck out rem which I mentioned in another comment: https://news.ycombinator.com/item?id=38787892 https://news.ycombinator.com/item?id=38787892
- wingerlang 2y agoOne alternative is ScreenMemory (https://screenmemory.app https://screenmemory.app) I made it, for full disclosure.
- aspenmayer 2y agoSimilar project for macOS which is also open source: https://rem.ing https://rem.ing https://github.com/jasonjmcghee/rem https://github.com/jasonjmcghee/rem Previously submitted on HN by the dev with some decent discussion 3 months ago here: https://news.ycombinator.com/item?id=38787892 https://news.ycombinator.com/item?id=38787892
- yogorenapan 2y agoIs there anyone here that has used it for an extended period of time? Would be interested to see if it’s actually helpful.
- aspenmayer 2y agoI used to use Google Desktop back when it existed for its viewed web page search feature mostly. It was pretty handy.
- xhevahir 2y agoGoogle Desktop was the first thing I thought of when I saw the link. One difference with Desktop though is that nowadays people are doing stuff on more than one device. Synching the data somehow or other would presumably involve the sort of cloud services that this developer is avoiding for privacy reasons.
- wingerlang 2y agoI can't speak for the mentioned app (rem), but I built my own app with a similar feature set called ScreenMemory (https://screenmemory.app https://screenmemory.app). Which I have obviously used for an extended period of time (coming up on 7 months I believe). My main and daily use case it to look back at the previous day - this is helpful for standups, retros, and so on. I skim through my days (sometimes weeks) to pick up on what I was working on - it's incredible how much "untracked" work is performed that you pick up on. Sometimes I forget who exactly I talked to about something, but just knowing the rough date I can usually find something to jog my memory. Obviously I am biased, but still!
- 2y ago
- poisonborz 2y agoCongrats on the great execution of this novel idea! Inspiring for everyone with a "why isn't there an app that does x" idea.
- nerdile 2y agoThe easiest way to find an app that does X, is to build a new one and then post it to HN and check the comments.
- novok 2y agorewind.ai is another example of this, but their recent pivot into cloud only storage for this and renaming to limitless.ai makes me glad that open source stuff like this popping up so you are not forced into a cloud storage situation. And I say this as a paying customer who will probably stop being one.
- jejeyyy77 2y agooh man, didnt know they completely pivoted. RIP
- dbish 2y agothey also seem to have shifted to hardware/"AI" wearables (lavalier mics?). hardware is not a promising shift imho
- fragmede 2y agothe rise of open source competitors makes it harder to compete as a software-only product
- dbish 2y agoI don’t think hardware is the right savior though, just moving faster with software, focusing on differentiation (ex: better understanding of the data for the task at hand rather then purely using OpenAI APIs). Hardware is the perfect way to kill your margins even with a successful product, look at the jawbone story
- maxloh 2y ago> Download ffmpeg (the download file name is: ffmpeg-master-latest-win64-gpl-shared.zip), extract all files in bin directory(excluding the bin directory itself) to C:\Windows\System32 (or other directories located in PATH) Copying ffmpeg to C:\Windows\System32 doesn't seem to be the correct way to install it.
- adzm 2y agoYeah just add the bin directory to the %path%. A lot of things have installed it to their own directory in appdata or program files.
- wongarsu 2y ago> A lot of things have installed it to their own directory in appdata or program files This is the windows way. In the early days (Windows 95/98) it was indeed common practice to put shared dlls and exes in C:\Windows or C:\Windows\System32, but without central oversight this was a nightmare: version conflicts were rampant, and uninstallers didn't know what could be safely removed. Everyone switched to installing everything into the program's install dir, and the world was much better off. Some poweruser tooling gets fancy by checking if ffmpeg is installed systemwide, and if not downloads it to the programs appdata folder. But that exposes you to versioning issues if that version is too old or too new
- nurple 2y agoThis is what's always confused me about the Unix FHS. A unified hierarchy makes (un)install complex and error-prone indeed. One of my favorite things about Nix is that it threw FHS away.
- deleted 2y ago[deleted]
- layer8 2y agoPer the GitHub readme, the video should be around 100-200 GB per year, not too bad.
- m3kw9 2y agoAvg how many hours a day? Does it delete scenes with no text like movies and games
- albert_e 2y agoI hope this becomes hardware feature soon on laptops -- where we dont need to sacrifice the main CPU/memory/storage to have this functionality.
- haruharuha 2y ago[dead]
- shiftpgdn 2y agoThis could easily be sold for $5/seat to giant corporations who want to stick spyware all over their employee workstations.
- cableshaft 2y agoIf I knew companies were recording my screen constantly I would just quit and save everyone the trouble of them getting on my case for not coding 100% of the time. I know my screen can be checked at my current client, but I also know the product owners don't really care as long as the work is getting done on time (or at least they haven't bugged me about it since I've starting working with them a year ago and have given very positive reviews about me to my company).
- cj 2y agoScreen recording probably won’t take off in this context, luckily. I think employers would be more interested in basic things like “did this person login their computer today, between what times were they active, how many hours in the day did they type at least 100 words?” … or things like “here’s a list of people on PTO today, which people not on this list haven’t typed on their keyboard today?” That sort of data would be easy to collect - keyloggers have been around forever, doesn’t trigger MacOS to show the “recording screen” notification, and the data is easier to aggregate and view across a large number of computers. I think the fact that keylogging didn’t explode as a method of tracking productivity during Covid WFH probably means we’re safe from screen recording for the foreseeable future.
- karencarits 2y agoFor Windows, there is also TimseSnapper [1], not open source, but the developer is sometimes here on HackerNews [1] https://timesnapper.com/ https://timesnapper.com/
- LeonB 2y agoCheers, yeh, I am here a bit.
- wartijn_ 2y agoThat’s fast :) Do you get notified when someone mentions your product, or are you _that_ active?
- LeonB 2y agoNeither, lol. Just happened across the comment within five minutes of it being posted. I think it was one of the first times I visited HN today.
- agons 2y ago> one of the first times I don't think you can say it's "neither"! :)
- pests 2y agoThe first first time, or the second first time, maybe the third?
- LeonB 2y agoProbably the first first time — but I don’t want to make a claim that’s untrue so I weakened the claim and broke the logic on the process, yeh.
- doodlebugging 2y agoThanks for TimeSnapper Pro. I started using it in 2010 I think after cross-testing the field of options available. I especially like that I was able to purchase a permanent license as opposed to being locked in to a SaaS model where my time data ended up in someone else's control. Everything right on my machine where if it is lost I get all the blame so I never have to stand up from my desk and shake my fist at the cloud. Excellent product for a consultant like myself to have on hand to document everything happening on their machine. The ability to change the screenshot interval and add notes along the timeline are real power tools for someone involved in multiple projects every day. You can secure your time record snapshots with a password. Playback at multiple speeds so you can jog your memory by browsing the timeline to the exact time when you flipped to something else. Classify by project, client, etc. Really a great bit of software. I once found one of my invoiced time periods being challenged during a regular audit of one of my client's operations. I frequently billed long hours since I worked long hours on 24-hour call (remote oil and gas data processing and support of international operations). The auditor called and requested anything I had that could support a particular time period where I billed "excessive" hours (>20 consecutive). I told them I would get them everything that I had and asked whether they were interested in any other dates or time periods or work for specific projects. They narrowed it to one single invoice entry so that made it simple for me. Thanks to TimeSnapper I was able to produce a GIF movie of the screenshots (I always used a 1 minute snap interval to allow fine-grained operations review) during the requested interval that documented everything happening on screen and I provided a PDF of the notes dumped from that interval documenting phone calls taken, etc. I also sent copies of my handwritten notes I always maintain as primary memory-jogging tool. In the return note to the auditor I asked if they were interested in any other dates or time periods and offered to send an archive on their request. The auditor called me up and we talked for a while and they thanked me for the detailed records and told me they wouldn't need anything else. It really helps that my wife is a CPA/Auditor/Fraud Examiner. Tools like TimeSnapper Pro helped me in my business to maintain excellent records about day-to-day operations that made it simple for me to invoice clients for all the time that I spent on their behalf. Thanks /u/LeonB
- hu3 2y agoIt would be cool to: 1) Make a Meta Smart Glasses take a photo every 2 seconds. 2) Send images to some server in the cloud. 3) Run OCR and object detection/labeling on these images. 4) Then present an app that allows searching (and chatting with) your past. I could then ask the LLM things like: 1) Where did I left my wallet? 2) Did I get my credit card back after paying the restaurant yesterday? (ADHD things, don't ask) 3) What was written in my daughter's new tshirt today? Bonus point if the app also records and transcribes audio so you could ask the LLM things like: 1) In the last meeting, what was the deadline that we settled on? 2) What was the phone number of that person I met in the park earlier today? 3) What was the name of the investor I met today? Bonus bonus point if it has access to your phone calls to so it can transcribe and index what others said. See Black Mirror episode "The Entire History of You": https://en.wikipedia.org/wiki/The_Entire_History_of_You https://en.wikipedia.org/wiki/The_Entire_History_of_You
- namanyayg 2y agoThat is exactly what the (highly criticised) Humane AI Pin does
- dbish 2y agoIt really doesn’t. It has no history or proactive understanding, it only runs a query when you ask it
- bee_rider 2y agoGenerally Black Mirror episodes are not intended to be aspirational.
- scotty79 2y agoI'm not abandoning my long standing aspiration just because Black Mirror told a scary story involving it in some manner.
- drdaeman 2y agoBlack Mirror is fiction, a lot of it works well thanks to our suspension of disbelief. Remove cloud from the equation, replacing it with a properly safeguarded fully owned hardware located on your own premises - basically make it privacy-respecting, minimizing abuse potential, do a proper audit, and it becomes a super desirable system for people with neurological differences (or just stressed folks - attention and memory suffer under the stress too), a glasses that enhance memory rather than vision. Sure, not without caveats and gotchas (don’t bring it to a poker night lol), but not that cyberpunky-bad either.
- compsciphd 2y agoWe built in almost 2 decades ago now (including the ability to scrub to a point in the past and resume execution from there) http://www.cs.columbia.edu/~orenl/papers/sosp07-dejaview.pdf http://www.cs.columbia.edu/~orenl/papers/sosp07-dejaview.pdf Abstract: As users interact with the world and their peers through their computers, it is becoming important to archive and later search the information that they have viewed. We present DejaView, a personal virtual computer recorder that provides a complete record of a desktop computing experience that a user can playback, browse, search, and revive seamlessly. DejaView records visual output, checkpoints corresponding application and file system state, and captures displayed text with contextual information to index the record. A user can then browse and search the record for any visual information that has been displayed on the desktop, and revive and interact with the desktop computing state corresponding to any point in the record. DejaView combines display, operating system, and file system virtualization to provide its functionality transparently without any modifications to applications, window systems, or operating system kernels. We have implemented DejaView and evaluated its performance on real-world desktop applications. Our results demonstrate that DejaView can provide continuous low-overhead recording without any user noticeable performance degradation, and allows browsing, search and playback of records fast enough for interactive use.
- hysan 2y agoIs the url correct? I get a file not found when I try to open it.
- wizzwizz4 2y agoHTTPS isn't a drop-in replacement for HTTP. Try visiting the HTTP site.
- water-data-dude 2y agoAh, gotta love link rot. This one works : https://www.cs.columbia.edu/~nieh/pubs/sosp2007_dejaview.pdf https://www.cs.columbia.edu/~nieh/pubs/sosp2007_dejaview.pdf This one’s broken: https://www.cs.columbia.edu/~orenl/papers/sosp07-dejaview.pdf https://www.cs.columbia.edu/~orenl/papers/sosp07-dejaview.pd... When I search for “dejaview” in the main index, I get the same broken link in the search results[0]. At first I thought they’d changed the URL structure (/pubs/ to /papers/), but if you visit [1] it works, but [2] doesn’t work. I guess “orenl” isn’t a member of the faculty anymore, so they tore down their page and removed all the associated resources. [0] https://www.cs.columbia.edu/g-search/?q=dejaview#gsc.tab=0&gsc.q=dejaview&gsc.page=1 https://www.cs.columbia.edu/g-search/?q=dejaview#gsc.tab=0&g... [1] https://www.cs.columbia.edu/~nieh/ https://www.cs.columbia.edu/~nieh/ [2] https://www.cs.columbia.edu/~orenl/ https://www.cs.columbia.edu/~orenl/
- bluelightning2k 2y agoAmazing that this was done by a non-professional dev. Huge cudos. (Tbh why not become a professional dev? You clearly have the skill and at least some interest.) There's a huge miss in this implementation though: 95% of the time what is on screen is web-browsing. So to go from nicely formatted markup with title tags -> video -> OCR is clearly missing am ore obvious path (either a proxy or browser extension).
- pmichaud 2y agoI'm not so sure. This generalizes, plus "nicely formatted markup" is a big assumption in the brave new world of modern web apps. I guess it's probably pretty reliable to look for visible text strings, but there's something cool about only caring about what is directly visible. I've often thought that screen readers might be better off now by attempting this visual approach. It used to not be possible, now it may be easier than relying on accessible markup.
- haruharuha 2y agoThank you! However, coding feels quite daunting to me, as my talents lean more towards design. The reason I embarked on this project is that I strongly desired to address a particular need, but had unable to find an appropriate alternative or willing programming friends to help. Due to my limited development capabilities, many aspects of this app don’t seem to be optimally or fundamentally implemented. Potentially, an open-source approach may inspire others with the same needs to collaborate or provide better solutions. And yes, most of my screen time is also web-browsing. Currently, I can use the web page's Windows title as a reference, which provides quite a bit of information. To enhance this feature, it might consider using like Chromedriver to gather more details, such as web links and page text, similar to what 'Rewind' offers.
- helpfulContrib 2y agoA long time ago, someone got "cryofreeze" working on Linux processes, where you could freeze a process to disk and unfreeze it later .. but this was purely experimental and doesn't seem to have persisted. Anyway, in my lab environment where I had this running .. over a decade ago now I guess .. I set up scripts to automatically cryofreeze the daemon processes I was writing, so I could indeed reset to the state in time, of interest during testing. This lab got re-comissioned for other things, and a needed kernel upgrade broke cryofreeze, and I never got back to it - but I have often wondered about it, as a temporal interface to some things would make for a whole new world of useful UI and other interation paradigms to explore. I remember there was something like this possible on WANG and Tandem systems back in the 80's, and it always has kind of piqued my interest as to why this isn't still considered a thing. Well, I guess syscall complexity has vastly changed since WANG and Tandem were around, lol. Anyway, I'd love to be able to do this on Linux again - I keep thinking to catch up with the state of the art of process freezing/unfreezing, so this has motivated me to do this - although I do confess that, as principally a Lua/C developer in my chosen market (embedded), I attain this re-playability in other ways (state sync'ing) which also allows me to move processes to other systems, relatively smoothly, also. Still, would love to have this for normal Linux processes, out of the box. Am I ignoring an obvious way to do this?
- theblazehen 2y agoHave you came across CRIU? https://criu.org/Main_Page https://criu.org/Main_Page
- helpfulContrib 2y ago[dead]
- modeless 2y agoHas anyone built something like this using accessibility APIs instead of (or in addition to) OCR? It seems like a waste to OCR everything when you could just get the text directly from the accessibility APIs. Also seems like potentially a good way to connect LLMs to UIs, and something like this would be the way to collect the training data.
- JoBrad 2y agoIt would also be great to have the foreground apps as metadata.
- ehsankia 2y agoAlso more general structured data. Like being able to search only the window titles, or text within a certain windows (Discord, Chrome).
- wingerlang 2y agoThis is what I added to my macOS app recently, foreground app metadata. It is displayed on the timeline if you look at the pictures on my website (screenmemory.app). For my use case it was night and day in UX.
- bitwize 2y agoDragon NaturallySpeaking supports voice commands like "click OK" and responds accordingly. Its solution to the problem of Microsoft Office doing its own custom widget rendering was to OCR the text on widgets and buttons to determine their labels. You need something like this far, far more often than you think you do. Developers will flummox you, they will NOT use the provided APIs.
- dbish 2y agoWe've done a bit of both for our screen seachable loom-like screen recorder, the problem is that the accessibility APIs differ greatly between Mac and Windows if you want to be OS agnostic and even on Windows all the apps tend to do things a little differently making it hard to say what did you actually "see", with some apps missing key data or implementing it incorrectly. OCR ends up being easier many times desptie thinking accessibility would be.
- ppqqrr 2y agoAnyone know if there’s an audio equivalent of this software?
- hu3 2y agoOn could plug https://github.com/openai/whisper https://github.com/openai/whisper to transcribe recorded audio.
- dbish 2y agoNot "always on" but if you enable a recording with us, it's all searchable and even attempts to write a doc of what you were talking about to scan or answer questions about later: https://augmend.com https://augmend.com. We do screen + audio, but audio alone does work.
- fastestjet 2y agohttps://flyonthewall-3499fa132151.herokuapp.com/ https://flyonthewall-3499fa132151.herokuapp.com/ I made one called flyonthewall, you can upload a m4a file and it gives back transcription from openAI whisper model. Basic setup (no accounts right now) that I use to transcribe plus summarize my audio notes from my dictaphone.
- oezi 2y agoI would be curious if anybody feels that such a tool wouldn't be too much a danger as spyware. In particular from a Chinese developer.
- rullelito 2y agoAny tool you download can be used as spyware.
- deleted 2y ago[deleted]
- ametrau 2y agoAny Chinese developer can call themselves John Smith. Even the spies.
- earleybird 2y agoI prefer to be also known as John Bigbooté.
- gnutrino 2y agoLooks very similar to https://apse.io/ https://apse.io/, which uses OCR to build a searchable index of everything you've seen on screen. I like the open source aspect of windrecorder.
- lazylion2 2y agoTIL you can screen record with ffmpeg
- cynicalsecurity 2y agoI can't think of any usage for that project other than spying for the employees. Just slightly modify the saving mechanism to send it to the corporate cloud and voila. AI profiling. Perfect for harassment at work and layoffs.
- lannisterstark 2y agoDespite your username, sometimes it's nice not to be cynical all the time. This is tremendously helpful for people like me who forget stuff often.
- Ylpertnodi 2y ago>sometimes it's nice not to be cynical all the time. I would propose you are both absolutely correct.
- anon115 2y agohow do i download it? i wanna try the msi not the command line
- msephton 2y agoI remember a few projects like this, the first I saw was called Savant Recall in 2014. But it failed to be selected for YC, so was set free as open-source. Napster co-founder Ritter picked it up, renamed it Atlas Recall (2016), gave it a new UI, secured $20M in funding. A year later it was suddenly shutdown. On LinkedIn says "acquired by Xinova". Another one I'd heard of was called Apse (2019).
- vitovito 2y agoSavant Recall demo video: https://www.youtube.com/watch?v=P_RtbBaWIG8 https://www.youtube.com/watch?v=P_RtbBaWIG8 Original Savant source? https://bitbucket.org/theluxury/savant/src/master/ https://bitbucket.org/theluxury/savant/src/master/ Videos of how the later Atlas Recall worked: https://www.youtube.com/@atlasinformatics3316/videos https://www.youtube.com/@atlasinformatics3316/videos
- msephton 2y agoInterestingly, Xinova itself was shut down in 2021. it seems it was some sort of patent hoarding company, though they called it a "network of inventors". https://www.geekwire.com/2021/invention-network-company-xinova-shut-5-years-spinning-intellectual-ventures/ https://www.geekwire.com/2021/invention-network-company-xino...
- lannisterstark 2y agoThere was this app, I don't remember the name, that was similar I think for MacOS. It would record the entire timeline, and you could just "scroll back in time - in real time btw-" to a specific date/time and see the open windows you had and what not. I distinctly remember seeing it somewhere (I think it was here), but I can't find what the app was called, despite all teh gooling.
- FL410 2y agoProbably rewind.ai
- hfe 2y ago> I made this app on Windows. The major inspiration from the early concept of Mac app Rewind and Black Mirror S1E3 "The Entire History of You"
- lannisterstark 2y agoThat'll show me to actually read the post than just gloss over it. Thanks lol.
- deleted 2y ago[deleted]
- nukeyoular 2y ago[dead]
- jackvalentine 2y agoI have had this thought for ages and I'm stoked to see someone having built it. It seems to create a huge risk for your data though. Any ideas about how you might go about securing against the application itself doing something untoward?
- akoboldfrying 2y agoIt runs locally, so you could just block outgoing network connections for the app.
- okokwhatever 2y agoNice idea, really, but it has the smell of a visual keylogger.
- maxglute 2y agoThis is great start. I've been wanting something like this for a while, 10-20gb per month feels like a lot though. Feel like archiving at quarter resolution and black and white and 32bit rate sound and eventually have AI upscale a good compromise.
- account42 2y ago> Oops! Something went wrong. > Please refresh and try again or.
- dmezzetti 2y agoHere is a similar open source project: https://github.com/Vincent-liuwingsang/unlost https://github.com/Vincent-liuwingsang/unlost
- unnouinceput 2y agoThe site is not usable on Firefox even after I temporarily allowed JS to run in my NoScript setup. Perhaps it wants me to disable uBlock Origin as well? Not gonna happen, so there is that. For me this is a no go area regardless of how cool the app is.
- tivert 2y ago> JavaScript must be enabled in order to use Notion. > Please enable JavaScript to continue. I run NoScript, and the way this block is constructed makes it impossible to actually enable JavaScript to run the site. Please don't redirect to a static page, because that page doesn't have any of the JavaScript present to enable on a case-by-case basis, the the redirect happens so fast I have no chance to enable JavaScript on the main page. Just enabling for notion.so itself is not sufficient to bypass the block.
- eddd-ddde 2y agoWow this sucks. They had a perfectly usable <noscript> and decided to use it in the worst way possible. Also I'm surprised html even allows for non-JS redirects.
- fnp84 2y agothis is ingenious, and most likely will open a thought process for the implementation of similar but extremely lightweight mechanisms in RATs and other malwares (like Rootkits)