4 ms·
Well, I'm not going to share either solution as this is actually a pretty useful utility that I plan on releasing, but the short answer is: 1) don't use ScreenC
by LatencyKills 7mo ago
Well, I'm not going to share either solution as this is actually a pretty useful utility that I plan on releasing, but the short answer is: 1) don't use ScreenCaptureKit, and 2) take advantage of what CGWindowListCreateImage() offers through the content server. This is a simple IPC mechanism that does not trigger all the SKC limitations (i.e., no multi-space or multi-desktop support). In fact, when using SKC, the user cannot even minimize the "watched" window.
Claude realized those issues right from the start.
One of the trickiest parts is tracking the window content while the window is moving - the content server doesn't, natively, provide that information.
- stavros 7mo agoHuh, Claude one-shotted it out of a single message from me. Man, LLMs have gotten good.
- LatencyKills 7mo agoNo it didn't. Like I said... it may have gotten something that worked but there is no way Claude got it to work while supporting multi-spaces, multi-desktops, and using under 2% cpu utilization. My solution can display app window content even when those windows are minimized, which is not something the content server supports. My point was that Claude realized all the SKC problems and came up with a solution that 99% of macOS devs wouldn't even know existed.
- TeMPOraL 7mo ago> it may have gotten something that worked but there is no way Claude got it to work while supporting multi-spaces, multi-desktops, and using under 2% cpu utilization. Maybe, but that's the magic of LLMs - they can now one-shot or few-shot (N<10) you something good enough for a specific user. Like, not supporting multi-desktops is fine if one doesn't use them (and if that changes, few more prompts about this particular issue - now the user actually knows specifically what they need - should close the gap).
- jacquesm 7mo agoAnd now it does.
- LatencyKills 7mo agoSorry, "now it does", what?
- TeMPOraL 7mo agoThe things it didn't, which you then helpfully spelled out.
- LatencyKills 7mo agoDo you believe my brief overview of the problem will help Claude identify the specific undocumented functions required for my solution? Is that how you think data gets fed back into models during training?
- TeMPOraL 7mo agoYes. I don't think you appreciate just how much information your comments provide. You just told us (and Claude) what the interesting problems are, and confirmed both the existence of relevant undocumented functions, and that they are the right solution to those problems. What you didn't flag as interesting, and possible challenges you did not mention (such as these APIs being flaky, or restricted to Apple first-party use, or such) is even more telling. Most hard problems are hard because of huge uncertainty around what's possible and how to get there. It's true for LLMs as much as it is for humans (and for the same reasons). Here, you gave solid answers to both, all but spelling out the solution. ETA: > Is that how you think data gets fed back into models during training? No, one comment chain on a niche site is not enough. It is, however, how the data gets fed into prompt, whether by user or autonomously (e.g. RAG).
- jacquesm 7mo agoI wonder how many 'ideas guys' will now think that with LLMs they can keep their precious to themselves while at the same bragging about them in online fora. Before they needed those pesky programmers negotiating for a slice of the pie, but this time it will be different. Next up: copyright protection and/or patents on prompts. Mark my words.
- RileyJames 7mo agoIt one-shotted what exactly? Because LatencyKills is clearly describing a broader set of requirements related to their solution.