5 ms·

I've never seen a text version that lets me select the text, that's bizarre

by yaomtc 3y ago

_rutinerad 3y ago
I’m assuming that he means that on (for example) Mac you can select text from any image and copy paste it. https://uk.pcmag.com/macos/138058/not-just-iphone-how-to-use-apples-live-text-feature-from-your-mac https://uk.pcmag.com/macos/138058/not-just-iphone-how-to-use...
- olliej 3y ago
  100% correct, I assumed windows also let people do that given text recognition is apparently trivial now (to the extent it's annoying - trying to drag images and get text selection instead is annoying :-/)
  - serf 3y ago
    Windows has gone sort of the opposite way, copy/paste is now often hindered if the engine recognizes the string to be sensitive or otherwise un-wise to copy to your clipboard. I've had a few instances on Windows 11 and surrounding software where ctrl-C as well as the context menu entry for 'Copy' were greyed out for this reason when skimming through logfiles, presumably because there was something about the line that triggered the MS "that's a password!" regex; stuuuupid stuff.
  - reddalo 3y ago
    I think Windows now does it as well, but of course (as all things Windows) it works only in very few apps (forget Win32 ones, for example).
olliej 3y ago
@_rutinerad got it - on macOS you can select text in any image, and I just assumed you could do that on windows as well (I figure in the context of linux it would be much more dependent on specific configuration so unilateral assumptions on behaviour would be questionable). It's honestly annoying as it frequently interferes with dragging images out of safari, except on those occasions when I do want the text when it's super useful. I think the iOS interface just tells you there's text in an image or photo and gives you the option to copy it rather than cursor based selection you get on Mac. [edit: from other comments it sounds like windows can do this but it's not always present, and not present in all circumstances, which makes me wonder how many cases in cocoa/uikit/swiftui it does not work]
- timschmidt 3y ago
  Am I the only one paranoid enough to think that this means Apple is now indexing even the text content of images stored on it's users computers?
  - olliej 3y ago
    That's literally a feature of the platform. If you open the photos app and type text in it will give you the photos containing that text. If your concern is "apple is harvesting my data" then no. All of apple's various analysis systems ("AI") are entirely local. This does mean you get a bunch of duplicated work as every device redoes the same analysis but on the other hand it saves you from "how do we defend against a compromised network".
    - timschmidt 3y ago
      > All of apple's various analysis systems ("AI") are entirely local. Even if that were true (I couldn't say, and I don't think anyone who doesn't have access to Apple source code and production systems could either) that wouldn't preclude Apple harvesting the results of said AI analysis. In fact, doing the analysis on users' devices would represent a shift of that processing from cloud to edge, representing a significant savings for Apple or anyone else in a similar position.
      - olliej 3y ago
        Apple's literal marketing message is that they do all the processing on device. It's not a shift for Apple, as apple has never done this analysis in the past, and only started doing the analysis once it could do it locally. The use cases we're talking about also don't work on a cloud based analysis, as you can't have text selection block on network uploads (generally slower than downloads), and it would require uploading every image you open to apple which would presumably be a lot of traffic, and an obvious privacy nightmare. It would also break for users who turn on the e2ee everything mode for iCloud.
        3y ago

I've never seen a text version that lets me select the text, that's bizarre

by yaomtc 3y ago

I've never seen a text version that lets me select the text, that's bizarre

I’m assuming that he means that on (for example) Mac you can select text from any image and copy paste it. https://uk.pcmag.com/macos/138058/not-just-iphone-how-to-use-apples-live-text-feature-from-your-mac https://uk.pcmag.com/macos/138058/not-just-iphone-how-to-use...

olliej 3y ago
100% correct, I assumed windows also let people do that given text recognition is apparently trivial now (to the extent it's annoying - trying to drag images and get text selection instead is annoying :-/)
- serf 3y ago
  Windows has gone sort of the opposite way, copy/paste is now often hindered if the engine recognizes the string to be sensitive or otherwise un-wise to copy to your clipboard. I've had a few instances on Windows 11 and surrounding software where ctrl-C as well as the context menu entry for 'Copy' were greyed out for this reason when skimming through logfiles, presumably because there was something about the line that triggered the MS "that's a password!" regex; stuuuupid stuff.
- reddalo 3y ago
  I think Windows now does it as well, but of course (as all things Windows) it works only in very few apps (forget Win32 ones, for example).

olliej 3y ago

@_rutinerad got it - on macOS you can select text in any image, and I just assumed you could do that on windows as well (I figure in the context of linux it would be much more dependent on specific configuration so unilateral assumptions on behaviour would be questionable). It's honestly annoying as it frequently interferes with dragging images out of safari, except on those occasions when I do want the text when it's super useful. I think the iOS interface just tells you there's text in an image or photo and gives you the option to copy it rather than cursor based selection you get on Mac. [edit: from other comments it sounds like windows can do this but it's not always present, and not present in all circumstances, which makes me wonder how many cases in cocoa/uikit/swiftui it does not work]

timschmidt 3y ago
Am I the only one paranoid enough to think that this means Apple is now indexing even the text content of images stored on it's users computers?
- olliej 3y ago
  That's literally a feature of the platform. If you open the photos app and type text in it will give you the photos containing that text. If your concern is "apple is harvesting my data" then no. All of apple's various analysis systems ("AI") are entirely local. This does mean you get a bunch of duplicated work as every device redoes the same analysis but on the other hand it saves you from "how do we defend against a compromised network".
  - timschmidt 3y ago
    > All of apple's various analysis systems ("AI") are entirely local. Even if that were true (I couldn't say, and I don't think anyone who doesn't have access to Apple source code and production systems could either) that wouldn't preclude Apple harvesting the results of said AI analysis. In fact, doing the analysis on users' devices would represent a shift of that processing from cloud to edge, representing a significant savings for Apple or anyone else in a similar position.
    - olliej 3y ago
      Apple's literal marketing message is that they do all the processing on device. It's not a shift for Apple, as apple has never done this analysis in the past, and only started doing the analysis once it could do it locally. The use cases we're talking about also don't work on a cloud based analysis, as you can't have text selection block on network uploads (generally slower than downloads), and it would require uploading every image you open to apple which would presumably be a lot of traffic, and an obvious privacy nightmare. It would also break for users who turn on the e2ee everything mode for iCloud.
      - 3y ago