7 ms·

> Nature will not be publishing any content in which photography, videos or illustrations have been created wholly or partly using generative AI, at least for t

> Nature will not be publishing any content in which photography, videos or illustrations have been created wholly or partly using generative AI, at least for the foreseeable future. Allow me a bit of a rhetorical question, what are the chances they already publish photos taken on devices that apply by default some form of AI-based generative/corrective algorithms like the "AI detail enhancement engine" by Samsung (the one they use to enhance photos of the moon)?

asynchronous 3y ago
Probably close to 100% at this point.
LapsangGuzzler 3y ago
> devices that apply by default some form of AI-based generative/corrective algorithms like the "AI detail enhancement engine" Isn't this a contradiction, though? My understanding is that generative AI is created entirely from software, using a network of previously created images as input. A corrective filter modifies an image taken directly from a sensor instead. I personally don't mind aesthetic corrective modifications to photos. I was an astronomy observatory last night and learned that most of the magnificent images we've seen of distant nebulas and galaxies have post-production coloring applied, they mostly look black and white coming off the sensor. Does the coloring fundamentally change our understanding of what it is that we're looking at? I don't think so, and that's where I draw the line.
- Imnimo 3y ago
  >Artists, filmmakers, illustrators and photographers whom we commission and work with will be asked to confirm that none of the work they submit has been generated or augmented using generative AI "or augmented"
  - golemotron 3y ago
    There's no hard line in the technology. This means that a ban is pointless because the landscape is going to keep changing. It's interesting to compare this to other situations where, say, law tries to create lines that aren't really there and the incentive to ignore imaginary ones is greater than the incentive to keep them. This seems to be a very common phenomenon with technology.
    - pxc 3y ago
      In the case of Nature, it functions as a statement of values that scientists publishing with Nature will be happy to comply with to the best of their abilities. I doubt that the editors are under some illusion that the nominal ban will create a hard line in reality. I'd be surprised to learn that that is their idea of success with this measure.
- ghaff 3y ago
  The context matters. There are image manipulations I might do to a photo I'm going to hang on the wall that wouldn't be kosher if I were shooting an event for a newspaper especially with respect to removing objects from the photo.
- charcircuit 3y ago
  Some generative AI tools let you input a base image to work off of. You can definitely use generative AI for just sharpening in these tools.
- dragonwriter 3y ago
  > My understanding is that generative AI is created entirely from software, using a network of previously created images as input. A corrective filter modifies an image taken directly from a sensor instead. Your understanding is incorrect, generative AI can modify an image taken from any source, as well as creating from scratch.
ghaff 3y ago
I think you can argue that there is significant daylight between "created wholly or partly using generative AI" and the sort of ML-based noise reduction, sharpening, etc. that you see in products like Lightroom and Photoshop. Of course, the whole area will evolve and rules like these will have to evolve as well. News photography has dealt with this since pre-digital although different publications may have different standards.
- hgsgm 3y ago
  They'll have to use an AI to determine what manipulation counts as AI.
chasing 3y ago
“Generative AI.” I know there are kind of weird edge cases. “My iPhone made the sunset way redder than it was in real life.” But I think we all know what they’re talking about and I suspect if you’re in a position for it to really be a concern then you will communicate with Nature and sort out what their comfort zone is.
- etrautmann 3y ago
  This is fascinating and gets pretty interesting. As a computational neuroscience person, some of the more advanced neural signal processing algorithms use generate models internally to model recorded neural data. The result is likely a smoothed, simplified, and hopefully more interpretable view of neural data, but there's no guarantee that some portion of the resulting multidimensional signal isn't hallucinated. As a result, most findings should be validated by verifying that some property of interest is present in the high dimensional raw neural data, though that's only conceptually possible sometimes.
  - deleted 3y ago
    [deleted]
Kapura 3y ago
I am not in the game at all but as I understand it Samsung doesn't make high quality DSLRs of the type used by photographers. I reckon that photographers would be asked to disclose this sort of thing if submitting to Nature in future.
CharlesW 3y ago
> …AI-based generative/corrective algorithms… Every photo is touched by "corrective algorithms". Nature is talking about generative AI specifically, which means using an LLM to generate part or all of an image. This precludes using Midjourney, Photoshop's new "generative fill", etc. I assume that what Samsung's "Space Zoom" feature does — replacing elements with higher-quality stock photography — was already disallowed. If so, whether the elements were identified/replaced manually or automatically isn't really a concern from an editorial perspective.
- ghaff 3y ago
  Yes, they already had guidelines for photographs. [1] e.g. "Digital images submitted with a manuscript for review should be minimally processed. A certain degree of image processing is acceptable for publication (and for some experiments, fields and techniques is unavoidable), but the final image must correctly represent the original data and conform to community standards. Editors may use software to screen images for manipulation." [1] https://www.nature.com/nature-portfolio/editorial-policies/image-integrity https://www.nature.com/nature-portfolio/editorial-policies/i...
  - dclowd9901 3y ago
    Sounds like this would allow for processed astronomical photography too. Methinks this questions wasn’t in earnest.
    - ghaff 3y ago
      No reason to assume bad faith. But some folks get very literal. And if you have a legitimate question, that's one of the things editors are for.
- MiguelX413 3y ago
  How might one use a Large Language Model “to generate part or all of an image”?
  - 3y ago
analog31 3y ago
I'm peripherally involved in this scene. The answer is that the journals don't want processed images, but of course the scientist doesn't always know what kind of processing happened to the image en route to their display and file system. The idea is that an image supposedly constitutes "data" and as such, should represent raw data. Also, what constitutes "raw data" is itself a matter of debate. How raw is raw? Like any interesting pursuit, scientific publishing struggles to keep up with developments in technology.
- codetrotter 3y ago
  > How raw is raw? Certainly no jpeg image produced by any digital camera is really “raw” as it will already have been through a debayering filter https://en.wikipedia.org/wiki/Bayer_filter https://en.wikipedia.org/wiki/Bayer_filter And then on top of that is the JPEG compression artifacts. But I do wonder how many raw files also contain data that has been debayered already. I have not looked into that. I know that with third party firmware such as Magic Lantern it is possible to get the image data without debayering. https://magiclantern.fm/ https://magiclantern.fm/ Likewise I know that the Camera Module 3 for Raspberry Pi is possible to retrieve the image data from without debayering.
  - morphicpro 3y ago
    [flagged]
  - sudosysgen 3y ago
    The RAW data for the vast majority of MILCs and DSLRs is pre-debayering.
    - userbinator 3y ago
      Even Android smartphones will give pre-debayering raw data from the sensor if you use the appropriate camera app. (There are quite a few cheap ones where the OEM debayering filter is just horrible, and the sensor is actually capable of much better quality.)
  - 3y ago
morphicpro 3y ago
There is also a big distinction between fully disclosing that an image has AI vs the non post production edits. So why not just ask people to just be honest and disclose about the means of creating the content and disclose the images as being "AI" created vs telling people they should to be purists in their craft. Is the claim that the art made by AI itself is harmful or that the act of making it causes harm. Where is the real harm in this given people are properly informed as the the type of content they are looking at? The reasons why is because this has nothing to do with art and expression and everything to do with control under the guise of fear.
- deleted 3y ago
  [deleted]
belter 3y ago
It was not a problem when presenting the EHT results and subsequent "images" - https://www.space.com/first-ever-black-hole-image-ai-makeover https://www.space.com/first-ever-black-hole-image-ai-makeove...
- criddell 3y ago
  Were those produced by a generative AI?
  - belter 3y ago
    No, but under the same algorithmic principles. How do you think that yellow color come about?
0xBABAD00C 3y ago
> what are the chances they already publish photos taken on devices that apply by default some form of AI-based generative/corrective algorithms 100%?
m3kw9 3y ago
For them purposely asking stable diffusion(not ok) for an image vs iPhone image processing(ok) would be your baseline for distinguishing. Picking at small details seem like a nice way to waste time, you just got to keep it simple
ChatGTP 3y ago
On the other hand, I really hate what these algorithms do to my photos, even my new iPhone which is considered good tech. So I get it.
renonce 3y ago
The part of the article discussing problems with generative AI seems to lay emphasis on "attribution". The problem with generative AI is not that they want raw and unedited images, but that they are unable to properly attribute generated images to their sources. That's what makes the distinction between a generative AI and regular image editing tools like cropping, scaling, filtering, etc. So something like "AI detail enhancement engine" should not be problematic under such a definition as the tool can simply be attributed to Samsung.

asynchronous 3y ago

Probably close to 100% at this point.

LapsangGuzzler 3y ago

> devices that apply by default some form of AI-based generative/corrective algorithms like the "AI detail enhancement engine" Isn't this a contradiction, though? My understanding is that generative AI is created entirely from software, using a network of previously created images as input. A corrective filter modifies an image taken directly from a sensor instead. I personally don't mind aesthetic corrective modifications to photos. I was an astronomy observatory last night and learned that most of the magnificent images we've seen of distant nebulas and galaxies have post-production coloring applied, they mostly look black and white coming off the sensor. Does the coloring fundamentally change our understanding of what it is that we're looking at? I don't think so, and that's where I draw the line.

Imnimo 3y ago
>Artists, filmmakers, illustrators and photographers whom we commission and work with will be asked to confirm that none of the work they submit has been generated or augmented using generative AI "or augmented"
- golemotron 3y ago
  There's no hard line in the technology. This means that a ban is pointless because the landscape is going to keep changing. It's interesting to compare this to other situations where, say, law tries to create lines that aren't really there and the incentive to ignore imaginary ones is greater than the incentive to keep them. This seems to be a very common phenomenon with technology.
  - pxc 3y ago
    In the case of Nature, it functions as a statement of values that scientists publishing with Nature will be happy to comply with to the best of their abilities. I doubt that the editors are under some illusion that the nominal ban will create a hard line in reality. I'd be surprised to learn that that is their idea of success with this measure.
ghaff 3y ago
The context matters. There are image manipulations I might do to a photo I'm going to hang on the wall that wouldn't be kosher if I were shooting an event for a newspaper especially with respect to removing objects from the photo.
charcircuit 3y ago
Some generative AI tools let you input a base image to work off of. You can definitely use generative AI for just sharpening in these tools.
dragonwriter 3y ago
> My understanding is that generative AI is created entirely from software, using a network of previously created images as input. A corrective filter modifies an image taken directly from a sensor instead. Your understanding is incorrect, generative AI can modify an image taken from any source, as well as creating from scratch.

ghaff 3y ago

I think you can argue that there is significant daylight between "created wholly or partly using generative AI" and the sort of ML-based noise reduction, sharpening, etc. that you see in products like Lightroom and Photoshop. Of course, the whole area will evolve and rules like these will have to evolve as well. News photography has dealt with this since pre-digital although different publications may have different standards.

hgsgm 3y ago
They'll have to use an AI to determine what manipulation counts as AI.

chasing 3y ago

“Generative AI.” I know there are kind of weird edge cases. “My iPhone made the sunset way redder than it was in real life.” But I think we all know what they’re talking about and I suspect if you’re in a position for it to really be a concern then you will communicate with Nature and sort out what their comfort zone is.

etrautmann 3y ago
This is fascinating and gets pretty interesting. As a computational neuroscience person, some of the more advanced neural signal processing algorithms use generate models internally to model recorded neural data. The result is likely a smoothed, simplified, and hopefully more interpretable view of neural data, but there's no guarantee that some portion of the resulting multidimensional signal isn't hallucinated. As a result, most findings should be validated by verifying that some property of interest is present in the high dimensional raw neural data, though that's only conceptually possible sometimes.
- deleted 3y ago
  [deleted]

Kapura 3y ago

I am not in the game at all but as I understand it Samsung doesn't make high quality DSLRs of the type used by photographers. I reckon that photographers would be asked to disclose this sort of thing if submitting to Nature in future.

CharlesW 3y ago

> …AI-based generative/corrective algorithms… Every photo is touched by "corrective algorithms". Nature is talking about generative AI specifically, which means using an LLM to generate part or all of an image. This precludes using Midjourney, Photoshop's new "generative fill", etc. I assume that what Samsung's "Space Zoom" feature does — replacing elements with higher-quality stock photography — was already disallowed. If so, whether the elements were identified/replaced manually or automatically isn't really a concern from an editorial perspective.

ghaff 3y ago
Yes, they already had guidelines for photographs. [1] e.g. "Digital images submitted with a manuscript for review should be minimally processed. A certain degree of image processing is acceptable for publication (and for some experiments, fields and techniques is unavoidable), but the final image must correctly represent the original data and conform to community standards. Editors may use software to screen images for manipulation." [1] https://www.nature.com/nature-portfolio/editorial-policies/image-integrity https://www.nature.com/nature-portfolio/editorial-policies/i...
- dclowd9901 3y ago
  Sounds like this would allow for processed astronomical photography too. Methinks this questions wasn’t in earnest.
  - ghaff 3y ago
    No reason to assume bad faith. But some folks get very literal. And if you have a legitimate question, that's one of the things editors are for.
MiguelX413 3y ago
How might one use a Large Language Model “to generate part or all of an image”?
- 3y ago

analog31 3y ago

I'm peripherally involved in this scene. The answer is that the journals don't want processed images, but of course the scientist doesn't always know what kind of processing happened to the image en route to their display and file system. The idea is that an image supposedly constitutes "data" and as such, should represent raw data. Also, what constitutes "raw data" is itself a matter of debate. How raw is raw? Like any interesting pursuit, scientific publishing struggles to keep up with developments in technology.

codetrotter 3y ago
> How raw is raw? Certainly no jpeg image produced by any digital camera is really “raw” as it will already have been through a debayering filter https://en.wikipedia.org/wiki/Bayer_filter https://en.wikipedia.org/wiki/Bayer_filter And then on top of that is the JPEG compression artifacts. But I do wonder how many raw files also contain data that has been debayered already. I have not looked into that. I know that with third party firmware such as Magic Lantern it is possible to get the image data without debayering. https://magiclantern.fm/ https://magiclantern.fm/ Likewise I know that the Camera Module 3 for Raspberry Pi is possible to retrieve the image data from without debayering.
- morphicpro 3y ago
  [flagged]
- sudosysgen 3y ago
  The RAW data for the vast majority of MILCs and DSLRs is pre-debayering.
  - userbinator 3y ago
    Even Android smartphones will give pre-debayering raw data from the sensor if you use the appropriate camera app. (There are quite a few cheap ones where the OEM debayering filter is just horrible, and the sensor is actually capable of much better quality.)
- 3y ago

morphicpro 3y ago

There is also a big distinction between fully disclosing that an image has AI vs the non post production edits. So why not just ask people to just be honest and disclose about the means of creating the content and disclose the images as being "AI" created vs telling people they should to be purists in their craft. Is the claim that the art made by AI itself is harmful or that the act of making it causes harm. Where is the real harm in this given people are properly informed as the the type of content they are looking at? The reasons why is because this has nothing to do with art and expression and everything to do with control under the guise of fear.

deleted 3y ago
[deleted]

belter 3y ago

It was not a problem when presenting the EHT results and subsequent "images" - https://www.space.com/first-ever-black-hole-image-ai-makeover https://www.space.com/first-ever-black-hole-image-ai-makeove...

criddell 3y ago
Were those produced by a generative AI?
- belter 3y ago
  No, but under the same algorithmic principles. How do you think that yellow color come about?

0xBABAD00C 3y ago

> what are the chances they already publish photos taken on devices that apply by default some form of AI-based generative/corrective algorithms 100%?

m3kw9 3y ago

For them purposely asking stable diffusion(not ok) for an image vs iPhone image processing(ok) would be your baseline for distinguishing. Picking at small details seem like a nice way to waste time, you just got to keep it simple

ChatGTP 3y ago

On the other hand, I really hate what these algorithms do to my photos, even my new iPhone which is considered good tech. So I get it.

renonce 3y ago

The part of the article discussing problems with generative AI seems to lay emphasis on "attribution". The problem with generative AI is not that they want raw and unedited images, but that they are unable to properly attribute generated images to their sources. That's what makes the distinction between a generative AI and regular image editing tools like cropping, scaling, filtering, etc. So something like "AI detail enhancement engine" should not be problematic under such a definition as the tool can simply be attributed to Samsung.