3 ms·

> …AI-based generative/corrective algorithms… Every photo is touched by "corrective algorithms". Nature is talking about generative AI specifically, which mean

> …AI-based generative/corrective algorithms… Every photo is touched by "corrective algorithms". Nature is talking about generative AI specifically, which means using an LLM to generate part or all of an image. This precludes using Midjourney, Photoshop's new "generative fill", etc. I assume that what Samsung's "Space Zoom" feature does — replacing elements with higher-quality stock photography — was already disallowed. If so, whether the elements were identified/replaced manually or automatically isn't really a concern from an editorial perspective.

ghaff 3y ago
Yes, they already had guidelines for photographs. [1] e.g. "Digital images submitted with a manuscript for review should be minimally processed. A certain degree of image processing is acceptable for publication (and for some experiments, fields and techniques is unavoidable), but the final image must correctly represent the original data and conform to community standards. Editors may use software to screen images for manipulation." [1] https://www.nature.com/nature-portfolio/editorial-policies/image-integrity https://www.nature.com/nature-portfolio/editorial-policies/i...
- dclowd9901 3y ago
  Sounds like this would allow for processed astronomical photography too. Methinks this questions wasn’t in earnest.
  - ghaff 3y ago
    No reason to assume bad faith. But some folks get very literal. And if you have a legitimate question, that's one of the things editors are for.
MiguelX413 3y ago
How might one use a Large Language Model “to generate part or all of an image”?
- PartiallyTyped 3y ago
  Something like this: Hey ChatGPT, write a prompt for midjourney to generate a realistic photo of XYZ with ABC parameters. Then plug it into Mid-journey. Technically the LLM isn't generating the image, and I agree, but I think their point is rather obvious and we need not be intentionally obtuse nor needlessly pedantic.
- seabass-labrax 3y ago
  'Text-to-image' systems like Stable Diffusion really are Large Language Models: an encoder such as BERT creates a mapping from text tokens to the latent space of the image generation model. As part of this training step, the system is learning the concepts of certain words and grammatical constructs. There are quite a few in-depth explanations of the whole system; here's one for instance: https://jalammar.github.io/illustrated-stable-diffusion/ https://jalammar.github.io/illustrated-stable-diffusion/

ghaff 3y ago

Yes, they already had guidelines for photographs. [1] e.g. "Digital images submitted with a manuscript for review should be minimally processed. A certain degree of image processing is acceptable for publication (and for some experiments, fields and techniques is unavoidable), but the final image must correctly represent the original data and conform to community standards. Editors may use software to screen images for manipulation." [1] https://www.nature.com/nature-portfolio/editorial-policies/image-integrity https://www.nature.com/nature-portfolio/editorial-policies/i...

dclowd9901 3y ago
Sounds like this would allow for processed astronomical photography too. Methinks this questions wasn’t in earnest.
- ghaff 3y ago
  No reason to assume bad faith. But some folks get very literal. And if you have a legitimate question, that's one of the things editors are for.

MiguelX413 3y ago

How might one use a Large Language Model “to generate part or all of an image”?

PartiallyTyped 3y ago
Something like this: Hey ChatGPT, write a prompt for midjourney to generate a realistic photo of XYZ with ABC parameters. Then plug it into Mid-journey. Technically the LLM isn't generating the image, and I agree, but I think their point is rather obvious and we need not be intentionally obtuse nor needlessly pedantic.
seabass-labrax 3y ago
'Text-to-image' systems like Stable Diffusion really are Large Language Models: an encoder such as BERT creates a mapping from text tokens to the latent space of the image generation model. As part of this training step, the system is learning the concepts of certain words and grammatical constructs. There are quite a few in-depth explanations of the whole system; here's one for instance: https://jalammar.github.io/illustrated-stable-diffusion/ https://jalammar.github.io/illustrated-stable-diffusion/