11 ms·

Photoshop for Text

183 points by kepano 4y ago

97s 4y ago
I hope we get some updates in Adobe the address the issue of data merging and downsizing text based on user input to fit a field box. It is incredible that simple things like that are not easy to do natively in the app.
timeon 4y ago
This is directly not on topic but reminds me that for most photo-editing Photoshop is overkill. Photoshop is more then adjusting channels and blurring/sharpening few areas.
oscribinn 4y ago
[flagged]
eimrine 4y ago
> Up until now, text editors have been focused on input. The next evolution of text editors will make it easy to alter Vim can do this, not in the sence you putting into the article but at least it does it without requiring using anything else except 3 rows of keyboard. > Text filters will allow you to paraphrase text, so that you can switch easily between styles of prose: literary, technical, journalistic, legal, and more. Pfff. Styles of prose are: trolling, documenting, cat-talking, legal and just making a list of something. Trolling cannot be augmented, documenting feature begs of some connections to reality, proper cat-talking requires throwing a lot of synonyms really fast, legal is kind of Java programming when you type one line and get 40, and augmenting lists might be done with either md-style but without requiring to draw every symbol of that ascii tables or excel-style but without gui.
- jakear 4y ago
  What do you mean by “cat-talking” here? Bing doesn’t yield much.
  - Minor49er 4y ago
    You don't want Bing. You want Bengal
- yreg 4y ago
  > Vim can do this, not in the sence you putting into the article So in what sense? Some other irrelevant sense?
nonrandomstring 4y ago
There's an area where we've been way, way ahead of the curve for 70 years now. It's sound. Long before GPT, image synthesis, video deep-fakes and these imagined "Photoshop for words", we had sound synthesis. That's a very useful marker. Because we can read the things people were saying about the future of sound, their hopes, fears and predictions as far back as the 1960s when Robert Moog and Wendy Carlos were patching modular synths. Most of the fears and predictions turned out to be rubbish. Musicians, orchestras and live events didn't get replaced. Instead we invented synth-pop bands. And many of the things technologists imagined people would want to do, turned out to be way off the mark. To my knowledge Isao Tomita was the only talented artist to "replace an orchestra" with synthesisers. Most people who used the tools "as intended" were artless, and forgettable. Everyone else ran riot in the parameter space - messing and subverting the technology to get the weirdest punk-ass squelches and wobbles possible. So I always have to look on these "How synthetic X is going to make the real X obsolete" with a pinch of salt.
- deleted 4y ago
  [deleted]
- ShamelessC 4y ago
  > So I always have to look on these "How synthetic X is going to make the real X obsolete" with a pinch of salt. From your comment, it seems that the linked article is far more fear-mongering than it is; I gathered a mostly optimistic tone from it. The final paragraph - > While some of these capabilities sound a bit scary at first, they will eventually become as mundane as “desaturate”, “Gaussian blur” or any regular image filter, and unlock new creative potential.
- NikolaNovak 4y ago
  I think there's difference between synthesizers, which I would compare to something like different paints and brushes and papers, and then Photoshop and Corel draw and whatnot; and where image generators are heading which is more analogous to automatic music generation. I will also put it forward that for reasons I'm ignorant of, eye seems to be more readily fooled than ear. 20 years ago with crappy tools all I had to do was smudge and clone a hydrant in a photo and it would effectively be gone for 99% of observers. But similarly primitive ways of trying to change or alter a sound file were immediatelly noticed by all listeners.
MonkeyMalarky 4y ago
Text style transfer or paraphrasing/rewriting without spending big bucks on fine-tuning a language model would be nice.
aziaziazi 4y ago
The essaie focuses on image and I can’t help making the parallel with digital sound editing tools, which are [in my very uneducated opinion] more numerous and diverse.
oliverbennett 4y ago
Some people are better at writing than editing, some people are better at editing than writing. I find it necessary to heavily edit when writing - up to, and including, this comment. I don’t mind it, and I don’t mind doing it to other people’s writing either, so this new way of doing things appeals to me. I’d be interested to hear what anyone who’s able one-shot their writing thinks of this. I feel like that type of person may have less of a desire for this kind of stuff?
- sho_hn 4y ago
  > I’d be interested to hear what anyone who’s able one-shot their writing thinks of this. I feel like that type of person may have less of a desire for this kind of stuff? Yep. I don't use spell-checking and I don't use auto-complete bars on phone keyboards, largely because I feel it keeps my skills sharp. I would use tools like the article describes when I deem them to have become necessary to stay competitive at what I do, but at the moment I don't feel it's clear-cut whether their use would promote or harm my faculties to write.
  - oliverbennett 4y ago
    That makes sense. What the long-term effects of such 'performance enhancing' tools might mean for how its users think, or externalise those thoughts, is hard to call. I also wonder what it means if everyone is farming out parts of their intellect to similar models, which might be limited pathologically, by training data or enforcement.
Agingcoder 4y ago
Well, discrete images are an approximation/sampling of some kind of continuous process in nature. We therefore have a whole arsenal of tools to process this signal, ie photoshop. Text? I'm not sure what kind of model we have here. Only now that we have word embedding and other nlp models can we hope to do the same kind of thing we do with text as we do with images?
dwrodri 4y ago
I think if you'd like a sample of what the author of the post is referring to, go checkout the Hemingway text editor[1]. It's very simple, and as far as I know makes no use of the advanced language models that have been in vogue for the past few years. What's really opening the door is thinking of these language models as hyper-efficient compression. With Stable diffusion, we can pack enormous amounts of artistic tedium into simple text prompts, with the weights getting small enough to fit on a flash drive[3]. Grammarly[2] also have a desktop app that appears to offer editing advice, although generally I think they're focused on grammatical correctness over anything else. The question I'm left asking myself at the end of the article is, to what end do we need to edit text like Photoshop? Part of me sees this "Photoshop for Text" as something that would be akin to "No Code" tech stacks. Good No-Code/Low-Code solutions usually allow to build specific classes of products (websites, 3D assets) in ways that are faster than the status quo. But anyone who spends enough time in a No Code stack eventually hits the wall where the people who designed the tool had to sacrifice the flexibility of text for the convenience of a GUI. I yearn for the day that we can set a language model loose on something like the NCBI database or arXiv and have it point out open problems in the field to new PhD students. Or have it figure out whether my ablation studies make sense. Or an AI that can generate math proofs for me. A lot of this linked to model interpretability and understanding, but I think the work that DeepMind is doing is showing that there might be a way to utilize this stuff in expert domains sooner than we think. 1: https://hemingwayapp.com/ https://hemingwayapp.com/ 2: https://www.grammarly.com/ https://www.grammarly.com/ 3: https://andys.page/posts/how-to-draw/# https://andys.page/posts/how-to-draw/#
CacheRules 4y ago
Do people really want to read machine generated fluff?
- luisegr 4y ago
  iI have read stuff generated by LaMDA and it is really good at generating synthesis of any topic, I like more what it generates than what I read on wikipedia
- petercooper 4y ago
  I fed your question to GPT-3 with the instruction to "answer this question in flowery, Shakespearean prose" simply because I assumed it would fail. The result? If by "machine generated fluff" thou dost refer to the banal, trite and insipid content that oft pollutes the Interwebs, then nay, I believe not that people desire to read such.
  - CyberDildonics 4y ago
    looks like a lot of hacker news comments
    - eyelidlessness 4y ago
      Ha! I disagree. I find it both kinder and more clear in its reasoning.
vanadium1st 4y ago
Autocomplete in Gmail has been getting more and more robust. At first it only suggested grammatical correction in words. Later it started giving advice on better sentence structure and then it just started suggesting whole sentences. Each of those steps I loved. More often then not it just says what I wanted to say, with less button pushes, and without all of the mistakes that I make as a non-native english speaker. I sound smarter in gmail and I like it. If I didn't have this experience, then giving the machine any input on what I write would seem crazy to me. I would think that language is too personal, too contextual, that I need control over every word and every letter. But now I love writing with the help of the machine. It still feels like me speaking, the machine doesn't add any extra context that I don't approve of. It really feels like the messages are still mine, and the autocomplete just helps me extract my thoughts from my head in a better and more effective way.
- bugfix-66 4y ago
  Imagine how your brain is atrophying. 1. (of body tissue or an organ) waste away, especially as a result of the degeneration of cells, or become vestigial during evolution. "without exercise, the muscles will atrophy"
  - deleted 4y ago
    [deleted]
  - nicbou 4y ago
    I don't think it's atrophying. We said the same thing about spell checkers, and my spelling seemingly hasn't suffered. I'm not a native speaker. It's nice to have training wheels sometimes, even for a language I'm familiar with.
  - fragmede 4y ago
    Just imagine how flaccid the math part of most of our brains must be. Quick! What's 67 * 42?
  - potatoman22 4y ago
    These tools will only become more pervasive, so why does it matter if that part of the brain "atrophies"? I'm sure people had the same worries about mental math during the rise of the calculator
mlsu 4y ago
I cannot stand the idea of this. I know it's coming, I know I can't stop it, but it will be a catastrophic loss. Most people are bad at writing. This will make them "better" at writing, but only in a certain way. I love reading things people write, because it gives me a window into their mind, how they think, who they are at a deep level. That is the joy of reading, and the structure of writing is a big part of that. Now, a lot of people's writing has become homogenized by the computerization of our world already, but only at a low level: spelling, basic grammar, and so on. When people have the ability to inpaint whole paragraphs, dreamed from the blob of internet text (which is mostly corporatized, computerized, email-ized, sterile in the way described above) we will lose something essential. And another problem arises. I send an email asking something, to a coworker or to a friend. They inpainted their response. Did they really understand what I was asking? Did we really communicate at all? Images, I guess, suffer from the same problem. But images are less interpersonal. They are communication, but not communication like writing is communication. In nearly all circumstances, images are less subtle than words are (the subtlety of visual communication happens irl, where such machines have yet to insert themselves).
- toss1 4y ago
  YES! Hugely important point. Worse yet, not only will the 'in-painting' obscure the writer's meaning, or even whether they even had meaning, it will ALSO render that meaning more generic, and eliminate the most information-dense surprising bits. All of these tools are merely synthesizing new text/images/code/etc. from millions of existing examples. When there is any doubt about the intent or the output, it fills in the MOST EXPECTED output. It does NOT fill in the possibly intended but unique image, phrase, code that would be the output of a unique insight. Real brilliance will be simply lost in the sauce. Ugh. I won't be using these, and I'll shun those who do.
  - jameshart 4y ago
    Right - in terms of information theory, automatically-generated infilled data communicates zero bytes of additional information. It could be compressed away entirely losslessly and I could instead generate it myself. There is no need to waste bandwidth sharing it. Communication only has value if it is surprising. That much has been known since Shannon.
severak_cz 4y ago
It's already there - as a Google docs, Wikis and other forms of collaborative editing. This is what makes text great - feedback from others and maybe even their input.
- ghaff 4y ago
  Collaborative editing mostly works in very specific ways. I do not work with editors in a real-time manner for the most part.
andrewstuart 4y ago
Isn't that Canva?
Der_Einzige 4y ago
The paper at COLING 2022 that I wrote, titled "Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studio" included a GUI constrained text generation studio that I market as being "Like Photoshop but for text" https://github.com/Hellisotherpeople/Constrained-Text-Generation-Studio https://github.com/Hellisotherpeople/Constrained-Text-Genera...
apienx 4y ago
Flawed premise IMHO. Images have inherently more entropy than text (all else being equal). More dimensions to "play around with" is what made Photoshop useful and popular. The "thought-processing" angle is cool though.
nicbou 4y ago
This is not comparable. Photoshop lets a conscious hand change the medium. Now we have tools that generate the message out of thin air. We're about to outsource human expression to machines. And why? To automate things we don't want to write about, and automate reading them. I picture a future of machines talking to each other in increasingly clever human language while humans on each end just get the straight talk: "Did you put the cover on the TPS reports?" "Yes, I did. I got the memo." This also ushers in a terrifying era of communication. "I'm so sorry for your loss", says a machine on behalf of someone, using all the right words but not feeling any of them. "Thank you" replies another, on behalf of someone else.
- sergiomattei 4y ago
  I’m confused. What’s the difference between “I’m sorry for your loss” written by a spell checker versus a human? The effect is the same on the reader. This looks hand-wavy to me.
  - nicbou 4y ago
    I expanded my thoughts into a short post, if you care to read it: https://nicolasbouliane.com/blog/machine-talk https://nicolasbouliane.com/blog/machine-talk Spelling and grammar checkers are amazing. It becomes a problem when we outsource expressing ourselves to a machine. If kind, thoughtful words come from an artificial brain, expressing sympathies just becomes a box-ticking exercise. We mocked "press F to pay respects", but we've already built it in real life. "Congratulate John about his new job" (LinkedIn), "Wish Jane a happy birthday" (Facebook). Having the machine write the message for us is the next step, and I find it terrifying. What value is there to a gesture if it asks nothing of us? No thoughtfulness, hell, no thought. At this point, you might as well go all-in. Attach a pen to a CNC, connect AI to it, and offer heartfelt handwritten notes as a service. Offer to connect it to social media and auto-detect heartfelt handwritten letter opportunities. Have that letter in the mail while the corpse is still warm.
    - potatoman22 4y ago
      I'll add one to your facebook/linkedin example: Hallmark cards.
CPUTranslator 4y ago
To me, this article misses the point, but only slightly. If we compare the proposed Photoshop for text and the actual Photoshop for images, those are very different tools in how they work with their respective mediums. Photoshop doesn’t use AI models to generate new things for the artist, photographer, whoever (it does have generation of course, but not in the same way). A tool like Photoshop for text, in the way described, seems more like a writing aid than a writing tool; something I can give vague commands like “make this paragraph more academic” (whatever that means), and it’ll spit out something approximating the academic style it analyzed. Whereas “auto balance the levels of this image” is much more concrete in what it means, there’s no approximation of “style.” I feel like Photoshop for text should be an editor that cares more about the structure of stories and who’s in them, with ways to organize big chunks in easy ways, rather than something that generates content for you.
- kepano 4y ago
  The article is more about user experience than implementation. When you apply an effect in Photoshop you can change all relevant pixels at once. Whereas we don’t have good tools to change all words in a document at once (while retaining the intended meaning)
  - consumer451 4y ago
    I get your point, but your comment made me search for a Photoshop Stable Diffusion plug-in. I had no idea. I’m decent with Photoshop but have not had a need for it lately. This makes me want to fire it back up. Each video is 1-2 minutes long. Mind blown again. https://youtu.be/XdctF1uTLBo https://youtu.be/XdctF1uTLBo https://youtu.be/mn1PV6HqXGU https://youtu.be/mn1PV6HqXGU https://youtu.be/Wo6ZDYFCWTY https://youtu.be/Wo6ZDYFCWTY
  - fragmede 4y ago
    Oh that would be cool. Write a paragraph about a red ball, then halfway through, decide it's a blue ball. The editor then highlights all occurrences of the variable and rewrites them for you. Like the "Refactor local variable name" bit in VSCode.
LexGray 4y ago
There are tons of filters people need today. There are no longer secretaries proofreading or lawyers reviewing given the get it done in an hour demands put on people. I would pay a ton for something that automatically fixes know what Twitter is angry about today so I don’t need to keep up on trends, be able to make sure language is gender neutral an unlikely to offend special interests, warn of internationally sensitive phrasing, reword anything that could be hate speech, warn when you are likely going to tick off the SEC. People are far too emotionally invested in what others say and it would be great if you could just send public text through a few filters in the name of not publicly embarrassing yourself. I should imaging a popup saying “that is dumb, illogical, and going to violate a TOS unless you change it like this” will shortly be a common popup instead of flags for review. The surprise is not that it is coming, but that it hasn’t been a thing for years given how flammable the internet is.
LegoZombieKing 4y ago
This idea is really interesting. I’ve been using Copilot since it became publicly available for personal side projects. Recently, I ran out of google drive space and had to use something other than google docs to write my essay for school. I booted up a text document and started writing, and then copilot started writing with me. I ran it through a plagiarism checker, and it turned out to be clean, which surprised me. Can’t say that what it generated was particularly insightful, however it was helpful to reach an obituary word count.
- mattsahr 4y ago
  > it was helpful to reach an obituary word count. I'm pretty sure you meant "obligatory" word count. But "obituary" is an awesome accident. Like Copilot is waiting, happy to sum up your life in a tidy paragraph, when the time comes.
  - LegoZombieKing 4y ago
    Hahaha, you got me there! I’ll be caught one day and pay for the hours I’ve saved. At the end of the day if I can’t read what I wrote thats on me.
ulfw 4y ago
Photoshop for text exists for decades. It's called a desktop publishing app. Those were all the rage in the 80s, from Pagemaker to QuarkXpress and even Microsoft Publisher.
quickthrower2 4y ago
The difference is our brains can immediately judge the result of an image transform, but it takes a re-read to see if the text transform hasn't done something bad, like lost the meaning of the text. A blurry image is less likely to get you into trouble than saying the wrong words, so it is critical to validate manually what it says. That said, such tools could be useful for recommendations on how to rephrase things.
rocqua 4y ago
One bif difference between image editing and text editing. I can see the effect of a change to an image at a glance. Judging whether it is a much worse outcome takes seconds usually. With wholesale changes to text, seeing if it inadvertently makes something worse takes minutes. It's orders of magnitude slower. That makes the editing process much slower, whilst still fraught with risk. I am not talking about not trusting AI to get it right. But about a general change that happens not to work. Maybe your academic writing was better of with one narrative paragraph. Maybe you want to change tense but forgot to mark a quote and you are putting false words in someone's mouth. We proofread changes by human editors. AI editors will require the same.
- cfn 4y ago
  Maybe we can develop number of text classifiers that would give you hints on the effect of the changes without you having to read it. Or maybe even use AI proof readers that would give you an opinion about the new text. It is an interesting field for sure.
- kang 4y ago
  Only if the image is small(within the viewspan). If the image was huge, the brain needs to hold context while you finish seeing it, similar to how brain needs to hold context while you finish reading the comprehension to derive meaning. (This is called sphota theory of grammer as given by ancient sanskrit scholar Bhartihari). Reading is zoomed out seeing. (Edit: Abhinavagupt further showed that sound naturally works on this principle, where the meaning of a word is concluded as the sounds finish hitting the eardrums.)
- dalmo3 4y ago
  This got me thinking. An image is 2D in space, 0D in time. Because we perceive 3D/1D, i.e. one dimension above in each aspect, we can see a whole image at once. We can't see an entire video (2D/1D) at once - we have to move in time. We can't see an entire 3D object at once either - we have to move in space. Technically, text is just as visual as an image. However, meaning is conveyed by the sequence in which we interpret the symbols, and that sequence is 0D in space, 1D in time. Maybe beings in an 2D time dimension would be able to parse text instantly?
qwerty456127 4y ago
I wish my text editor had a field where I could enter a snippet of python code which would be fed the opened/selected text to process it and replace it with its output. I often do a lot of logically and sequentially complex search-and-replace which becomes a lot of manual labor. As a simple example" replace all the keyboard quotes with “open-close quote pairs” while making sure the final dot is placed outside rather than “inside the quote.”
- Altho 4y ago
  would shell commands not be a good way to do that ?
  - jcparkyn 4y ago
    Having a live preview of the diff before any changes were applied (like vscode and most editors do when using find/replace) would make this a lot more useful.
  - qwerty456127 4y ago
    I do it this way now - push a file through a script in the command line. But I would be more happy with a handy gui-mouse-click-copy-paste solution. Ideally I'd like a panel in something Sublime/Gedit-like where I would just paste a python function, click run and have it applied without even saving the file to the hard drive.
- alpaca128 4y ago
  In Vim you can already push marked text through any shell command, including python scripts that read from stdin. You could probably make this work with a few lines and a keybinding. I rarely need more than the builtin macro system though, as macros can basically do anything including regex-based search it can do any formatting or change as long as the steps stay the same each iteration.
  - qwerty456127 4y ago
    Vim is not my type yet (perhaps later as I grow up for it), I'd prefer something like Sublime, Kate or Gedit.
amylodosys 4y ago
I feel as the person who wrote this never worked in publishing... As an ex-editor now working in IT I find this post a bit... ludicrous? Anyone else feels that or am I missing something here?
seydor 4y ago
I often call vim the "photoshop of text". command mode has a rigidity like an image , but you can cut copy and paste parts of it. AI text generations will be more like games than like photoshop. Photoshop is 2 step: do and undo, but editing text is more like a continuous stream of refinements.
- alpaca128 4y ago
  Yes, I think macros and regex-based Vim features are more fitting for the comparison to Photoshop than AI-powered features which are a new addition to image/video editing as well. Piping a selection through a shell command or regex based replace is like applying a filter to a selected set of pixels.
T3OU-736 4y ago
How interesting. The more things change, the more they stay the same. "Photoshop for Text" is, arguably, called "Typesetting". Aldus Pagemaker and Quark XPress were both quite popular at this task.
coldtea 4y ago
Some of the stuff he describes like "summarizing text" and such is not like the core Photoshop (or all of Photoshop as it was until very recently, like 5-6 years ago), but more like AI stuff - see: #. For core text processing that would be similar to how a bitmap editor processes text (fitering, replacement, conversion, etc.), there are some tools, like the aptly named: https://textsoap.com/mac/ https://textsoap.com/mac/ # Photoshop has since added some AI stuff, like for object removal and such, but its main functionality wasn't and is still not about that).
dwighttk 4y ago
>The filters will be … as good as if you wrote the text yourself. Always the promise

183 points by kepano 4y ago

97s 4y ago

I hope we get some updates in Adobe the address the issue of data merging and downsizing text based on user input to fit a field box. It is incredible that simple things like that are not easy to do natively in the app.

timeon 4y ago

This is directly not on topic but reminds me that for most photo-editing Photoshop is overkill. Photoshop is more then adjusting channels and blurring/sharpening few areas.

oscribinn 4y ago

[flagged]

eimrine 4y ago

> Up until now, text editors have been focused on input. The next evolution of text editors will make it easy to alter Vim can do this, not in the sence you putting into the article but at least it does it without requiring using anything else except 3 rows of keyboard. > Text filters will allow you to paraphrase text, so that you can switch easily between styles of prose: literary, technical, journalistic, legal, and more. Pfff. Styles of prose are: trolling, documenting, cat-talking, legal and just making a list of something. Trolling cannot be augmented, documenting feature begs of some connections to reality, proper cat-talking requires throwing a lot of synonyms really fast, legal is kind of Java programming when you type one line and get 40, and augmenting lists might be done with either md-style but without requiring to draw every symbol of that ascii tables or excel-style but without gui.

jakear 4y ago
What do you mean by “cat-talking” here? Bing doesn’t yield much.
- Minor49er 4y ago
  You don't want Bing. You want Bengal
yreg 4y ago
> Vim can do this, not in the sence you putting into the article So in what sense? Some other irrelevant sense?

nonrandomstring 4y ago

There's an area where we've been way, way ahead of the curve for 70 years now. It's sound. Long before GPT, image synthesis, video deep-fakes and these imagined "Photoshop for words", we had sound synthesis. That's a very useful marker. Because we can read the things people were saying about the future of sound, their hopes, fears and predictions as far back as the 1960s when Robert Moog and Wendy Carlos were patching modular synths. Most of the fears and predictions turned out to be rubbish. Musicians, orchestras and live events didn't get replaced. Instead we invented synth-pop bands. And many of the things technologists imagined people would want to do, turned out to be way off the mark. To my knowledge Isao Tomita was the only talented artist to "replace an orchestra" with synthesisers. Most people who used the tools "as intended" were artless, and forgettable. Everyone else ran riot in the parameter space - messing and subverting the technology to get the weirdest punk-ass squelches and wobbles possible. So I always have to look on these "How synthetic X is going to make the real X obsolete" with a pinch of salt.

deleted 4y ago
[deleted]
ShamelessC 4y ago
> So I always have to look on these "How synthetic X is going to make the real X obsolete" with a pinch of salt. From your comment, it seems that the linked article is far more fear-mongering than it is; I gathered a mostly optimistic tone from it. The final paragraph - > While some of these capabilities sound a bit scary at first, they will eventually become as mundane as “desaturate”, “Gaussian blur” or any regular image filter, and unlock new creative potential.
NikolaNovak 4y ago
I think there's difference between synthesizers, which I would compare to something like different paints and brushes and papers, and then Photoshop and Corel draw and whatnot; and where image generators are heading which is more analogous to automatic music generation. I will also put it forward that for reasons I'm ignorant of, eye seems to be more readily fooled than ear. 20 years ago with crappy tools all I had to do was smudge and clone a hydrant in a photo and it would effectively be gone for 99% of observers. But similarly primitive ways of trying to change or alter a sound file were immediatelly noticed by all listeners.

MonkeyMalarky 4y ago

Text style transfer or paraphrasing/rewriting without spending big bucks on fine-tuning a language model would be nice.

aziaziazi 4y ago

The essaie focuses on image and I can’t help making the parallel with digital sound editing tools, which are [in my very uneducated opinion] more numerous and diverse.

oliverbennett 4y ago

Some people are better at writing than editing, some people are better at editing than writing. I find it necessary to heavily edit when writing - up to, and including, this comment. I don’t mind it, and I don’t mind doing it to other people’s writing either, so this new way of doing things appeals to me. I’d be interested to hear what anyone who’s able one-shot their writing thinks of this. I feel like that type of person may have less of a desire for this kind of stuff?

sho_hn 4y ago
> I’d be interested to hear what anyone who’s able one-shot their writing thinks of this. I feel like that type of person may have less of a desire for this kind of stuff? Yep. I don't use spell-checking and I don't use auto-complete bars on phone keyboards, largely because I feel it keeps my skills sharp. I would use tools like the article describes when I deem them to have become necessary to stay competitive at what I do, but at the moment I don't feel it's clear-cut whether their use would promote or harm my faculties to write.
- oliverbennett 4y ago
  That makes sense. What the long-term effects of such 'performance enhancing' tools might mean for how its users think, or externalise those thoughts, is hard to call. I also wonder what it means if everyone is farming out parts of their intellect to similar models, which might be limited pathologically, by training data or enforcement.

Agingcoder 4y ago

Well, discrete images are an approximation/sampling of some kind of continuous process in nature. We therefore have a whole arsenal of tools to process this signal, ie photoshop. Text? I'm not sure what kind of model we have here. Only now that we have word embedding and other nlp models can we hope to do the same kind of thing we do with text as we do with images?

dwrodri 4y ago

I think if you'd like a sample of what the author of the post is referring to, go checkout the Hemingway text editor[1]. It's very simple, and as far as I know makes no use of the advanced language models that have been in vogue for the past few years. What's really opening the door is thinking of these language models as hyper-efficient compression. With Stable diffusion, we can pack enormous amounts of artistic tedium into simple text prompts, with the weights getting small enough to fit on a flash drive[3]. Grammarly[2] also have a desktop app that appears to offer editing advice, although generally I think they're focused on grammatical correctness over anything else. The question I'm left asking myself at the end of the article is, to what end do we need to edit text like Photoshop? Part of me sees this "Photoshop for Text" as something that would be akin to "No Code" tech stacks. Good No-Code/Low-Code solutions usually allow to build specific classes of products (websites, 3D assets) in ways that are faster than the status quo. But anyone who spends enough time in a No Code stack eventually hits the wall where the people who designed the tool had to sacrifice the flexibility of text for the convenience of a GUI. I yearn for the day that we can set a language model loose on something like the NCBI database or arXiv and have it point out open problems in the field to new PhD students. Or have it figure out whether my ablation studies make sense. Or an AI that can generate math proofs for me. A lot of this linked to model interpretability and understanding, but I think the work that DeepMind is doing is showing that there might be a way to utilize this stuff in expert domains sooner than we think. 1: https://hemingwayapp.com/ https://hemingwayapp.com/ 2: https://www.grammarly.com/ https://www.grammarly.com/ 3: https://andys.page/posts/how-to-draw/# https://andys.page/posts/how-to-draw/#

CacheRules 4y ago

Do people really want to read machine generated fluff?

luisegr 4y ago
iI have read stuff generated by LaMDA and it is really good at generating synthesis of any topic, I like more what it generates than what I read on wikipedia
petercooper 4y ago
I fed your question to GPT-3 with the instruction to "answer this question in flowery, Shakespearean prose" simply because I assumed it would fail. The result? If by "machine generated fluff" thou dost refer to the banal, trite and insipid content that oft pollutes the Interwebs, then nay, I believe not that people desire to read such.
- CyberDildonics 4y ago
  looks like a lot of hacker news comments
  - eyelidlessness 4y ago
    Ha! I disagree. I find it both kinder and more clear in its reasoning.

vanadium1st 4y ago

Autocomplete in Gmail has been getting more and more robust. At first it only suggested grammatical correction in words. Later it started giving advice on better sentence structure and then it just started suggesting whole sentences. Each of those steps I loved. More often then not it just says what I wanted to say, with less button pushes, and without all of the mistakes that I make as a non-native english speaker. I sound smarter in gmail and I like it. If I didn't have this experience, then giving the machine any input on what I write would seem crazy to me. I would think that language is too personal, too contextual, that I need control over every word and every letter. But now I love writing with the help of the machine. It still feels like me speaking, the machine doesn't add any extra context that I don't approve of. It really feels like the messages are still mine, and the autocomplete just helps me extract my thoughts from my head in a better and more effective way.

bugfix-66 4y ago
Imagine how your brain is atrophying. 1. (of body tissue or an organ) waste away, especially as a result of the degeneration of cells, or become vestigial during evolution. "without exercise, the muscles will atrophy"
- deleted 4y ago
  [deleted]
- nicbou 4y ago
  I don't think it's atrophying. We said the same thing about spell checkers, and my spelling seemingly hasn't suffered. I'm not a native speaker. It's nice to have training wheels sometimes, even for a language I'm familiar with.
- fragmede 4y ago
  Just imagine how flaccid the math part of most of our brains must be. Quick! What's 67 * 42?
- potatoman22 4y ago
  These tools will only become more pervasive, so why does it matter if that part of the brain "atrophies"? I'm sure people had the same worries about mental math during the rise of the calculator

mlsu 4y ago

I cannot stand the idea of this. I know it's coming, I know I can't stop it, but it will be a catastrophic loss. Most people are bad at writing. This will make them "better" at writing, but only in a certain way. I love reading things people write, because it gives me a window into their mind, how they think, who they are at a deep level. That is the joy of reading, and the structure of writing is a big part of that. Now, a lot of people's writing has become homogenized by the computerization of our world already, but only at a low level: spelling, basic grammar, and so on. When people have the ability to inpaint whole paragraphs, dreamed from the blob of internet text (which is mostly corporatized, computerized, email-ized, sterile in the way described above) we will lose something essential. And another problem arises. I send an email asking something, to a coworker or to a friend. They inpainted their response. Did they really understand what I was asking? Did we really communicate at all? Images, I guess, suffer from the same problem. But images are less interpersonal. They are communication, but not communication like writing is communication. In nearly all circumstances, images are less subtle than words are (the subtlety of visual communication happens irl, where such machines have yet to insert themselves).

toss1 4y ago
YES! Hugely important point. Worse yet, not only will the 'in-painting' obscure the writer's meaning, or even whether they even had meaning, it will ALSO render that meaning more generic, and eliminate the most information-dense surprising bits. All of these tools are merely synthesizing new text/images/code/etc. from millions of existing examples. When there is any doubt about the intent or the output, it fills in the MOST EXPECTED output. It does NOT fill in the possibly intended but unique image, phrase, code that would be the output of a unique insight. Real brilliance will be simply lost in the sauce. Ugh. I won't be using these, and I'll shun those who do.
- jameshart 4y ago
  Right - in terms of information theory, automatically-generated infilled data communicates zero bytes of additional information. It could be compressed away entirely losslessly and I could instead generate it myself. There is no need to waste bandwidth sharing it. Communication only has value if it is surprising. That much has been known since Shannon.

severak_cz 4y ago

It's already there - as a Google docs, Wikis and other forms of collaborative editing. This is what makes text great - feedback from others and maybe even their input.

ghaff 4y ago
Collaborative editing mostly works in very specific ways. I do not work with editors in a real-time manner for the most part.

andrewstuart 4y ago

Isn't that Canva?

Der_Einzige 4y ago

The paper at COLING 2022 that I wrote, titled "Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studio" included a GUI constrained text generation studio that I market as being "Like Photoshop but for text" https://github.com/Hellisotherpeople/Constrained-Text-Generation-Studio https://github.com/Hellisotherpeople/Constrained-Text-Genera...

apienx 4y ago

Flawed premise IMHO. Images have inherently more entropy than text (all else being equal). More dimensions to "play around with" is what made Photoshop useful and popular. The "thought-processing" angle is cool though.

nicbou 4y ago

This is not comparable. Photoshop lets a conscious hand change the medium. Now we have tools that generate the message out of thin air. We're about to outsource human expression to machines. And why? To automate things we don't want to write about, and automate reading them. I picture a future of machines talking to each other in increasingly clever human language while humans on each end just get the straight talk: "Did you put the cover on the TPS reports?" "Yes, I did. I got the memo." This also ushers in a terrifying era of communication. "I'm so sorry for your loss", says a machine on behalf of someone, using all the right words but not feeling any of them. "Thank you" replies another, on behalf of someone else.

sergiomattei 4y ago
I’m confused. What’s the difference between “I’m sorry for your loss” written by a spell checker versus a human? The effect is the same on the reader. This looks hand-wavy to me.
- nicbou 4y ago
  I expanded my thoughts into a short post, if you care to read it: https://nicolasbouliane.com/blog/machine-talk https://nicolasbouliane.com/blog/machine-talk Spelling and grammar checkers are amazing. It becomes a problem when we outsource expressing ourselves to a machine. If kind, thoughtful words come from an artificial brain, expressing sympathies just becomes a box-ticking exercise. We mocked "press F to pay respects", but we've already built it in real life. "Congratulate John about his new job" (LinkedIn), "Wish Jane a happy birthday" (Facebook). Having the machine write the message for us is the next step, and I find it terrifying. What value is there to a gesture if it asks nothing of us? No thoughtfulness, hell, no thought. At this point, you might as well go all-in. Attach a pen to a CNC, connect AI to it, and offer heartfelt handwritten notes as a service. Offer to connect it to social media and auto-detect heartfelt handwritten letter opportunities. Have that letter in the mail while the corpse is still warm.
  - potatoman22 4y ago
    I'll add one to your facebook/linkedin example: Hallmark cards.

CPUTranslator 4y ago

To me, this article misses the point, but only slightly. If we compare the proposed Photoshop for text and the actual Photoshop for images, those are very different tools in how they work with their respective mediums. Photoshop doesn’t use AI models to generate new things for the artist, photographer, whoever (it does have generation of course, but not in the same way). A tool like Photoshop for text, in the way described, seems more like a writing aid than a writing tool; something I can give vague commands like “make this paragraph more academic” (whatever that means), and it’ll spit out something approximating the academic style it analyzed. Whereas “auto balance the levels of this image” is much more concrete in what it means, there’s no approximation of “style.” I feel like Photoshop for text should be an editor that cares more about the structure of stories and who’s in them, with ways to organize big chunks in easy ways, rather than something that generates content for you.

kepano 4y ago
The article is more about user experience than implementation. When you apply an effect in Photoshop you can change all relevant pixels at once. Whereas we don’t have good tools to change all words in a document at once (while retaining the intended meaning)
- consumer451 4y ago
  I get your point, but your comment made me search for a Photoshop Stable Diffusion plug-in. I had no idea. I’m decent with Photoshop but have not had a need for it lately. This makes me want to fire it back up. Each video is 1-2 minutes long. Mind blown again. https://youtu.be/XdctF1uTLBo https://youtu.be/XdctF1uTLBo https://youtu.be/mn1PV6HqXGU https://youtu.be/mn1PV6HqXGU https://youtu.be/Wo6ZDYFCWTY https://youtu.be/Wo6ZDYFCWTY
- fragmede 4y ago
  Oh that would be cool. Write a paragraph about a red ball, then halfway through, decide it's a blue ball. The editor then highlights all occurrences of the variable and rewrites them for you. Like the "Refactor local variable name" bit in VSCode.

LexGray 4y ago

There are tons of filters people need today. There are no longer secretaries proofreading or lawyers reviewing given the get it done in an hour demands put on people. I would pay a ton for something that automatically fixes know what Twitter is angry about today so I don’t need to keep up on trends, be able to make sure language is gender neutral an unlikely to offend special interests, warn of internationally sensitive phrasing, reword anything that could be hate speech, warn when you are likely going to tick off the SEC. People are far too emotionally invested in what others say and it would be great if you could just send public text through a few filters in the name of not publicly embarrassing yourself. I should imaging a popup saying “that is dumb, illogical, and going to violate a TOS unless you change it like this” will shortly be a common popup instead of flags for review. The surprise is not that it is coming, but that it hasn’t been a thing for years given how flammable the internet is.

LegoZombieKing 4y ago

This idea is really interesting. I’ve been using Copilot since it became publicly available for personal side projects. Recently, I ran out of google drive space and had to use something other than google docs to write my essay for school. I booted up a text document and started writing, and then copilot started writing with me. I ran it through a plagiarism checker, and it turned out to be clean, which surprised me. Can’t say that what it generated was particularly insightful, however it was helpful to reach an obituary word count.

mattsahr 4y ago
> it was helpful to reach an obituary word count. I'm pretty sure you meant "obligatory" word count. But "obituary" is an awesome accident. Like Copilot is waiting, happy to sum up your life in a tidy paragraph, when the time comes.
- LegoZombieKing 4y ago
  Hahaha, you got me there! I’ll be caught one day and pay for the hours I’ve saved. At the end of the day if I can’t read what I wrote thats on me.

ulfw 4y ago

Photoshop for text exists for decades. It's called a desktop publishing app. Those were all the rage in the 80s, from Pagemaker to QuarkXpress and even Microsoft Publisher.

quickthrower2 4y ago

The difference is our brains can immediately judge the result of an image transform, but it takes a re-read to see if the text transform hasn't done something bad, like lost the meaning of the text. A blurry image is less likely to get you into trouble than saying the wrong words, so it is critical to validate manually what it says. That said, such tools could be useful for recommendations on how to rephrase things.

rocqua 4y ago

One bif difference between image editing and text editing. I can see the effect of a change to an image at a glance. Judging whether it is a much worse outcome takes seconds usually. With wholesale changes to text, seeing if it inadvertently makes something worse takes minutes. It's orders of magnitude slower. That makes the editing process much slower, whilst still fraught with risk. I am not talking about not trusting AI to get it right. But about a general change that happens not to work. Maybe your academic writing was better of with one narrative paragraph. Maybe you want to change tense but forgot to mark a quote and you are putting false words in someone's mouth. We proofread changes by human editors. AI editors will require the same.

cfn 4y ago
Maybe we can develop number of text classifiers that would give you hints on the effect of the changes without you having to read it. Or maybe even use AI proof readers that would give you an opinion about the new text. It is an interesting field for sure.
kang 4y ago
Only if the image is small(within the viewspan). If the image was huge, the brain needs to hold context while you finish seeing it, similar to how brain needs to hold context while you finish reading the comprehension to derive meaning. (This is called sphota theory of grammer as given by ancient sanskrit scholar Bhartihari). Reading is zoomed out seeing. (Edit: Abhinavagupt further showed that sound naturally works on this principle, where the meaning of a word is concluded as the sounds finish hitting the eardrums.)
dalmo3 4y ago
This got me thinking. An image is 2D in space, 0D in time. Because we perceive 3D/1D, i.e. one dimension above in each aspect, we can see a whole image at once. We can't see an entire video (2D/1D) at once - we have to move in time. We can't see an entire 3D object at once either - we have to move in space. Technically, text is just as visual as an image. However, meaning is conveyed by the sequence in which we interpret the symbols, and that sequence is 0D in space, 1D in time. Maybe beings in an 2D time dimension would be able to parse text instantly?

qwerty456127 4y ago

I wish my text editor had a field where I could enter a snippet of python code which would be fed the opened/selected text to process it and replace it with its output. I often do a lot of logically and sequentially complex search-and-replace which becomes a lot of manual labor. As a simple example" replace all the keyboard quotes with “open-close quote pairs” while making sure the final dot is placed outside rather than “inside the quote.”

Altho 4y ago
would shell commands not be a good way to do that ?
- jcparkyn 4y ago
  Having a live preview of the diff before any changes were applied (like vscode and most editors do when using find/replace) would make this a lot more useful.
- qwerty456127 4y ago
  I do it this way now - push a file through a script in the command line. But I would be more happy with a handy gui-mouse-click-copy-paste solution. Ideally I'd like a panel in something Sublime/Gedit-like where I would just paste a python function, click run and have it applied without even saving the file to the hard drive.
alpaca128 4y ago
In Vim you can already push marked text through any shell command, including python scripts that read from stdin. You could probably make this work with a few lines and a keybinding. I rarely need more than the builtin macro system though, as macros can basically do anything including regex-based search it can do any formatting or change as long as the steps stay the same each iteration.
- qwerty456127 4y ago
  Vim is not my type yet (perhaps later as I grow up for it), I'd prefer something like Sublime, Kate or Gedit.

amylodosys 4y ago

I feel as the person who wrote this never worked in publishing... As an ex-editor now working in IT I find this post a bit... ludicrous? Anyone else feels that or am I missing something here?

seydor 4y ago

I often call vim the "photoshop of text". command mode has a rigidity like an image , but you can cut copy and paste parts of it. AI text generations will be more like games than like photoshop. Photoshop is 2 step: do and undo, but editing text is more like a continuous stream of refinements.

alpaca128 4y ago
Yes, I think macros and regex-based Vim features are more fitting for the comparison to Photoshop than AI-powered features which are a new addition to image/video editing as well. Piping a selection through a shell command or regex based replace is like applying a filter to a selected set of pixels.

T3OU-736 4y ago

How interesting. The more things change, the more they stay the same. "Photoshop for Text" is, arguably, called "Typesetting". Aldus Pagemaker and Quark XPress were both quite popular at this task.

coldtea 4y ago

Some of the stuff he describes like "summarizing text" and such is not like the core Photoshop (or all of Photoshop as it was until very recently, like 5-6 years ago), but more like AI stuff - see: #. For core text processing that would be similar to how a bitmap editor processes text (fitering, replacement, conversion, etc.), there are some tools, like the aptly named: https://textsoap.com/mac/ https://textsoap.com/mac/ # Photoshop has since added some AI stuff, like for object removal and such, but its main functionality wasn't and is still not about that).

dwighttk 4y ago

>The filters will be … as good as if you wrote the text yourself. Always the promise