3 ms·

Surely, an LLM could handle this, if it were to get fed the entire book in chunks. "Change all mentions of the word 'flat' in its connotation as the British wor

Surely, an LLM could handle this, if it were to get fed the entire book in chunks. "Change all mentions of the word 'flat' in its connotation as the British word for 'apartment', to say 'apartment'. Make no other changes to the manuscript." Then have it double-check its work afterwards. "Does every instance of the word 'apartment' in this text make sense? Does every instance of the word 'flat' in this text make sense in American English?" and then run this prompt on each pair of words that you want it to do. When it's done, run a quick diffcheck on it & check its work to learn of any gotchas.

LordShredda 3y ago
Why the hell does it have to be AI? A regex would be faster to run, test, and diff check. Even better,instead of paying a programmer to waste your time just pay an actual editor to proof read it
- TeMPOraL 3y ago
  It doesn't. It perhaps shouldn't. That said, I found myself doing little text mangling tasks with GPT-4 instead of appropriate command line tools, because fuck if I remember the flags to sort and unique, and it's much faster to just describe what I want and have the LLM take a crack at it. This is why, I think, you'll see a lot of simple(ish) tasks being done by LLMs, at couple orders of magnitude more compute cost - it's just that much more convenient. EDIT: nevermind. That was about just doing regex replace. But if you want to do it correctly in an automated way, LLMs are indeed the best tool for the job in the general case, because they understand how language work, in a way that you can't really formalize in code. No, feeding the complete definition of English grammar to the computer won't help, because a) it probably doesn't exist, and b) even if it does, it's merely a suggestion - natural language is a living thing, and is not bound by fixed rulesets.
  - lalopalota 3y ago
    Typing out the instructions, copying and pasting the data, then analyzing the output for correctness is faster than running `sort -h` ?
    - TeMPOraL 3y ago
      Faster than recalling or finding out again that it's `sort -h`, and not `sort -n` or `xargs -wtf -- sort -xkcd {}` or whatever.
    - messe 3y ago
      It can be less mental effort, not necessarily faster.
- RheingoldRiver 3y ago
  But the LLM already "understands" the changes that need to be made. A regex to figure out if "flat" means "apartment" or not? Consider the text (which I just made up & I'm really not a fiction writer so I apologize for it being terrible): "Oh, I live near there too! House or flat?" Stanley hesitated. These questions were getting more and more personal. Was this her idea of casual conversation? Or was she trying to get to know him personally? Well, he thought, what could go wrong if I treat this like a conversation. "Flat," he said. And then asked a question of his own. "What kind of pop do you like?" How are you going to decide if "flat" is talking about non-fizzy pop/soda, or a dwelling space with a regex? It's a lot closer to the word "pop." Of course, GPT handles this perfectly https://chat.openai.com/share/fb6a56a2-749c-4e71-9506-541bb84cc610 https://chat.openai.com/share/fb6a56a2-749c-4e71-9506-541bb8...
  - hunter2_ 3y ago
    Some places in America do say "pop" instead of "soda," so I'm hesitant to call it perfect... But I'll admit I agree that this is well beyond the scope of regex.
- numpad0 3y ago
  Because replacing all appearances of apartment with apartment just apartment out isn’t going to work?
asddubs 3y ago
instead of feeding it into a black box and hoping it does the right thing, how about just paying someone to manually find each instance and only replace it if appropriate. it's tedious and grueling but assuming every other word isn't "flat" should only take a few hours.
anlaw 3y ago
Surely the effort to manage LLMs is wasted on keeping a publisher from having to do the job they’re paid for. Are we really going to let big corp automate away their role in the social contract? Ostensibly Random House needs to provide real output not just be a brand that farms out all the work to AI, and thus captures value on the fiat ledger. The number of rent seeker non-contributors is too damn high. We cannot base society off dying peoples hallucinations about how the world works anymore. This is bonkers. I never asked to exist. Why am I constrained to beliefs like Bill Gates and the like are divine in contemporary ways?
andrewfong 3y ago
If you're going to use an ML model, you might as well use an existing language translation model rather than asking ChatGPT or the like to convert words one by one. You'd probably get better results treating American English and British English as entirely different languages rather than assuming one is the same as the other but with a handful of different words.

LordShredda 3y ago

Why the hell does it have to be AI? A regex would be faster to run, test, and diff check. Even better,instead of paying a programmer to waste your time just pay an actual editor to proof read it

TeMPOraL 3y ago
It doesn't. It perhaps shouldn't. That said, I found myself doing little text mangling tasks with GPT-4 instead of appropriate command line tools, because fuck if I remember the flags to sort and unique, and it's much faster to just describe what I want and have the LLM take a crack at it. This is why, I think, you'll see a lot of simple(ish) tasks being done by LLMs, at couple orders of magnitude more compute cost - it's just that much more convenient. EDIT: nevermind. That was about just doing regex replace. But if you want to do it correctly in an automated way, LLMs are indeed the best tool for the job in the general case, because they understand how language work, in a way that you can't really formalize in code. No, feeding the complete definition of English grammar to the computer won't help, because a) it probably doesn't exist, and b) even if it does, it's merely a suggestion - natural language is a living thing, and is not bound by fixed rulesets.
- lalopalota 3y ago
  Typing out the instructions, copying and pasting the data, then analyzing the output for correctness is faster than running `sort -h` ?
  - TeMPOraL 3y ago
    Faster than recalling or finding out again that it's `sort -h`, and not `sort -n` or `xargs -wtf -- sort -xkcd {}` or whatever.
  - messe 3y ago
    It can be less mental effort, not necessarily faster.
RheingoldRiver 3y ago
But the LLM already "understands" the changes that need to be made. A regex to figure out if "flat" means "apartment" or not? Consider the text (which I just made up & I'm really not a fiction writer so I apologize for it being terrible): "Oh, I live near there too! House or flat?" Stanley hesitated. These questions were getting more and more personal. Was this her idea of casual conversation? Or was she trying to get to know him personally? Well, he thought, what could go wrong if I treat this like a conversation. "Flat," he said. And then asked a question of his own. "What kind of pop do you like?" How are you going to decide if "flat" is talking about non-fizzy pop/soda, or a dwelling space with a regex? It's a lot closer to the word "pop." Of course, GPT handles this perfectly https://chat.openai.com/share/fb6a56a2-749c-4e71-9506-541bb84cc610 https://chat.openai.com/share/fb6a56a2-749c-4e71-9506-541bb8...
- hunter2_ 3y ago
  Some places in America do say "pop" instead of "soda," so I'm hesitant to call it perfect... But I'll admit I agree that this is well beyond the scope of regex.
numpad0 3y ago
Because replacing all appearances of apartment with apartment just apartment out isn’t going to work?

asddubs 3y ago

instead of feeding it into a black box and hoping it does the right thing, how about just paying someone to manually find each instance and only replace it if appropriate. it's tedious and grueling but assuming every other word isn't "flat" should only take a few hours.

anlaw 3y ago

Surely the effort to manage LLMs is wasted on keeping a publisher from having to do the job they’re paid for. Are we really going to let big corp automate away their role in the social contract? Ostensibly Random House needs to provide real output not just be a brand that farms out all the work to AI, and thus captures value on the fiat ledger. The number of rent seeker non-contributors is too damn high. We cannot base society off dying peoples hallucinations about how the world works anymore. This is bonkers. I never asked to exist. Why am I constrained to beliefs like Bill Gates and the like are divine in contemporary ways?

andrewfong 3y ago

If you're going to use an ML model, you might as well use an existing language translation model rather than asking ChatGPT or the like to convert words one by one. You'd probably get better results treating American English and British English as entirely different languages rather than assuming one is the same as the other but with a handful of different words.