4 ms·

Cheese and Chalk. It is very different to generate synthetic datasets to assist in targeted training , vs ingesting LLM output from web scraping.

abeppu 2y ago
Yes -- said another way, if you're an ML researcher and you have human-provided (scraped) data, and an ability to generate synthetic data, then until recently, you had a controllable parameter: how much of your training data for your new model should be synthetic? You can vary this, run multiple experiments, and choose how much synthetic data to use -- and you can vary the specific configs about how that synthetic data is generated. If synthetic data is mixed into your upstream data sources in a way you cannot control, then your ML team loses a valuable controllable parameter.
- miki123211 2y ago
  You still have some that control, but in a much more indirect way. There are three kinds of data now, synthetic, pre-2022 and current. Everything pre-2022 is definitely written by humans, synthetic data is still synthetic, and post-2022 is a mix of both. I wouldn't be surprised if "AI detectors" work somewhat for this use case. They're biased, far from accurate and a terrible idea if you need to make important decisions (like whether to expel a student for cheating), but there's quite a large room for errors here.
  - uberswe 2y ago
    > Everything pre-2022 is definitely written by humans I'm not sure if methods like article spinning counts as written by humans. This is something you could automate before AI and it would take a human written article and randomly swap words with similar meaning throughout to make it seem original.
    - derefr 2y ago
      Don’t forget machine-translated texts, where until ~2017 the translation was likely done by something much dumber / semantically lossy than an LLM, and after 2017 was basically done by an early form of LLM (the Transformers architecture originating in Google Translate.) Many historical English-language news reports published on the English-language websites of foreign news media from non-English-speaking countries, from 1998 (Babelfish era) to ~a few months ago, may be unreliable training data for this reason.
  - mrbungie 2y ago
    They do work detecting LLM outputs that are sampled "naively" (when the model/user is really not trying to pass it as human output). I copied a prompt translated from spanish to english using ChatGPT Plus in a GPT-4o Azure OpenAI Service endpoint. It did work in Spanish but didn't run in english because the default AOS Content Filters detected a jailbreak intent. It was quite weird.
lawlessone 2y ago
I think this is it. Generated data is ok if you're curating it to make sure nothing bad, wrong or insensible comes in. Basically still needs a human in the loop.
- surfingdino 2y ago
  Then why not remove this crap (LLMs) from the loop altogether? How did we get from "AI will replace you" to "your new job will be an AIs janitor" in the space of about 12 months?
  - sieste 2y ago
    there is nothing wrong with being a janitor. you could also call it "AI editor" instead of you want to insert a job title sounds more prestigious. some people find it easier and more enjoyable to edit a first draft generated by a language model based on instructions than writing that first draft themselves.
    - the8thbit 2y ago
      I gotta say, Claude is a godsend for building out quick prototypes of ideas, especially when those ideas require domain specific knowledge that you know a little about but aren't specialized in. Which is most interesting programming projects. Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course.
      - visarga 2y ago
        > Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course. You just start faster, but end at the same time. If you really need to understand something there is no LLM shortcut. I spent hours interrogating Claude, in the same time I could have studied from a book and gotten even better grounding.
mcswell 2y ago
Yeah, I raised the same issue before reading your post; ninja'd I am. I like your "cheese and chalk".
- ddingus 2y ago
  I always preferred sugar and shit. Obviously, that is profane. But, I consider profanity be seen as the part of speech it really is.
  - groby_b 2y ago
    Profane is, if you will, fucking fine. But "cheese and chalk" is a great analogy because both are sources of calcium, but cheese is much better for the human body. It carries useful info.
  - progmetaldev 2y ago
    Also drives your point home more efficiently. While it may be profane, there's far more speech available with far less "use" that is intentionally profane to spark a reaction without regard to what that reaction may be. Shock value for attention, rather than to carry home a point.

abeppu 2y ago

Yes -- said another way, if you're an ML researcher and you have human-provided (scraped) data, and an ability to generate synthetic data, then until recently, you had a controllable parameter: how much of your training data for your new model should be synthetic? You can vary this, run multiple experiments, and choose how much synthetic data to use -- and you can vary the specific configs about how that synthetic data is generated. If synthetic data is mixed into your upstream data sources in a way you cannot control, then your ML team loses a valuable controllable parameter.

miki123211 2y ago
You still have some that control, but in a much more indirect way. There are three kinds of data now, synthetic, pre-2022 and current. Everything pre-2022 is definitely written by humans, synthetic data is still synthetic, and post-2022 is a mix of both. I wouldn't be surprised if "AI detectors" work somewhat for this use case. They're biased, far from accurate and a terrible idea if you need to make important decisions (like whether to expel a student for cheating), but there's quite a large room for errors here.
- uberswe 2y ago
  > Everything pre-2022 is definitely written by humans I'm not sure if methods like article spinning counts as written by humans. This is something you could automate before AI and it would take a human written article and randomly swap words with similar meaning throughout to make it seem original.
  - derefr 2y ago
    Don’t forget machine-translated texts, where until ~2017 the translation was likely done by something much dumber / semantically lossy than an LLM, and after 2017 was basically done by an early form of LLM (the Transformers architecture originating in Google Translate.) Many historical English-language news reports published on the English-language websites of foreign news media from non-English-speaking countries, from 1998 (Babelfish era) to ~a few months ago, may be unreliable training data for this reason.
- mrbungie 2y ago
  They do work detecting LLM outputs that are sampled "naively" (when the model/user is really not trying to pass it as human output). I copied a prompt translated from spanish to english using ChatGPT Plus in a GPT-4o Azure OpenAI Service endpoint. It did work in Spanish but didn't run in english because the default AOS Content Filters detected a jailbreak intent. It was quite weird.

lawlessone 2y ago

I think this is it. Generated data is ok if you're curating it to make sure nothing bad, wrong or insensible comes in. Basically still needs a human in the loop.

surfingdino 2y ago
Then why not remove this crap (LLMs) from the loop altogether? How did we get from "AI will replace you" to "your new job will be an AIs janitor" in the space of about 12 months?
- sieste 2y ago
  there is nothing wrong with being a janitor. you could also call it "AI editor" instead of you want to insert a job title sounds more prestigious. some people find it easier and more enjoyable to edit a first draft generated by a language model based on instructions than writing that first draft themselves.
  - the8thbit 2y ago
    I gotta say, Claude is a godsend for building out quick prototypes of ideas, especially when those ideas require domain specific knowledge that you know a little about but aren't specialized in. Which is most interesting programming projects. Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course.
    - visarga 2y ago
      > Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course. You just start faster, but end at the same time. If you really need to understand something there is no LLM shortcut. I spent hours interrogating Claude, in the same time I could have studied from a book and gotten even better grounding.

mcswell 2y ago

Yeah, I raised the same issue before reading your post; ninja'd I am. I like your "cheese and chalk".

ddingus 2y ago
I always preferred sugar and shit. Obviously, that is profane. But, I consider profanity be seen as the part of speech it really is.
- groby_b 2y ago
  Profane is, if you will, fucking fine. But "cheese and chalk" is a great analogy because both are sources of calcium, but cheese is much better for the human body. It carries useful info.
- progmetaldev 2y ago
  Also drives your point home more efficiently. While it may be profane, there's far more speech available with far less "use" that is intentionally profane to spark a reaction without regard to what that reaction may be. Shock value for attention, rather than to carry home a point.