29 ms·

I think this is it. Generated data is ok if you're curating it to make sure nothing bad, wrong or insensible comes in. Basically still needs a human in the lo

I think this is it. Generated data is ok if you're curating it to make sure nothing bad, wrong or insensible comes in. Basically still needs a human in the loop.

surfingdino 2y ago
Then why not remove this crap (LLMs) from the loop altogether? How did we get from "AI will replace you" to "your new job will be an AIs janitor" in the space of about 12 months?
- sieste 2y ago
  there is nothing wrong with being a janitor. you could also call it "AI editor" instead of you want to insert a job title sounds more prestigious. some people find it easier and more enjoyable to edit a first draft generated by a language model based on instructions than writing that first draft themselves.
  - the8thbit 2y ago
    I gotta say, Claude is a godsend for building out quick prototypes of ideas, especially when those ideas require domain specific knowledge that you know a little about but aren't specialized in. Which is most interesting programming projects. Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course.
    - visarga 2y ago
      > Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course. You just start faster, but end at the same time. If you really need to understand something there is no LLM shortcut. I spent hours interrogating Claude, in the same time I could have studied from a book and gotten even better grounding.
      - the8thbit 2y ago
        As I said, "there's a place for that too, of course." I don't think Claude is a good choice if you're trying to prototype a project which uses tools that you don't understand conceptually. However, if you already have a pretty good understanding of the tools, and you're good at reading code, documenting desired functionality, and writing user story requirements then its an amazing shortcut. Basically, if you are prepared to be the team lead or architect of a project then Claude can function as a junior dev who: * has a pretty good score on hackerrank * happens to have the exact right domain specific knowledge for the project you want to build * still gets disoriented by medium and large sized codebases, as many juniors are wont to do (you will need to take over as the main developer, or involve an intermediate or senior developer once the project grows to that size) As an example, the other day I wanted to prototype a project using typescript, react-konva, and tone.js. I already have a strong understanding of typescript, react, HTML canvas, and FM synthesis. What I don't have is an encyclopedic knowledge of the APIs these specific tools expose, nor do I have code sitting in front of me which effectively combines them. If I document the functionality I want well, Claude is really good at taking that documentation and building out either that prototype or the foundation for that prototype. Another thing that I find that helps is to add an intermediate step. Describe the functionality you want the prototype to achieve, and then ask Claude to write a project proposal which documents this functionality and breaks the procedure for producing that functionality into actionable steps. You can then save the artifact it generates to the project files, and have it iterate through that. You'll eventually veer off course as the functionality you want shifts, or the order and granularity of tasks diverges from the plan which was originally designed, but it acts as a way to start a project with a much stronger foundation than just saying "I want a thing that does X. Now make it do Y too. Now make it do Z as well. etc..." Another way to use Claude effectively, which I also utilized for the project I'm talking about, is to use Claude for throwaway prototyping. Rather than having Claude build out a single prototype, and then taking the reigns from there, have it build out one prototype, then scrap that one and have it build another from scratch, then scrap that and have it build a third from scratch. Each iteration you'll learn a little more about how the functionality and structure you specified actually operates, and what Claude struggles with in relation to your project. This allows the next prototype to be built out with a little more of the functionality you want, and a little bit of a cleaner architecture. Throwaway prototyping like that is probably the best way to do development (imo), because it increases the likelihood that your final product has a strong foundation, and smooths out the development process dramatically. You don't carry the baggage of the learning process into the final product or the next prototype. However, this traditionally creates an enormous upfront cost, as we end up having to build out the same functionality many times, just to have it once in the end product. But with Claude, I can accomplish the same number of from-scratch iterations in 1 day as it would take me to build out myself in 2 weeks, making this a suitable approach for any project that has a limited enough scope to use Claude for prototyping. That is to say, you're not going to prototype an Unreal Engine competitor using Claude, but prototypes for a browser based FM synth toy are well within its wheelhouse.
- lawlessone 2y ago
  >Then why not remove this crap (LLMs) from the loop altogether Because reading is faster than writing. Someone could spend a few years or even most of their life writing a book that can be read in a matter of hours days or weeks. Humans writing have to proofread their own work. Or occasionally even pay someone else to do it.
abeppu 2y ago
No, bad/wrong/nonsense is not the only risk here. You're missing the main point that the authors are making: the shape of the distribution gets changed by this process. A model trained on human data will produce fewer high-perplexity examples than it was trained on (you can see this in Fig 1b, even between generation 0 and 1). In a literal information theory sense, these perplexity values indicate how much information is in each example. Over successive generations models have less actual information to learn from even if they have the same volume of text.
- visarga 2y ago
  LLMs are milking us of knowledge and skills, repackage them and give it back to us. Models interact with the internet, humans and code execution. They are exploring. Lots of exploring now happens in the chat room, a place where ideas are first tried out. With billions of users, the volume of information LLMs collect from us is huge. We bring references, guidance and feedback right into its mouth, the LLM doesn't even need to do anything like crawling. Imagine how many things we know, things we accumulated in our life experience, that were never written down anywhere. That information was lost to others. But now we use LLM assistants, so they get to be in the loop and collect tidbits of human life experience that is not written on the internet. And soon they will also work on audio/video and travel with us everywhere, seeing what we show them.
- Pinkthinker 2y ago
  I think that maybe we are too harsh in expecting LLMs to be perfect. If they are based off of human input that is incorrect then we might propagate such errors. But they will still be quicker and much more reliable than most people. Isn’t this good enough? After all, we are willing to accept flaws in people, even including the president. I suspect that the way forward will be to progressively clean the LLM input data as each error gets identified.
visarga 2y ago
> Basically still needs a human in the loop. Yes, and big LLM developers have millions of humans in the loop. That's why they provide free access, for human in the loop filtering & guidance. If I go to chatGPT and solve a coding task, maybe the first 3 ideas don't work and the 4th works. It can do RLHF setting the first 3 with negative and the fourth with positive score. They just used me to test their model and create a datapoint. Using LLM is useful both ways - for humans, we get assistance, and LLMs get feedback for their outputs. This seems like the new form of "you are the product".

I think this is it. Generated data is ok if you're curating it to make sure nothing bad, wrong or insensible comes in. Basically still needs a human in the loop.

surfingdino 2y ago

Then why not remove this crap (LLMs) from the loop altogether? How did we get from "AI will replace you" to "your new job will be an AIs janitor" in the space of about 12 months?

sieste 2y ago
there is nothing wrong with being a janitor. you could also call it "AI editor" instead of you want to insert a job title sounds more prestigious. some people find it easier and more enjoyable to edit a first draft generated by a language model based on instructions than writing that first draft themselves.
- the8thbit 2y ago
  I gotta say, Claude is a godsend for building out quick prototypes of ideas, especially when those ideas require domain specific knowledge that you know a little about but aren't specialized in. Which is most interesting programming projects. Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course.
  - visarga 2y ago
    > Sure, I could do it myself, but it would take more time, each step would have less momentum, and I'd have to think more while I do it. Which, there's a place for that too, of course. You just start faster, but end at the same time. If you really need to understand something there is no LLM shortcut. I spent hours interrogating Claude, in the same time I could have studied from a book and gotten even better grounding.
    - the8thbit 2y ago
      As I said, "there's a place for that too, of course." I don't think Claude is a good choice if you're trying to prototype a project which uses tools that you don't understand conceptually. However, if you already have a pretty good understanding of the tools, and you're good at reading code, documenting desired functionality, and writing user story requirements then its an amazing shortcut. Basically, if you are prepared to be the team lead or architect of a project then Claude can function as a junior dev who: * has a pretty good score on hackerrank * happens to have the exact right domain specific knowledge for the project you want to build * still gets disoriented by medium and large sized codebases, as many juniors are wont to do (you will need to take over as the main developer, or involve an intermediate or senior developer once the project grows to that size) As an example, the other day I wanted to prototype a project using typescript, react-konva, and tone.js. I already have a strong understanding of typescript, react, HTML canvas, and FM synthesis. What I don't have is an encyclopedic knowledge of the APIs these specific tools expose, nor do I have code sitting in front of me which effectively combines them. If I document the functionality I want well, Claude is really good at taking that documentation and building out either that prototype or the foundation for that prototype. Another thing that I find that helps is to add an intermediate step. Describe the functionality you want the prototype to achieve, and then ask Claude to write a project proposal which documents this functionality and breaks the procedure for producing that functionality into actionable steps. You can then save the artifact it generates to the project files, and have it iterate through that. You'll eventually veer off course as the functionality you want shifts, or the order and granularity of tasks diverges from the plan which was originally designed, but it acts as a way to start a project with a much stronger foundation than just saying "I want a thing that does X. Now make it do Y too. Now make it do Z as well. etc..." Another way to use Claude effectively, which I also utilized for the project I'm talking about, is to use Claude for throwaway prototyping. Rather than having Claude build out a single prototype, and then taking the reigns from there, have it build out one prototype, then scrap that one and have it build another from scratch, then scrap that and have it build a third from scratch. Each iteration you'll learn a little more about how the functionality and structure you specified actually operates, and what Claude struggles with in relation to your project. This allows the next prototype to be built out with a little more of the functionality you want, and a little bit of a cleaner architecture. Throwaway prototyping like that is probably the best way to do development (imo), because it increases the likelihood that your final product has a strong foundation, and smooths out the development process dramatically. You don't carry the baggage of the learning process into the final product or the next prototype. However, this traditionally creates an enormous upfront cost, as we end up having to build out the same functionality many times, just to have it once in the end product. But with Claude, I can accomplish the same number of from-scratch iterations in 1 day as it would take me to build out myself in 2 weeks, making this a suitable approach for any project that has a limited enough scope to use Claude for prototyping. That is to say, you're not going to prototype an Unreal Engine competitor using Claude, but prototypes for a browser based FM synth toy are well within its wheelhouse.
lawlessone 2y ago
>Then why not remove this crap (LLMs) from the loop altogether Because reading is faster than writing. Someone could spend a few years or even most of their life writing a book that can be read in a matter of hours days or weeks. Humans writing have to proofread their own work. Or occasionally even pay someone else to do it.

abeppu 2y ago

No, bad/wrong/nonsense is not the only risk here. You're missing the main point that the authors are making: the shape of the distribution gets changed by this process. A model trained on human data will produce fewer high-perplexity examples than it was trained on (you can see this in Fig 1b, even between generation 0 and 1). In a literal information theory sense, these perplexity values indicate how much information is in each example. Over successive generations models have less actual information to learn from even if they have the same volume of text.

visarga 2y ago
LLMs are milking us of knowledge and skills, repackage them and give it back to us. Models interact with the internet, humans and code execution. They are exploring. Lots of exploring now happens in the chat room, a place where ideas are first tried out. With billions of users, the volume of information LLMs collect from us is huge. We bring references, guidance and feedback right into its mouth, the LLM doesn't even need to do anything like crawling. Imagine how many things we know, things we accumulated in our life experience, that were never written down anywhere. That information was lost to others. But now we use LLM assistants, so they get to be in the loop and collect tidbits of human life experience that is not written on the internet. And soon they will also work on audio/video and travel with us everywhere, seeing what we show them.
Pinkthinker 2y ago
I think that maybe we are too harsh in expecting LLMs to be perfect. If they are based off of human input that is incorrect then we might propagate such errors. But they will still be quicker and much more reliable than most people. Isn’t this good enough? After all, we are willing to accept flaws in people, even including the president. I suspect that the way forward will be to progressively clean the LLM input data as each error gets identified.

visarga 2y ago

> Basically still needs a human in the loop. Yes, and big LLM developers have millions of humans in the loop. That's why they provide free access, for human in the loop filtering & guidance. If I go to chatGPT and solve a coding task, maybe the first 3 ideas don't work and the 4th works. It can do RLHF setting the first 3 with negative and the fourth with positive score. They just used me to test their model and create a datapoint. Using LLM is useful both ways - for humans, we get assistance, and LLMs get feedback for their outputs. This seems like the new form of "you are the product".