4 ms·

>It's also very much like crypto where for every one person doing something useful with it, there are 20 trying to exploit the newness and low comprehension the

>It's also very much like crypto where for every one person doing something useful with it, there are 20 trying to exploit the newness and low comprehension the general public have of the tech This is definitely not correct in terms of numbers, there are many more people using LLMs well than have ever used crypto for any real use case. Also it's worth considering that the only real use cases of crypto are illegal, from noble stuff like busting sanctions to get food to hungry children, through to bribe evasion, bribe payment, tax evasion, drug deals, hiring hitmen, and child sexual exploitation/trafficking. In general, crypto produced close to zero for global society, even when it wasn't being used as an overt and intentional scam. LLMs are producing significant value for society right now, because OpenAI gave everone API access to a very weird intern who has incredible knowledge breadth and makes dumb mistakes. Interns (or "relatively low intelligence/experience human workers who need handholding for difficult and sometimes easy problems, with an occasional flash of insight") have always been controversial as to whether they actually provide value from the perspective of the person who has to manage the intern, but from the perspective of the company/society it's unquestionable that they do provide significant value. Different people put different value on having a collaborator at all, some people do not want to handhold anyone or work with anyone who's mistakes they ever have to work around. It is nevertheless true that in aggregate for knowledge work, "worker + intern" is more economically productive than just "worker", outside of very, very specialist use cases. This just wasn't possible with GPT-2, and even GPT-3.5 is not quite at a quality where I'd really compare it to a normal intern. No other machine aside from the human brain was even close. That's the tech now, the worst it will ever be. Whatever comes next with a major leap (GPT-5, Claude 3 or Gemini if they're good, maybe Llama 3 or the next Mistral if they can get improved significantly by the OS community before the next GPT release) is going to be either a reliable version of the same intern, or the same intern with better intelligence and comprehension that still suffers from reliability issues, or a major step up where they're equivalent in productivity to a full-blown knowledge worker in some high percentage of cases. It's already important now, it's only going to get more important. As for OpenAI specifically, I think they have a very good chance of continuing to lead the pack, particularly with the this cringe-y GPTs/GPT Builder/GPT Store thing. It's pretty transparent that this is them getting data to train an AI on how to spin up agentic AIs to accomplish specific tasks, because they'll have the data on how the GPT Builder is used and the data on how useful and effective the GPTs it builds are, so they can do things like dramatically overweight the most effective and useful GPTs for training their internal "GPT Auto Builder". They'll be running a store for these things as well as effectively controlling the operating system they run in, so purchases, ratings, time using a GPT, sentiment analysis in the GPT text log to detect success, plus explicit in-GPT feedback (the thumbs up and down, feedback submission form) will all be data they can feed into their machine, to make an AI that can build good GPTs for a task and an AI that can evaluate their performance and an AI that can most effectively get good performance out of a GPT. That's going to be huge, particularly the signals that have real economic costs to users (I know they haven't announced it, but I think eventually they're going to make it so you can purchase GPTs) because that starts to pull away rose-tinted glasses and the fog of futuristic sheen and get some more unvarnished data on how much people actually value these specific things. That data means eventually you should be able to just ask ChatGPT to do something for you, and if it can't do it natively it will be trained to be able to spin up a task-specific GPT with access to the correct tools, docs etc, then have the GPT Whisperer AI use it to get the right answer with a bunch of backup data, and return you the answer with the option to see the work. This is also a pretty auditable process, which makes a lot of the legal and AI safety folks happy. I don't see another company that is similarly well-placed in terms of having the tech, talent, compute, product, and roadmap to pull this off.

somewhereoutth 3y ago
> It is nevertheless true that in aggregate for knowledge work, "worker + intern" is more economically productive than just "worker" It can be true, but I don't think it is always or necessarily often true. The overall value proposition for interns (and apprentices etc) for society as a whole is that they will then go on to become the professional knowledge workers that they have learnt from. LLMs won't become that, so the value proposition is limited and localized. Remember the adage about garage mechanics, where the price goes up depending on how much 'help' the customer wants to give?
- lucubratory 3y ago
  It is economically true in aggregate, although you're right that the worker overseeing the intern very often feels like it isn't true. The mechanic you mentioned could fall into that category, but I think it's more likely that it's just one of those jobs that's not knowledge work so what I'm saying doesn't really apply, because there are physical space limits to collaboration in realspace work that are much more readily obtrusive than equivalents in knowledge work. Basically, good knowledge worker A & bad knowledge worker B are never going to be "A + B" productive together, but they're also extremely unlikely to be <A productive, only in very specific situations that generally have to do with destructive operations by B (deleting files, using up limited tools like licenses or submissions, etc), and LLMs have been built specifically to not be able to engage in destructive operations. Contrast that with physical work, where two people can't physically occupy the same space and if they try they are likely to end up at a lower productivity than either one of them could have achieved alone because they quite literally trip over each other. About the interns as hiring component, that's the official reason for interns and they are used that way, but they are also just valuable in and of themselves as labour. Companies that aren't even hiring and won't be in six months will still take on interns, because it's free labour. Interns are the equivalent of manual labourers in knowledge work, able to do what you tell them but not yet possessing the skills to do many complex tasks alone. And I wouldn't be so sure that the "LLMs as interns" relationship we have now doesn't result in them learning. Obviously on an architecture level any individual instance can only learn in-context and that's wiped when the context is, but in aggregate they are learning from their "time on the job". The next version of GPT is going to be directly better at least partially because of all of the data that ChatGPT 4 being an effective intern brings in. It could be anywhere from just an intern at 6 months rather than an intern on the first week, through to an actual hired employee somewhere during their first year. Even if it was technologically possible I don't think OpenAI would deploy their next product if it was as good as a senior employee in some substantial number of domains, because that would be too disruptive of a leap for society. But anywhere on the intern scale that's better than GPT-4 up to approximately an employee one year in I think is feasible for their next release, and that will be in large part thanks to the data they get from their current deployment.

somewhereoutth 3y ago

> It is nevertheless true that in aggregate for knowledge work, "worker + intern" is more economically productive than just "worker" It can be true, but I don't think it is always or necessarily often true. The overall value proposition for interns (and apprentices etc) for society as a whole is that they will then go on to become the professional knowledge workers that they have learnt from. LLMs won't become that, so the value proposition is limited and localized. Remember the adage about garage mechanics, where the price goes up depending on how much 'help' the customer wants to give?

lucubratory 3y ago
It is economically true in aggregate, although you're right that the worker overseeing the intern very often feels like it isn't true. The mechanic you mentioned could fall into that category, but I think it's more likely that it's just one of those jobs that's not knowledge work so what I'm saying doesn't really apply, because there are physical space limits to collaboration in realspace work that are much more readily obtrusive than equivalents in knowledge work. Basically, good knowledge worker A & bad knowledge worker B are never going to be "A + B" productive together, but they're also extremely unlikely to be <A productive, only in very specific situations that generally have to do with destructive operations by B (deleting files, using up limited tools like licenses or submissions, etc), and LLMs have been built specifically to not be able to engage in destructive operations. Contrast that with physical work, where two people can't physically occupy the same space and if they try they are likely to end up at a lower productivity than either one of them could have achieved alone because they quite literally trip over each other. About the interns as hiring component, that's the official reason for interns and they are used that way, but they are also just valuable in and of themselves as labour. Companies that aren't even hiring and won't be in six months will still take on interns, because it's free labour. Interns are the equivalent of manual labourers in knowledge work, able to do what you tell them but not yet possessing the skills to do many complex tasks alone. And I wouldn't be so sure that the "LLMs as interns" relationship we have now doesn't result in them learning. Obviously on an architecture level any individual instance can only learn in-context and that's wiped when the context is, but in aggregate they are learning from their "time on the job". The next version of GPT is going to be directly better at least partially because of all of the data that ChatGPT 4 being an effective intern brings in. It could be anywhere from just an intern at 6 months rather than an intern on the first week, through to an actual hired employee somewhere during their first year. Even if it was technologically possible I don't think OpenAI would deploy their next product if it was as good as a senior employee in some substantial number of domains, because that would be too disruptive of a leap for society. But anywhere on the intern scale that's better than GPT-4 up to approximately an employee one year in I think is feasible for their next release, and that will be in large part thanks to the data they get from their current deployment.