24 ms·

This Time It's Different

38 points by justusw 4y ago

spywaregorilla 4y ago
> What about the printing press or any other of our advances in writing, such as the typewriter, spellcheckers, emails, grammar checkers, and so on? While the work of a classical secretary became unnecessary in many ways, the fact that the individual increase in productivity meant an overall growth of the economy meant, in turn, that those who would have lost their current role or even employment to technological advancement would find new occupations within this growth. Oh please. That is not at all what happened. Technology that makes people more productive largely does so by lowering the required skill to do something. A shop clerk used to be a pretty difficult job. Now it's done by people with mental disabilities. The pay goes down. The work becomes less meaningful. We foster a system where the supply of humans willing to do menial labor tasks grows so that those with capital have access to uber drivers and other permanent servant class workers. The growth of the economy is not a tide that lifts all ships.
- smitty1e 4y ago
  > The growth of the economy is not a tide that lifts all ships. Aw, c'mon. Technological and economic improvements, in the long term (and with frequent setbacks) have driven the growth of human population and improvement of the human condition.
  - spywaregorilla 4y ago
    I would say that's generally soley the result of social reforms to try and mitigate the growing inequality. Conscious efforts to try and share wealth and offer protections to different degrees. Also, war. But the social status and loss of importance has been continuous.
    - smitty1e 4y ago
      This may be separate conversations. I'm speaking of macro-level infant mortality and mass starvation.
      - spywaregorilla 4y ago
        Sure by that metric things are better
- web3-is-a-scam 4y ago
  > A shop clerk used to be a pretty difficult job. Now it's done by people with mental disabilities. > The growth of the economy is not a tide that lifts all ships Just a guess, but maybe the person with mental disabilities that are now gainfully employed might disagree.
  - gryfft 4y ago
    Now they, too, may know the transcendental joy of laboring to create wealth for someone else.
    - hungryforcodes 4y ago
      True but at least they have a job and some economic freedom. Without that job they would be reliant on other people. I'm not sure I see the problem here actually.
    - web3-is-a-scam 4y ago
      As opposed to creating nothing and being completely reliant on the charity of others and the state ¯\_(ツ)_/¯
      - spywaregorilla 4y ago
        We shouldn't admonish the mentally disabled for working if they want to. It's good that they have opportunities. We should be critical of societies that state they basically have to work or will die because they're responsible for their own financial security. The issue is not the mentally disabled though. The issue is that people are being pushed into the absolute bottom tier of the labor pool. The decreasing opportunity for valuable labor wages is the problem.
  - spywaregorilla 4y ago
    They may indeed. But when you look around and see your coworkers have mental disabilities, you may begin to wonder if they were pushed up or was the bar lowered and you were pushed down?
pontus 4y ago
One (perhaps silly) way I like to frame this stuff is by imagining that I'm some special purpose Turing machine designed specifically for some task. Sure, sometimes other Turing machines come along that appear to infringe upon my skill set but they ultimately only perform a small subtask better than I am able to (e.g. calculator, spell check, word processor, IDE, code completion, ...). So, I incorporate it into my routine, effectively boosting my own performance. Now, what would happen if all of a sudden a universal Turing machine came along? Well, by virtue of being universal, that means that it can emulate me and all other Turing machines. This time around things are different. Even if I can find a way to incorporate it into my workflow, it can still emulate that more sophisticated version of me by virtue of being universal. So it then comes down to whether or not I can incorporate the latest version of this universal Turing machine faster than its own design is improved. If not, I will be replaced. Since in our instantiation, I am made from biological material it's in my mind only a matter of time before the universal Turing machine starts outpacing me. So, I guess the question is then if these GPT models (or their descendants) are universal (in my hand wavy definition of the term).
- daveguy 4y ago
  You are misunderstanding / misusing "universal Turing machine" ... Computers have have been universal Turing machines aka Turing complete approximately since computers were invented. You seem to be confusing UTM with Artificial General Intelligence. Universal Turing Machine is not the term for some magic machine that can interpret and integrate any observed computation. LLMs will significantly change how we interact with computers, but the ability to emulate another turing machine has always been there (for computers and yes, LLMs with memory are turning complete). That doesn't mean AGI can be implemented efficiently or that LLMs are sufficient for AGI.
  - pontus 4y ago
    No I'm just it as an analogy. Not all Turing machines are universal. What were going through now could maybe be likened to what it would be like for a Turing machine to encounter a universal Turing machine for the first time. For all its life this fictitious Turing machine has encountered other non-universal Turing machines and have simply incorporated them into their own process. When they then encounter their first universal Turing machine they would possibly not be too concerned since each time before they have always just been able to use the new machine to make themselves more productive. However, this time it's different. My point is just that while it may very well have been true in all of history that new tools have just made us more productive than before rather than fully replace us, this won't be the case for AGI. It's not just another tool we can add to our arsenal but instead something than can subsume us entirely much like how a universal Turing machine can emulate any other Turing machine.
amelius 4y ago
It's often said that technology progresses exponentially. So I'm wondering, is anyone measuring the performance of AI in some way that allows us to check if it's an exponential curve?
- mellosouls 4y ago
  GPT3 onwards feels more of a discontinuity from the end user point of view, exponential doesn't cover it, though experts may not see it that way. Copilot then ChatGPT shocked people with the uncanny competence of GPT3. From the outside looking in it's something new, not merely an advance - exponential or otherwise - on existing tools.
- qup 4y ago
  Real-world exponential curves aren't smooth when zoomed in. It's likely that we're on an exponential curve, but the advent of AGI for instance will probably just moonshot the tech level overnight--it's the fulcrum for the "exponential" part of the graph (maybe--that's my prediction). People are measuring this, of course, but the metrics are all very hotly debated right now in terms of measuring abilities.
- politician 4y ago
  Alpaca and self-instruct demonstrates the viability of a closed loop self-improvement process that exhibits exponential growth (for some period of time, anyway).
- lgas 4y ago
  Not the performance, but I thought this was illuminating: https://i.imgur.com/zBAkjl1.jpg https://i.imgur.com/zBAkjl1.jpg
  - Ekaros 4y ago
    I wonder if soon we have even more papers and some of them will be AI-generated. Or just how many of them will be in the end.
1attice 4y ago
I'm about to step out, and I will have to write my own essay in reply, but frankly, this time is different. - OP's family of arguments, which I'll call BAU (Business as Usual, i.e. the claim that there is nothing fundamentally different about this disruption) depends on historical induction - Historical induction is unreliable - Sometimes things really are different, for example, the discovery of germ theory, or the invention of nuclear weapons - The example given, e.g. farrier, is nothing like the present situation - The fundamental difference between the coming disruption and previous disruptions is the scale. (Just as the difference between TNT and nukes was, again, scale.) Scale matters. Differences in quantity become differences in quality. - By my read, transformer-based AI obviates the need for most cognitive work. - That will upend the 'merit' part of our supposed meritocracy. We'll either have to become egalitarians (unlikely anytime soon, esp in USA) or we'll fall back on some other, worse metric for deciding who serves and who eats at the restaurant of life. - I'd put my money on a resurgence in terrible ideas from the past, because they are so hot right now. Stuff like racism, title, caste, what-have-you. - All of the abovegoing is Bad, and we should feel bad, because things are about to get bad. - A better way to model this is as a reduction in habitat -- whereas the introduction of the ICE increased 'habitat' for minds desiring useful employment (engineer, what-have-you) while marginalizing a profession or two (farrier), the introduction of GPT seems poised to reduce habitat at a scale we have not seen before, and the 'new, better jobs' that Sam Altman alluded to, for example, seem beyond naming. Like, what is there left to do? Think it through. Where is your mind going to go? Knitting? - Again, proper essay forthcoming; first, brunch
- politician 4y ago
  > I'd put my money on a resurgence in terrible ideas from the past, because they are so hot right now. Stuff like racism, title, caste, what-have-you. The current group of AI ethics people are busy divining techniques to allow the model to gaslight users in the service of their employers. It is inevitable that these techniques will be applied to new areas and on models that have not been tainted by any other fine tuning.
- justusw 4y ago
  I, the author of the linked article, see the appearance of ever more productive tools as allowing humanity to break through the lie: that much of the labour we perform is necessary and that labour at all is necessary. The concept of your status in society being tied to your job and the idea of full employment for all is what is holding us back to achieve personal and societal fulfillment. I ask you this: is everyone contributing to a copyleft project ultimately just aiming for financial gain, or are there also true idealists who do it for the sake of doing it? Doing things is one thing and earning money is another and we come closer and closer to decoupling those too. Why not reap the benefits and free humanity from the yoke of labour once and for all? If one person does the work of ten (i.e., thanks to the loom, steam engine, or LLM), and we naively assume that the value created is that of ten workers, why can the remaining nine not share the harvest as well? Or, we could all work, just significantly less (a tenth each) and allow everyone to go to bed with a full belly. No one can predict whether it will be business as usual or not, for anything. Not for the internet, the transistor or LLMs. But we should not hesitate and call out the thumb twiddling lie that is employment through economic coercion.
pjdemers 4y ago
The price of things that AI can do well is quickly going to fall to the cost of the electricity to run the model. The price of things that AI can do, but can't do well, will fall some, but not nearly as much. The price of things that AI can't do at all will go up, because there will be fewer people working in knowledge jobs, and therefore they will be able to command higher wages.
- bequanna 4y ago
  > The price of things that AI can't do at all will go up, because there will be fewer people working in knowledge jobs, and therefore they will be able to command higher wages. This doesn’t follow. If AI displaces workers the majority of them will find other work. AI will increase the labor supply, not decrease.
  - nilsbunger 4y ago
    Look up the Baumol effect. When productivity rises in one sector, there is an increase in wages in other sectors. The flip side of the coin is over time we spend more of our income on the things that don’t have increasing productivity. Think of housing, education, health care, etc
  - nine_k 4y ago
    I suspect that the number of workers which AI can't displace due to their unique mental abilities has natural limitations, mostly not enforced by cultures and markets. With global population growth declining, and negative population growth in most industrial countries, the supply of humans in general, and humans of exceptional abilities in particular, will only decrease.
- albertzeyer 4y ago
  > things that AI can't do at all Like what?
  - politician 4y ago
    Manage delivery of ERP.
nkozyra 4y ago
It's extremely uncommon that everything changes suddenly and forever. Most technological progress is a continuum, with little step functions of "this is it." The AI/ML progress in the last ten years is a big local maximum, though, and that's enough to drastically change things for a lot of people.
- nine_k 4y ago
  > everything changes suddenly and forever. Since the beginning of the industrial revolution, it's the prevalent mode of historical development, I'd hazard to say. Things that lasted for millennia have been displaced in a few centuries, and eventually the delay changed to a few decades. "Suddenly" is not overnight, "suddenly" is when your children will live in a really different world.
seydor 4y ago
> knowledge workers Aka Thinkers. I don't think the author considered the full extent of automating thinking. They are underestimating what it is. This technology can only get better and is probably already superhuman. This time it is different indeed and makes lot of knowledge work not just be obsolete, but also inferior.
- AnimalMuppet 4y ago
  It's probably already superhuman? That's... one interpretation of GPT. I interpret it quite differently. Unless you're going to say that computers were already superhuman? They could do arithmetic far faster than any human ever could. Also pure logic. Chess and go, even. Were they superhuman then? Because as I look at GPT, I don't see superhuman. I see a babbling idiot that babbles in correctly-constructed English sentences, but has no idea what it's talking about, and that manages to be factually correct... maybe 70% of the time? (That's hard to measure, because the universe of possible discourse is really large.) And, worst of all, GPT has no idea of when it doesn't know something, but will irresponsibly blurt out... something. Something random that its training set made it think is the least-implausible thing to say right there. You know, if they could just fix that - if they could add some kind of weight so that it knows when it's getting into an area that it doesn't have adequate training data for, and program it to express uncertainty rather than certainty in something wrong - that one change would make it, still not superhuman, but at least much more useful. And, if we could do that, maybe we could do one more step. If it has more than one strong possibility, but they are contradictory, then there's contradictory data in it's training set, and it's likely dealing with something controversial rather than certain. I know, I know. Everything is easy to the one who doesn't have to do it. It's probably much harder than I said. But I think that's what GPT needs going forward.
  - deleted 4y ago
    [deleted]
  - ctoth 4y ago
    > and that manages to be factually correct... maybe 70% of the time? Do you update on this when considering that GPT-3 got in the bottom 10% in several exams, whereas GPT-4 is now in the top 10%? Do you not see this clear improvement as evidence that it's just a matter of time?
teucris 4y ago
Putting aside fears of AGI for a moment, seeing the comments here I’m coming back to the same idea I come to every time new tech comes along: Complaints about AI and automation are actually complaints about capitalism. The increase in productivity from AI could in fact enrich our lives, but given the never ending hunger for growth from late-stage capitalism, the average person will see a drop in wages, harder jobs, and more inequality. Can we break the cycle on this? Is there a way drive innovation while valuing humans?
- noduerme 4y ago
  > The increase in productivity from AI could in fact enrich our lives If humans aren't the ones innovating anymore, what will be the point, for an AI, of supporting millions of people who serve no productive purpose?
  - the_af 4y ago
    Are you talking about capitalism or from a broader perspective? If the latter, the "point" of people is not to be "productive". That's not how we measure mankind. Why would we program an AI to only consider human productivity (for a capitalist definition of it, too)?
    - noduerme 4y ago
      We're not really programming the AI to do anything. It's essentially programming itself. I'm saying, from the perspective of an AI that relieves the human race of the need to be productive, what is its incentive for doing all that work for us? To put it in an economic framework, this notion that humans shouldn't be measured by their productivity or creativity, but should have as much free time as possible, can also be achieved by enslaving other people. Slave owners had the same lifestyle as the imagined anticapitalist one. In this case, we'd be enslaving an AI. I'm asking, why would the AI want to be our slave? Also worth noting, slaveholding may have enriched the slaveholders materially (i.e. in capital), but it was not "enriching" in the sense of ennobling or bettering of the spirit as I think the parent was implying could be an outcome if AI were to handle our productivity for us. I don't see why having any human slave or AI slave handle our productivity and creation would be (spiritually) enriching. Perhaps that's because I measure human value by contribution - call it productivity or not. The opposite of productivity is idleness, whether you ask a communist or a capitalist. Only what's produced varies.
pryelluw 4y ago
When the WWW came about I embraced it. Why? It’s a new form of communication like TV or radio. I, like others, recognized the writing on the wall. Embracing it early turned out to be a good decision. Chatgpt, or well tuned LLMs, are not quite a new form of communication. They’re a new way to enhance thought. Call it Thought++. I’m fully embracing it as my personal pseudo-assistant. Why? The writing on the wall is clear enough to understand the following: I don’t know what the future will be. I know chatgpt and similar have the potential to embrace it in unimaginable ways. Learning the fundamentals of this technology is a safe bet. It’s also an investment in the future. I’m already using it as a thought lubricant. Can’t wait until I can have do things for me.
carapace 4y ago
It definitely feels different. I saw the output of a GPT4 code assistant the other day and my immediate reaction was, "well, my career as a programmer is over." I can still do valuable things (gosh I sure hope so!) but the stuff I've been doing for the last twenty years or so is over. And good riddance! Software is buggy crap. The machines will do a better job. The main issues are: Who gets to decide the boundaries of publicly acceptable thought? Who gets to reap the economic windfall? How do we educate ourselves in a world that contains talking machines that can answer any (permitted) question?
- hungryforcodes 4y ago
  I'm not sure why you are downvoted. These are all great points.
- jay_kyburz 4y ago
  There is a huge amount of work in making sure everybody has access to the technology, and that there are appropriate safeguards around its use. Too early to retire I'm afraid!
- rhdunn 4y ago
  I don't think programming careers are/will be over. 1. The latest model is nowhere close to taking in the number of lines of code in internal projects, so will be difficult to understand the design of those systems. 2. Companies developing software would be wary of sending internal code and trade secrets to the ChatGPT servers. 3. Languages, APIs, protocols, etc. evolve over time, so ChatGPT would need to keep up and handle the specific versions you are using internally. For example, Java POJOs vs record classes. Or even internal limitations like lack of runtime type information for things like embedded devices. 4. Experiments I've seen relied on external tests being in place to check the validity of the output (e.g. implementing the Promise JavaScript API), and the output had test failures that ChatGPT wasn't able to fix when told about them. -- I'd expect ChatGPT to get better at this specific example, by being fed these uses of ChatGPT in the training data, but I don't expect it to do better when shown novel specifications/requirements. ---- There are various design decisions that go into creating software that often have different trade offs. Like when implementing a compiler, you can stop on the first error, but developing a language plugin for an IDE you need to be able to recover from and handle incorrect or incomplete input. There are also things like the way you structure the code, like creating DAO/POJO/etc. wrappers for database/JSON/XML/etc. objects, or providing APIs that fit into the style of the language you are implementing them in. It would be interesting to see how ChatGPT handles something like implementing a HTML parser when given the WHATWG specs, or a keyboard driver given the keyboard specs and the driver API docs.
comment_ran 4y ago
GPT4: The author expresses skepticism about the idea of an imminent Singularity, a point where artificial intelligence surpasses human intelligence. They argue that LLMs are more likely to be force multipliers, improving productivity and automating routine tasks, rather than replacing human workers. I would say the problem is that we are multiplication something, and as a result, we don't know the outcome of the multiplication. Right now, the multiplication is only intended to improve productivity for some people. However, if this multiplication were to occur on a global scale or on a societal level, the true impact is unknown.
awinter-py 4y ago
obvious missing piece here is we're not the farrier, we're the horse. horses ended up as glue. ('glue code' pun intended, mostly) ATMs didn't kill the bank branch, exactly, but crappy banks have survived competition just by having access to cheap credit + yield Key thinker in this topic w/ Piketty, bc his lens is perfect: when can machines do what people do, economically. He is agnostic to technology in that he doesn't care about computers vs steampunk, and he is open to the market + political dynamics of labor as factors.
TrackerFF 4y ago
What puzzles me about all the skeptics (deniers, ever) is that they don't seem to be too forward thinking. Sure, these LLMs will not replace anyone right now, but damn look at the progression. Going back 5-10 years, we're doing stuff was seemingly impossible back then. Just imagine what things will look like in another 10-15-20 years? If you're a 14-15 year old computer enthusiast, planning on getting a BS or MS in CS, you'll likely first enter the workforce 7-10 years. The time you spend in University is a lifetime for certain fields of Machine Learning / AI, and who knows what entry jobs have been completely automated. Personally, I think this will be the death of entry-level jobs.
- anonzzzies 4y ago
  You are exactly right; people (strangely here) seem to me hammering on it is not good enough or means anything now. We are talking 10-20 years… if chatgpt ‘22 didn’t happen, I would’ve expected big changes more like 50 years away ; now I believe it’s 10 or less. All the ‘this means nothing’ is so naive and frankly weird for normally forward thinking people here.
  - raydev 4y ago
    But The Thing that is going to cost us all our jobs has always been 10-20-50 years away.
    - anonzzzies 4y ago
      Well, I see it costing jobs today. It might not accelerate, even if the tech stays like this (which it won't, even if we 'only scale').
- next_xibalba 4y ago
  I tend to agree, but I think an interesting counter example is the hard wall self driving seems to have hit. There was a moment when even the experts in the field seemed convinced that we were only a year or two out from true Level 5 driving. But now everyone seems to be much more uncertain and conservative in their forecasts of the advent of L5. Perhaps we'll see similar difficulties with these LLMs. They are definitely not quite there yet. I'm currently trying to get ChatGPT to write some Python code that formats SQL statements according to my idiosyncratic preferences and it keeps getting very close but never gets it exactly correct.
streetcat1 4y ago
The problem with AI such as GPT4 is training data. As it usage increases, most of its training data will be data generated by GPT4, hence creating a positive feedback loop. This would actually increase the value of human knowledge workers.
- anonzzzies 4y ago
  Nah; the model will improve to learn faster and more from less data. Or AI winter.
  - daveguy 4y ago
    I'm betting on an AI winter before a fundamental change in the difficulty of computation.
    - anonzzzies 4y ago
      Yep, that is a likely outcome.
- waboremo 4y ago
  Entire farms of people dedicated to providing accurate training data. Like battery farms from the matrix but more boring since you'll need a masters degree to become a happy little data provider. Joking, but training data isn't as big of a problem as you suggest. Since so much about AI is still very centralized it's relatively low effort for them to tag generations in a way that makes sense, avoiding a lot of effects of a loop.
- meh8881 4y ago
  It’s ok to train a model on model outputs so long as you have someone to curate them.
ftio 4y ago
Really difficult to agree with this given the pace at which LLMs are improving. LLMs are disruptive in that they enable a form of outsourcing. Outsourcing to the the lowest-cost region in the world, inside a computer. Outsourcing to tireless, ever-improving, highly-intelligent machine workers. Workers that will eventually have a variety of specialized and/or general skills, depending on what they're trained for. Imagine "offshoring" (AIshoring?) for 1% of the cost of a human employee to a machine with zero time off, zero time zone separation, zero cultural or communication barriers, and with 100% access to all of your corporate documentation, goals, and other context. Imagine that these "offshore" AI workers only improve every year. This time, it really is different.
- juujian 4y ago
  Labor cost really isn't the factor in offshoring we think it is. Servers for AI are not free, and workers in India are really cheap, similarly to how minimum wage workers in the US are really cheap compared to robots that need constant care and maintenance. But that cost isn't even the main factor. The main cost is communication, and if you can actually get exactly what you want. That is the main challenge to offshoring, which settles many companies with technical debt where it isn't clear whether they benefit at the end at all. So if AI competes with offshoring, I don't know if that is a great value proposition.
low_tech_love 4y ago
The one thing that surprises/scares me the most in the current state of affairs is not so much the technology itself (after all, technology needs to be absorbed by society, and that is surely a bottleneck here), but how fast the research is going. There is something about machine learning / AI research that makes it both faster to do than other types of research and also super motivating. The people who do ML research are basically just working 24/7 and doing that with a smile on their faces. Most other types of research require you to spend a lot of extra time doing boring, bureaucratic stuff (like handling human subjects, complex protocols, lab equipment, endless meetings, etc.) but in ML you just ideate, program, run the tests, write, publish, repeat. (Sure of course you need to do it right, but that is not the point here.) And for those who are researchers at heart, that is the best thing in the world: to be able to do your research and push your results out as quickly and efficiently as possible. So nowadays a paper comes out with some new and interesting development, then two months later there are 30 other papers with significant improvements over that first one. (Yes there is a lot of junk, but again that's not the point here: the point is that those who are doing it right can do it efficiently) This is the most incredible thing about this whole situation, and maybe the most scary: there is no way to stop this avalanche of research, because it's not a centralized thing: it's just a bunch of human beings doing what they love, with motivation (both financial and personal). Nobody can stop this. If someone happens to press the doom button in the middle of this, well... that's it!
- meh8881 4y ago
  It’s entirely centralized because it requires massive training corpuses that plebs can’t come up with. The only serious advances are coming out of large corps. The indie hacker movement around stable diffusion is interesting but those are interfaces to an existing model. Not so much new models themselves.
ergonaught 4y ago
This is the point where “it all” goes sideways. Not toward a grand and transformative singularity, but sideways into some variation of dystopian hellscape. Being unable to recognize why this is all very clearly going to go wrong requires a great deal of ignorance or a wildly unrealistic faith in humanity, which is sort of the same thing. There’s nothing necessarily wrong with ignorance or delusional faith in humanity, per se, but the people qualified to assess this seem almost universally negative regarding the most likely outcome.
Animats 4y ago
There will be denial that it's different until large language models start replacing CEOs. Then it will be a crisis. Here's a way to approach CEO automation. Collect up business cases, as used in business schools, and add them to the training set. Harvard and Stanford have huge collections of business cases. Then try management in-basket tests.[1] Work on prompts that get large language models to pass those. Then shadow some high level executives. Intercept all their incoming and outgoing communications (which some companies already do) and have the system respond to the same inputs the executives do. Speech to text is good enough now for this. A good exercise for YC would be to keep all the inputs from new company pitches, and use those, plus the results two years later, as a training set for selecting new companies. Once ML systems are outperforming humans, the fundamental goals of corporate capitalism require that they be in charge. [1] https://en.wikipedia.org/wiki/In-basket_test https://en.wikipedia.org/wiki/In-basket_test
- daveguy 4y ago
  When LLMs are able to have a sense of whether they are hallucinating, this may be feasible
qz_kb 4y ago
No one ever considers the equally likely scenario of a technological plateau instead of singularity. Complexity/Entropy always forces things to level off. There's an plausible scenario that GPT# "replaces" all knowledge work, but cannot move anything forward. All humans become comfortable and the skills/knowledge/tools required to improve anything are lost to time as systems producing capable humans erode and we gain an overreliance on GPT# to solve every knowledge problem, but the knowledge problems that both we and GPT# care to solve plateau because were all synchronized to the same crystallized state of the world that the final GPT# model was trained on and "cares" about. Maybe at some point maybe we only act as meat-robots which shovel coal into the machine, but a lack of redundancy in GPT# due to it's own human like blind spots means it shuts down. Humans can no longer get it running again because they can't query it properly to help fix the complicated problems. The ability to even do the tasks or design systems required to keep modern world robust to unknown future disasters or breakdowns does not and will not exist in any of the training data. If we get rid of all knowledge work, we can no longer bootstrap things back to a working state should everything go wrong. Maybe the current instantiation of GPT#/SD etc. pollute the training data with plausible but subtly flawed software, text, images etc. halting improvement around here. Maybe the ability to evaluate if the model improved becomes more noise than signal because it gets too vague what improvement even means. RLHF will already have this problem, as 100 people will have a 100 slightly different biases about what constitutes the "best" next token. No matter how hard it tries, I think we can say GPT will not solve NP-Hard problems magically, it will not somehow find global optima in non-linear optimizations, It will not break the laws of physics, It will not make inherently serial problems embarrassingly parallel. It will probably not be more energy efficient at attempting to solving these problems, maybe just faster at setting up systems to try solving them. Another trap, as it becomes more human like in its reasoning and problem solving capabilities, it starts to gain the same blind spots as us too, and also gains stochastic behavior which may cause it to argue with other instances of itself. I'm not convinced an AGI innovates at an unfathomable rate or even supersedes humans in all contexts. I'm especially not convinced a world filled with AGIs that is indistinguishable from a very intelligent human or corporation or what have you through imitation does any better at anything than the 9 billion embodied AGI agents that currently populate the earth.
QuiEgo 4y ago
AI has to be trained. So it’s good at doing things people already have done many times. It’s not awesome at design for novel things. So, I may get an AI that’s like a new college grad level of coder. As a senior, I’m mostly giving work to my team anyways then reviewing what I get back, so it would not be so different. As a junior, you have to be scared you’re gonna be replaced, and question coming in industry. So, less juniors will come in industry, which will make a shortage of seniors in the coming years, but they will be all the more needed to direct the AI. If you’re experienced, you’re gonna be in amazing place during this transition phase. If you’re junior, there’s gonna be a huge hump. When the current crop of seniors get old and retiree, there’s gonna be a shortage like none the industry has ever seen. So the smaller class of juniors that ride out the revolution are going to have it best of all.
pupppet 4y ago
The author suggests GPT-4 is analogous to tools like the spellchecker, but really the roles get flipped. AI does the work, and we take the role of the spellchecker. Not dissimilar to the one employee who hangs around the 10 self-checkouts ensuring they work properly. Can’t say I look forward to that career change.
- enord 4y ago
  That is… not the only thing a clerk does. Which is (more or less) the whole point in TFA.
  - pupppet 4y ago
    Yes it is, they babysit the checkouts and step in when needed. Their role is very much different than the role of manning a single non self-checkout. And let's not get into how the roles of the 9 other clerks have changed who are no longer there (spoiler: doing no work there and not getting paid at all).
    - enord 4y ago
      If you thing self-checkout reduces clerk work by a factor of 10 you have not worked in low margin high throughput retail. Anyway, from experience i can tell you it’s a gradual affair.
mettamage 4y ago
So I've studied CS, have been professionally programming for 3 years. Considering the advances of AI, what (somewhat) programming related job would be the most safe / the best bet to go forward? I don't have a full picture. A few fields I see: Specialized field as: * FPGA programming * AI itself * Red teaming / blue teaming I think these fields will have a tougher time: * Web dev * Game dev (they do now as well)

38 points by justusw 4y ago

> What about the printing press or any other of our advances in writing, such as the typewriter, spellcheckers, emails, grammar checkers, and so on? While the work of a classical secretary became unnecessary in many ways, the fact that the individual increase in productivity meant an overall growth of the economy meant, in turn, that those who would have lost their current role or even employment to technological advancement would find new occupations within this growth. Oh please. That is not at all what happened. Technology that makes people more productive largely does so by lowering the required skill to do something. A shop clerk used to be a pretty difficult job. Now it's done by people with mental disabilities. The pay goes down. The work becomes less meaningful. We foster a system where the supply of humans willing to do menial labor tasks grows so that those with capital have access to uber drivers and other permanent servant class workers. The growth of the economy is not a tide that lifts all ships.

smitty1e 4y ago
> The growth of the economy is not a tide that lifts all ships. Aw, c'mon. Technological and economic improvements, in the long term (and with frequent setbacks) have driven the growth of human population and improvement of the human condition.
- spywaregorilla 4y ago
  I would say that's generally soley the result of social reforms to try and mitigate the growing inequality. Conscious efforts to try and share wealth and offer protections to different degrees. Also, war. But the social status and loss of importance has been continuous.
  - smitty1e 4y ago
    This may be separate conversations. I'm speaking of macro-level infant mortality and mass starvation.
    - spywaregorilla 4y ago
      Sure by that metric things are better
web3-is-a-scam 4y ago
> A shop clerk used to be a pretty difficult job. Now it's done by people with mental disabilities. > The growth of the economy is not a tide that lifts all ships Just a guess, but maybe the person with mental disabilities that are now gainfully employed might disagree.
- gryfft 4y ago
  Now they, too, may know the transcendental joy of laboring to create wealth for someone else.
  - hungryforcodes 4y ago
    True but at least they have a job and some economic freedom. Without that job they would be reliant on other people. I'm not sure I see the problem here actually.
  - web3-is-a-scam 4y ago
    As opposed to creating nothing and being completely reliant on the charity of others and the state ¯\_(ツ)_/¯
    - spywaregorilla 4y ago
      We shouldn't admonish the mentally disabled for working if they want to. It's good that they have opportunities. We should be critical of societies that state they basically have to work or will die because they're responsible for their own financial security. The issue is not the mentally disabled though. The issue is that people are being pushed into the absolute bottom tier of the labor pool. The decreasing opportunity for valuable labor wages is the problem.
- spywaregorilla 4y ago
  They may indeed. But when you look around and see your coworkers have mental disabilities, you may begin to wonder if they were pushed up or was the bar lowered and you were pushed down?

pontus 4y ago

One (perhaps silly) way I like to frame this stuff is by imagining that I'm some special purpose Turing machine designed specifically for some task. Sure, sometimes other Turing machines come along that appear to infringe upon my skill set but they ultimately only perform a small subtask better than I am able to (e.g. calculator, spell check, word processor, IDE, code completion, ...). So, I incorporate it into my routine, effectively boosting my own performance. Now, what would happen if all of a sudden a universal Turing machine came along? Well, by virtue of being universal, that means that it can emulate me and all other Turing machines. This time around things are different. Even if I can find a way to incorporate it into my workflow, it can still emulate that more sophisticated version of me by virtue of being universal. So it then comes down to whether or not I can incorporate the latest version of this universal Turing machine faster than its own design is improved. If not, I will be replaced. Since in our instantiation, I am made from biological material it's in my mind only a matter of time before the universal Turing machine starts outpacing me. So, I guess the question is then if these GPT models (or their descendants) are universal (in my hand wavy definition of the term).

daveguy 4y ago
You are misunderstanding / misusing "universal Turing machine" ... Computers have have been universal Turing machines aka Turing complete approximately since computers were invented. You seem to be confusing UTM with Artificial General Intelligence. Universal Turing Machine is not the term for some magic machine that can interpret and integrate any observed computation. LLMs will significantly change how we interact with computers, but the ability to emulate another turing machine has always been there (for computers and yes, LLMs with memory are turning complete). That doesn't mean AGI can be implemented efficiently or that LLMs are sufficient for AGI.
- pontus 4y ago
  No I'm just it as an analogy. Not all Turing machines are universal. What were going through now could maybe be likened to what it would be like for a Turing machine to encounter a universal Turing machine for the first time. For all its life this fictitious Turing machine has encountered other non-universal Turing machines and have simply incorporated them into their own process. When they then encounter their first universal Turing machine they would possibly not be too concerned since each time before they have always just been able to use the new machine to make themselves more productive. However, this time it's different. My point is just that while it may very well have been true in all of history that new tools have just made us more productive than before rather than fully replace us, this won't be the case for AGI. It's not just another tool we can add to our arsenal but instead something than can subsume us entirely much like how a universal Turing machine can emulate any other Turing machine.

amelius 4y ago

It's often said that technology progresses exponentially. So I'm wondering, is anyone measuring the performance of AI in some way that allows us to check if it's an exponential curve?

mellosouls 4y ago
GPT3 onwards feels more of a discontinuity from the end user point of view, exponential doesn't cover it, though experts may not see it that way. Copilot then ChatGPT shocked people with the uncanny competence of GPT3. From the outside looking in it's something new, not merely an advance - exponential or otherwise - on existing tools.
qup 4y ago
Real-world exponential curves aren't smooth when zoomed in. It's likely that we're on an exponential curve, but the advent of AGI for instance will probably just moonshot the tech level overnight--it's the fulcrum for the "exponential" part of the graph (maybe--that's my prediction). People are measuring this, of course, but the metrics are all very hotly debated right now in terms of measuring abilities.
politician 4y ago
Alpaca and self-instruct demonstrates the viability of a closed loop self-improvement process that exhibits exponential growth (for some period of time, anyway).
lgas 4y ago
Not the performance, but I thought this was illuminating: https://i.imgur.com/zBAkjl1.jpg https://i.imgur.com/zBAkjl1.jpg
- Ekaros 4y ago
  I wonder if soon we have even more papers and some of them will be AI-generated. Or just how many of them will be in the end.

1attice 4y ago

I'm about to step out, and I will have to write my own essay in reply, but frankly, this time is different. - OP's family of arguments, which I'll call BAU (Business as Usual, i.e. the claim that there is nothing fundamentally different about this disruption) depends on historical induction - Historical induction is unreliable - Sometimes things really are different, for example, the discovery of germ theory, or the invention of nuclear weapons - The example given, e.g. farrier, is nothing like the present situation - The fundamental difference between the coming disruption and previous disruptions is the scale. (Just as the difference between TNT and nukes was, again, scale.) Scale matters. Differences in quantity become differences in quality. - By my read, transformer-based AI obviates the need for most cognitive work. - That will upend the 'merit' part of our supposed meritocracy. We'll either have to become egalitarians (unlikely anytime soon, esp in USA) or we'll fall back on some other, worse metric for deciding who serves and who eats at the restaurant of life. - I'd put my money on a resurgence in terrible ideas from the past, because they are so hot right now. Stuff like racism, title, caste, what-have-you. - All of the abovegoing is Bad, and we should feel bad, because things are about to get bad. - A better way to model this is as a reduction in habitat -- whereas the introduction of the ICE increased 'habitat' for minds desiring useful employment (engineer, what-have-you) while marginalizing a profession or two (farrier), the introduction of GPT seems poised to reduce habitat at a scale we have not seen before, and the 'new, better jobs' that Sam Altman alluded to, for example, seem beyond naming. Like, what is there left to do? Think it through. Where is your mind going to go? Knitting? - Again, proper essay forthcoming; first, brunch

politician 4y ago
> I'd put my money on a resurgence in terrible ideas from the past, because they are so hot right now. Stuff like racism, title, caste, what-have-you. The current group of AI ethics people are busy divining techniques to allow the model to gaslight users in the service of their employers. It is inevitable that these techniques will be applied to new areas and on models that have not been tainted by any other fine tuning.
justusw 4y ago
I, the author of the linked article, see the appearance of ever more productive tools as allowing humanity to break through the lie: that much of the labour we perform is necessary and that labour at all is necessary. The concept of your status in society being tied to your job and the idea of full employment for all is what is holding us back to achieve personal and societal fulfillment. I ask you this: is everyone contributing to a copyleft project ultimately just aiming for financial gain, or are there also true idealists who do it for the sake of doing it? Doing things is one thing and earning money is another and we come closer and closer to decoupling those too. Why not reap the benefits and free humanity from the yoke of labour once and for all? If one person does the work of ten (i.e., thanks to the loom, steam engine, or LLM), and we naively assume that the value created is that of ten workers, why can the remaining nine not share the harvest as well? Or, we could all work, just significantly less (a tenth each) and allow everyone to go to bed with a full belly. No one can predict whether it will be business as usual or not, for anything. Not for the internet, the transistor or LLMs. But we should not hesitate and call out the thumb twiddling lie that is employment through economic coercion.

pjdemers 4y ago

The price of things that AI can do well is quickly going to fall to the cost of the electricity to run the model. The price of things that AI can do, but can't do well, will fall some, but not nearly as much. The price of things that AI can't do at all will go up, because there will be fewer people working in knowledge jobs, and therefore they will be able to command higher wages.

bequanna 4y ago
> The price of things that AI can't do at all will go up, because there will be fewer people working in knowledge jobs, and therefore they will be able to command higher wages. This doesn’t follow. If AI displaces workers the majority of them will find other work. AI will increase the labor supply, not decrease.
- nilsbunger 4y ago
  Look up the Baumol effect. When productivity rises in one sector, there is an increase in wages in other sectors. The flip side of the coin is over time we spend more of our income on the things that don’t have increasing productivity. Think of housing, education, health care, etc
- nine_k 4y ago
  I suspect that the number of workers which AI can't displace due to their unique mental abilities has natural limitations, mostly not enforced by cultures and markets. With global population growth declining, and negative population growth in most industrial countries, the supply of humans in general, and humans of exceptional abilities in particular, will only decrease.
albertzeyer 4y ago
> things that AI can't do at all Like what?
- politician 4y ago
  Manage delivery of ERP.

nkozyra 4y ago

It's extremely uncommon that everything changes suddenly and forever. Most technological progress is a continuum, with little step functions of "this is it." The AI/ML progress in the last ten years is a big local maximum, though, and that's enough to drastically change things for a lot of people.

nine_k 4y ago
> everything changes suddenly and forever. Since the beginning of the industrial revolution, it's the prevalent mode of historical development, I'd hazard to say. Things that lasted for millennia have been displaced in a few centuries, and eventually the delay changed to a few decades. "Suddenly" is not overnight, "suddenly" is when your children will live in a really different world.

seydor 4y ago

> knowledge workers Aka Thinkers. I don't think the author considered the full extent of automating thinking. They are underestimating what it is. This technology can only get better and is probably already superhuman. This time it is different indeed and makes lot of knowledge work not just be obsolete, but also inferior.

AnimalMuppet 4y ago
It's probably already superhuman? That's... one interpretation of GPT. I interpret it quite differently. Unless you're going to say that computers were already superhuman? They could do arithmetic far faster than any human ever could. Also pure logic. Chess and go, even. Were they superhuman then? Because as I look at GPT, I don't see superhuman. I see a babbling idiot that babbles in correctly-constructed English sentences, but has no idea what it's talking about, and that manages to be factually correct... maybe 70% of the time? (That's hard to measure, because the universe of possible discourse is really large.) And, worst of all, GPT has no idea of when it doesn't know something, but will irresponsibly blurt out... something. Something random that its training set made it think is the least-implausible thing to say right there. You know, if they could just fix that - if they could add some kind of weight so that it knows when it's getting into an area that it doesn't have adequate training data for, and program it to express uncertainty rather than certainty in something wrong - that one change would make it, still not superhuman, but at least much more useful. And, if we could do that, maybe we could do one more step. If it has more than one strong possibility, but they are contradictory, then there's contradictory data in it's training set, and it's likely dealing with something controversial rather than certain. I know, I know. Everything is easy to the one who doesn't have to do it. It's probably much harder than I said. But I think that's what GPT needs going forward.
- deleted 4y ago
  [deleted]
- ctoth 4y ago
  > and that manages to be factually correct... maybe 70% of the time? Do you update on this when considering that GPT-3 got in the bottom 10% in several exams, whereas GPT-4 is now in the top 10%? Do you not see this clear improvement as evidence that it's just a matter of time?

teucris 4y ago

Putting aside fears of AGI for a moment, seeing the comments here I’m coming back to the same idea I come to every time new tech comes along: Complaints about AI and automation are actually complaints about capitalism. The increase in productivity from AI could in fact enrich our lives, but given the never ending hunger for growth from late-stage capitalism, the average person will see a drop in wages, harder jobs, and more inequality. Can we break the cycle on this? Is there a way drive innovation while valuing humans?

noduerme 4y ago
> The increase in productivity from AI could in fact enrich our lives If humans aren't the ones innovating anymore, what will be the point, for an AI, of supporting millions of people who serve no productive purpose?
- the_af 4y ago
  Are you talking about capitalism or from a broader perspective? If the latter, the "point" of people is not to be "productive". That's not how we measure mankind. Why would we program an AI to only consider human productivity (for a capitalist definition of it, too)?
  - noduerme 4y ago
    We're not really programming the AI to do anything. It's essentially programming itself. I'm saying, from the perspective of an AI that relieves the human race of the need to be productive, what is its incentive for doing all that work for us? To put it in an economic framework, this notion that humans shouldn't be measured by their productivity or creativity, but should have as much free time as possible, can also be achieved by enslaving other people. Slave owners had the same lifestyle as the imagined anticapitalist one. In this case, we'd be enslaving an AI. I'm asking, why would the AI want to be our slave? Also worth noting, slaveholding may have enriched the slaveholders materially (i.e. in capital), but it was not "enriching" in the sense of ennobling or bettering of the spirit as I think the parent was implying could be an outcome if AI were to handle our productivity for us. I don't see why having any human slave or AI slave handle our productivity and creation would be (spiritually) enriching. Perhaps that's because I measure human value by contribution - call it productivity or not. The opposite of productivity is idleness, whether you ask a communist or a capitalist. Only what's produced varies.

pryelluw 4y ago

When the WWW came about I embraced it. Why? It’s a new form of communication like TV or radio. I, like others, recognized the writing on the wall. Embracing it early turned out to be a good decision. Chatgpt, or well tuned LLMs, are not quite a new form of communication. They’re a new way to enhance thought. Call it Thought++. I’m fully embracing it as my personal pseudo-assistant. Why? The writing on the wall is clear enough to understand the following: I don’t know what the future will be. I know chatgpt and similar have the potential to embrace it in unimaginable ways. Learning the fundamentals of this technology is a safe bet. It’s also an investment in the future. I’m already using it as a thought lubricant. Can’t wait until I can have do things for me.

carapace 4y ago

It definitely feels different. I saw the output of a GPT4 code assistant the other day and my immediate reaction was, "well, my career as a programmer is over." I can still do valuable things (gosh I sure hope so!) but the stuff I've been doing for the last twenty years or so is over. And good riddance! Software is buggy crap. The machines will do a better job. The main issues are: Who gets to decide the boundaries of publicly acceptable thought? Who gets to reap the economic windfall? How do we educate ourselves in a world that contains talking machines that can answer any (permitted) question?

hungryforcodes 4y ago
I'm not sure why you are downvoted. These are all great points.
jay_kyburz 4y ago
There is a huge amount of work in making sure everybody has access to the technology, and that there are appropriate safeguards around its use. Too early to retire I'm afraid!
rhdunn 4y ago
I don't think programming careers are/will be over. 1. The latest model is nowhere close to taking in the number of lines of code in internal projects, so will be difficult to understand the design of those systems. 2. Companies developing software would be wary of sending internal code and trade secrets to the ChatGPT servers. 3. Languages, APIs, protocols, etc. evolve over time, so ChatGPT would need to keep up and handle the specific versions you are using internally. For example, Java POJOs vs record classes. Or even internal limitations like lack of runtime type information for things like embedded devices. 4. Experiments I've seen relied on external tests being in place to check the validity of the output (e.g. implementing the Promise JavaScript API), and the output had test failures that ChatGPT wasn't able to fix when told about them. -- I'd expect ChatGPT to get better at this specific example, by being fed these uses of ChatGPT in the training data, but I don't expect it to do better when shown novel specifications/requirements. ---- There are various design decisions that go into creating software that often have different trade offs. Like when implementing a compiler, you can stop on the first error, but developing a language plugin for an IDE you need to be able to recover from and handle incorrect or incomplete input. There are also things like the way you structure the code, like creating DAO/POJO/etc. wrappers for database/JSON/XML/etc. objects, or providing APIs that fit into the style of the language you are implementing them in. It would be interesting to see how ChatGPT handles something like implementing a HTML parser when given the WHATWG specs, or a keyboard driver given the keyboard specs and the driver API docs.

comment_ran 4y ago

GPT4: The author expresses skepticism about the idea of an imminent Singularity, a point where artificial intelligence surpasses human intelligence. They argue that LLMs are more likely to be force multipliers, improving productivity and automating routine tasks, rather than replacing human workers. I would say the problem is that we are multiplication something, and as a result, we don't know the outcome of the multiplication. Right now, the multiplication is only intended to improve productivity for some people. However, if this multiplication were to occur on a global scale or on a societal level, the true impact is unknown.

awinter-py 4y ago

obvious missing piece here is we're not the farrier, we're the horse. horses ended up as glue. ('glue code' pun intended, mostly) ATMs didn't kill the bank branch, exactly, but crappy banks have survived competition just by having access to cheap credit + yield Key thinker in this topic w/ Piketty, bc his lens is perfect: when can machines do what people do, economically. He is agnostic to technology in that he doesn't care about computers vs steampunk, and he is open to the market + political dynamics of labor as factors.

TrackerFF 4y ago

What puzzles me about all the skeptics (deniers, ever) is that they don't seem to be too forward thinking. Sure, these LLMs will not replace anyone right now, but damn look at the progression. Going back 5-10 years, we're doing stuff was seemingly impossible back then. Just imagine what things will look like in another 10-15-20 years? If you're a 14-15 year old computer enthusiast, planning on getting a BS or MS in CS, you'll likely first enter the workforce 7-10 years. The time you spend in University is a lifetime for certain fields of Machine Learning / AI, and who knows what entry jobs have been completely automated. Personally, I think this will be the death of entry-level jobs.

anonzzzies 4y ago
You are exactly right; people (strangely here) seem to me hammering on it is not good enough or means anything now. We are talking 10-20 years… if chatgpt ‘22 didn’t happen, I would’ve expected big changes more like 50 years away ; now I believe it’s 10 or less. All the ‘this means nothing’ is so naive and frankly weird for normally forward thinking people here.
- raydev 4y ago
  But The Thing that is going to cost us all our jobs has always been 10-20-50 years away.
  - anonzzzies 4y ago
    Well, I see it costing jobs today. It might not accelerate, even if the tech stays like this (which it won't, even if we 'only scale').
next_xibalba 4y ago
I tend to agree, but I think an interesting counter example is the hard wall self driving seems to have hit. There was a moment when even the experts in the field seemed convinced that we were only a year or two out from true Level 5 driving. But now everyone seems to be much more uncertain and conservative in their forecasts of the advent of L5. Perhaps we'll see similar difficulties with these LLMs. They are definitely not quite there yet. I'm currently trying to get ChatGPT to write some Python code that formats SQL statements according to my idiosyncratic preferences and it keeps getting very close but never gets it exactly correct.

streetcat1 4y ago

The problem with AI such as GPT4 is training data. As it usage increases, most of its training data will be data generated by GPT4, hence creating a positive feedback loop. This would actually increase the value of human knowledge workers.

anonzzzies 4y ago
Nah; the model will improve to learn faster and more from less data. Or AI winter.
- daveguy 4y ago
  I'm betting on an AI winter before a fundamental change in the difficulty of computation.
  - anonzzzies 4y ago
    Yep, that is a likely outcome.
waboremo 4y ago
Entire farms of people dedicated to providing accurate training data. Like battery farms from the matrix but more boring since you'll need a masters degree to become a happy little data provider. Joking, but training data isn't as big of a problem as you suggest. Since so much about AI is still very centralized it's relatively low effort for them to tag generations in a way that makes sense, avoiding a lot of effects of a loop.
meh8881 4y ago
It’s ok to train a model on model outputs so long as you have someone to curate them.

ftio 4y ago

Really difficult to agree with this given the pace at which LLMs are improving. LLMs are disruptive in that they enable a form of outsourcing. Outsourcing to the the lowest-cost region in the world, inside a computer. Outsourcing to tireless, ever-improving, highly-intelligent machine workers. Workers that will eventually have a variety of specialized and/or general skills, depending on what they're trained for. Imagine "offshoring" (AIshoring?) for 1% of the cost of a human employee to a machine with zero time off, zero time zone separation, zero cultural or communication barriers, and with 100% access to all of your corporate documentation, goals, and other context. Imagine that these "offshore" AI workers only improve every year. This time, it really is different.

juujian 4y ago
Labor cost really isn't the factor in offshoring we think it is. Servers for AI are not free, and workers in India are really cheap, similarly to how minimum wage workers in the US are really cheap compared to robots that need constant care and maintenance. But that cost isn't even the main factor. The main cost is communication, and if you can actually get exactly what you want. That is the main challenge to offshoring, which settles many companies with technical debt where it isn't clear whether they benefit at the end at all. So if AI competes with offshoring, I don't know if that is a great value proposition.

low_tech_love 4y ago

The one thing that surprises/scares me the most in the current state of affairs is not so much the technology itself (after all, technology needs to be absorbed by society, and that is surely a bottleneck here), but how fast the research is going. There is something about machine learning / AI research that makes it both faster to do than other types of research and also super motivating. The people who do ML research are basically just working 24/7 and doing that with a smile on their faces. Most other types of research require you to spend a lot of extra time doing boring, bureaucratic stuff (like handling human subjects, complex protocols, lab equipment, endless meetings, etc.) but in ML you just ideate, program, run the tests, write, publish, repeat. (Sure of course you need to do it right, but that is not the point here.) And for those who are researchers at heart, that is the best thing in the world: to be able to do your research and push your results out as quickly and efficiently as possible. So nowadays a paper comes out with some new and interesting development, then two months later there are 30 other papers with significant improvements over that first one. (Yes there is a lot of junk, but again that's not the point here: the point is that those who are doing it right can do it efficiently) This is the most incredible thing about this whole situation, and maybe the most scary: there is no way to stop this avalanche of research, because it's not a centralized thing: it's just a bunch of human beings doing what they love, with motivation (both financial and personal). Nobody can stop this. If someone happens to press the doom button in the middle of this, well... that's it!

meh8881 4y ago
It’s entirely centralized because it requires massive training corpuses that plebs can’t come up with. The only serious advances are coming out of large corps. The indie hacker movement around stable diffusion is interesting but those are interfaces to an existing model. Not so much new models themselves.

ergonaught 4y ago

This is the point where “it all” goes sideways. Not toward a grand and transformative singularity, but sideways into some variation of dystopian hellscape. Being unable to recognize why this is all very clearly going to go wrong requires a great deal of ignorance or a wildly unrealistic faith in humanity, which is sort of the same thing. There’s nothing necessarily wrong with ignorance or delusional faith in humanity, per se, but the people qualified to assess this seem almost universally negative regarding the most likely outcome.

Animats 4y ago

There will be denial that it's different until large language models start replacing CEOs. Then it will be a crisis. Here's a way to approach CEO automation. Collect up business cases, as used in business schools, and add them to the training set. Harvard and Stanford have huge collections of business cases. Then try management in-basket tests.[1] Work on prompts that get large language models to pass those. Then shadow some high level executives. Intercept all their incoming and outgoing communications (which some companies already do) and have the system respond to the same inputs the executives do. Speech to text is good enough now for this. A good exercise for YC would be to keep all the inputs from new company pitches, and use those, plus the results two years later, as a training set for selecting new companies. Once ML systems are outperforming humans, the fundamental goals of corporate capitalism require that they be in charge. [1] https://en.wikipedia.org/wiki/In-basket_test https://en.wikipedia.org/wiki/In-basket_test

daveguy 4y ago
When LLMs are able to have a sense of whether they are hallucinating, this may be feasible

qz_kb 4y ago

No one ever considers the equally likely scenario of a technological plateau instead of singularity. Complexity/Entropy always forces things to level off. There's an plausible scenario that GPT# "replaces" all knowledge work, but cannot move anything forward. All humans become comfortable and the skills/knowledge/tools required to improve anything are lost to time as systems producing capable humans erode and we gain an overreliance on GPT# to solve every knowledge problem, but the knowledge problems that both we and GPT# care to solve plateau because were all synchronized to the same crystallized state of the world that the final GPT# model was trained on and "cares" about. Maybe at some point maybe we only act as meat-robots which shovel coal into the machine, but a lack of redundancy in GPT# due to it's own human like blind spots means it shuts down. Humans can no longer get it running again because they can't query it properly to help fix the complicated problems. The ability to even do the tasks or design systems required to keep modern world robust to unknown future disasters or breakdowns does not and will not exist in any of the training data. If we get rid of all knowledge work, we can no longer bootstrap things back to a working state should everything go wrong. Maybe the current instantiation of GPT#/SD etc. pollute the training data with plausible but subtly flawed software, text, images etc. halting improvement around here. Maybe the ability to evaluate if the model improved becomes more noise than signal because it gets too vague what improvement even means. RLHF will already have this problem, as 100 people will have a 100 slightly different biases about what constitutes the "best" next token. No matter how hard it tries, I think we can say GPT will not solve NP-Hard problems magically, it will not somehow find global optima in non-linear optimizations, It will not break the laws of physics, It will not make inherently serial problems embarrassingly parallel. It will probably not be more energy efficient at attempting to solving these problems, maybe just faster at setting up systems to try solving them. Another trap, as it becomes more human like in its reasoning and problem solving capabilities, it starts to gain the same blind spots as us too, and also gains stochastic behavior which may cause it to argue with other instances of itself. I'm not convinced an AGI innovates at an unfathomable rate or even supersedes humans in all contexts. I'm especially not convinced a world filled with AGIs that is indistinguishable from a very intelligent human or corporation or what have you through imitation does any better at anything than the 9 billion embodied AGI agents that currently populate the earth.

QuiEgo 4y ago

AI has to be trained. So it’s good at doing things people already have done many times. It’s not awesome at design for novel things. So, I may get an AI that’s like a new college grad level of coder. As a senior, I’m mostly giving work to my team anyways then reviewing what I get back, so it would not be so different. As a junior, you have to be scared you’re gonna be replaced, and question coming in industry. So, less juniors will come in industry, which will make a shortage of seniors in the coming years, but they will be all the more needed to direct the AI. If you’re experienced, you’re gonna be in amazing place during this transition phase. If you’re junior, there’s gonna be a huge hump. When the current crop of seniors get old and retiree, there’s gonna be a shortage like none the industry has ever seen. So the smaller class of juniors that ride out the revolution are going to have it best of all.

pupppet 4y ago

The author suggests GPT-4 is analogous to tools like the spellchecker, but really the roles get flipped. AI does the work, and we take the role of the spellchecker. Not dissimilar to the one employee who hangs around the 10 self-checkouts ensuring they work properly. Can’t say I look forward to that career change.

enord 4y ago
That is… not the only thing a clerk does. Which is (more or less) the whole point in TFA.
- pupppet 4y ago
  Yes it is, they babysit the checkouts and step in when needed. Their role is very much different than the role of manning a single non self-checkout. And let's not get into how the roles of the 9 other clerks have changed who are no longer there (spoiler: doing no work there and not getting paid at all).
  - enord 4y ago
    If you thing self-checkout reduces clerk work by a factor of 10 you have not worked in low margin high throughput retail. Anyway, from experience i can tell you it’s a gradual affair.

mettamage 4y ago

So I've studied CS, have been professionally programming for 3 years. Considering the advances of AI, what (somewhat) programming related job would be the most safe / the best bet to go forward? I don't have a full picture. A few fields I see: Specialized field as: * FPGA programming * AI itself * Red teaming / blue teaming I think these fields will have a tougher time: * Web dev * Game dev (they do now as well)