7 ms·

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship t

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a non-tech friend and I was trying to impart to him my feeling that all knowledge work, and a lot of mundane work is getting disrupted. Even though I have many tech-friends who are "Large Language Model doubters", my friend got it this morning. Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. For myself, I am spending a very large amount of time experimenting with the OpenAI APIs, LangChain, and Llama-Index - and I am enjoying myself tremendously.

ImHereToVote 4y ago
Thanks for the future you helped create for us young people. Hopefully the next generation is more merciful to 'useless' people than the previous one.
- goldfeld 4y ago
  This is shortsighted. AI slave labor is the only route to a true overproduction and marxism, a welfare state. Or every body become slaves.
  - ImHereToVote 4y ago
    Who needs slaves? What use will they be? Why overproduce?
    - coldtea 4y ago
      The ultimate use of slaves (or servants, or employees) is about having power other other people (them) - this goes beyond production needs... The average person might not care for this, but the psychopaths who crave (and are in) power do...
    - hgsgm 4y ago
      Overproduce so that the plebs have something after the elites seize the lion's share
      - ImHereToVote 4y ago
        Who needs plebs? I mean, a couple of thousand. Maybe, But beyond that?
        grugagag 4y ago
        Who needs people? Elites or not, what use will they bring? To do what? And elites do need plebs without which they’d have no point of reference to set themselves apart
anon7725 4y ago
> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career. I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to day work and opportunities for employment. 25 years ago in university, I did roughly the same stuff as I do today. Languages are higher on the abstraction ladder, networks and hardware are faster, etc, but stuff is still roughly the same. I can see a real discontinuity in the work on the horizon.
- roflyear 4y ago
  I think it'll end at using chatgpt as a great tool. And that seems fine. I think much of the anxiety here is in the unknown. You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually. Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.
  - UncleOxidant 4y ago
    Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?
    - bcrosby95 4y ago
      Your opinion on this probably depends on what your job currently looks like. Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me. But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.
      - coldtea 4y ago
whimsicalism 4y ago
Is that what it is? I've been struggling to figure out why people have had such difficulty seeing the clear impact is going to have this
UncleOxidant 4y ago
> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. Part of me wonders if at least some of the layoffs we've seen in the last couple of months are because companies such as Google, Microsoft, Amazon, etc. (the ones that have pretty clear visibility into what's happening in the AI space) are realizing that they aren't going to need as many knowledge workers in the not-so-distant future. I think there was always this idea in the back of our minds that this was going to happen someday. But someday was always like 15 to 20 years out. Looks like someday is knocking on our door now.
- drdec 4y ago
  > Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at it. You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses. Yes, it's going to be different but I don't think we know exactly what is going to happen yet.
  - kefabean 4y ago
    Kind of like a Jevon’s paradox for software engineering!
  - UncleOxidant 4y ago
    > You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses. And in a lot of cases they're not going to need to hire a software developer to do develop it.
olalonde 4y ago
Indeed, it's fascinating to witness a sizable segment of the HN community and distinguished intellectuals like Chomsky displaying absolute denial. I've started bookmarking the comments so I can look back at them in 5 years and have a good laugh. Some from a few months back are already aging badly[0]. [0] https://news.ycombinator.com/item?id=34197033 https://news.ycombinator.com/item?id=34197033
- WC3w6pXxgGd 4y ago
  Has Chomsky been right about anything?
  - Izkata 4y ago
    My first introduction to him was when one uncontacted tribe or anothers' language refuted something he thought was fundamental to humans. In the decade(s) since I kinda think "Chomsky was wrong" is pretty much the only reason people bring him up.
    - Calavar 4y ago
      I see a lot of irony in this. When Chomsky proposed generative grammars, his theory of universal language acquisition, and so on, they were radical ideas that upturned the central cannon of linguistics. Time has been on his side - entire schools and subfields of linguistics went extinct as more evidence emerged that Chomsky was fundamentally right. Basically every computer language and data format in existence is parsed/lexed in ways inspired by his models of language. But now Chomsky is considered the stodgy old establishment, and whenever one of his theories is contradicted somewhere on the margins people shout "Aha! He was wrong the whole time!" and ignore the 99% of cases where his models are still the best ones we have.
      - hgsgm 4y ago
        What's even more bizarre is the ChatGPT is proving him right, that a neural net can build logical grammar, and he is denying it!
noonething 4y ago
are you finding langchain and llama-index similar?
zzleeper 4y ago
Hi Mark, One question out of academic curiosity: I'm exploring ways to use these tools for research projects in econ and am struggling to see a good angle. For instance, suppose I have lots of PDF reports on how firms have evolved on each quarter (10,000 reports or any other number beyond what I can read). Can LLMs be used to spit out variables based on these reports? EG: indicator variables (optimistic-vs-pesimistic), categories, etc. that I can then use as data to test economic models? I tried simple cases with ChatGPT3.5 and got the feeling it's great for outputting narrative text, but felt mediocre for when the output was more narrowly defined into categories (which was surprising).
- mark_l_watson 4y ago
  Look at the LangChain and Llama-Index (used to be called GPT-Index) projects that make smaller projects that need to use a large amount of text data do-able. There is also support for reading PDF files (and many other data sources), and pre-computing embeddings. If you spend a short while looking at example code in the documentation, find something that is similar to your requirements (e.g., semantic search, conversational chat about a set of documents, etc.), and build on that.
  - zzleeper 4y ago
    Amazing; thanks a lot!

ImHereToVote 4y ago

Thanks for the future you helped create for us young people. Hopefully the next generation is more merciful to 'useless' people than the previous one.

goldfeld 4y ago
This is shortsighted. AI slave labor is the only route to a true overproduction and marxism, a welfare state. Or every body become slaves.
- ImHereToVote 4y ago
  Who needs slaves? What use will they be? Why overproduce?
  - coldtea 4y ago
    The ultimate use of slaves (or servants, or employees) is about having power other other people (them) - this goes beyond production needs... The average person might not care for this, but the psychopaths who crave (and are in) power do...
  - hgsgm 4y ago
    Overproduce so that the plebs have something after the elites seize the lion's share
    - ImHereToVote 4y ago
      Who needs plebs? I mean, a couple of thousand. Maybe, But beyond that?
      - grugagag 4y ago
        Who needs people? Elites or not, what use will they bring? To do what? And elites do need plebs without which they’d have no point of reference to set themselves apart

anon7725 4y ago

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career. I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to day work and opportunities for employment. 25 years ago in university, I did roughly the same stuff as I do today. Languages are higher on the abstraction ladder, networks and hardware are faster, etc, but stuff is still roughly the same. I can see a real discontinuity in the work on the horizon.

roflyear 4y ago
I think it'll end at using chatgpt as a great tool. And that seems fine. I think much of the anxiety here is in the unknown. You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually. Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.
- UncleOxidant 4y ago
  Yes, this stuff is fun to work with and use as an aid to programming. But I think what's dawning on a lot of us is that it seems to be able to facilitate a large increase productivity. And given that large productivity increase will industry need as many software developers (and other knowledge workers)?
  - bcrosby95 4y ago
    Your opinion on this probably depends on what your job currently looks like. Everywhere I've worked there's been significantly more stuff we wanted to do than we could actually do. Speeding that up looks awesome to me. But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.
    - coldtea 4y ago

whimsicalism 4y ago

Is that what it is? I've been struggling to figure out why people have had such difficulty seeing the clear impact is going to have this

UncleOxidant 4y ago

> Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through. Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. Part of me wonders if at least some of the layoffs we've seen in the last couple of months are because companies such as Google, Microsoft, Amazon, etc. (the ones that have pretty clear visibility into what's happening in the AI space) are realizing that they aren't going to need as many knowledge workers in the not-so-distant future. I think there was always this idea in the back of our minds that this was going to happen someday. But someday was always like 15 to 20 years out. Looks like someday is knocking on our door now.

drdec 4y ago
> Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at it. You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses. Yes, it's going to be different but I don't think we know exactly what is going to happen yet.
- kefabean 4y ago
  Kind of like a Jevon’s paradox for software engineering!
- UncleOxidant 4y ago
  > You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses. And in a lot of cases they're not going to need to hire a software developer to do develop it.

olalonde 4y ago

Indeed, it's fascinating to witness a sizable segment of the HN community and distinguished intellectuals like Chomsky displaying absolute denial. I've started bookmarking the comments so I can look back at them in 5 years and have a good laugh. Some from a few months back are already aging badly[0]. [0] https://news.ycombinator.com/item?id=34197033 https://news.ycombinator.com/item?id=34197033

WC3w6pXxgGd 4y ago
Has Chomsky been right about anything?
- Izkata 4y ago
  My first introduction to him was when one uncontacted tribe or anothers' language refuted something he thought was fundamental to humans. In the decade(s) since I kinda think "Chomsky was wrong" is pretty much the only reason people bring him up.
  - Calavar 4y ago
    I see a lot of irony in this. When Chomsky proposed generative grammars, his theory of universal language acquisition, and so on, they were radical ideas that upturned the central cannon of linguistics. Time has been on his side - entire schools and subfields of linguistics went extinct as more evidence emerged that Chomsky was fundamentally right. Basically every computer language and data format in existence is parsed/lexed in ways inspired by his models of language. But now Chomsky is considered the stodgy old establishment, and whenever one of his theories is contradicted somewhere on the margins people shout "Aha! He was wrong the whole time!" and ignore the 99% of cases where his models are still the best ones we have.
    - hgsgm 4y ago
      What's even more bizarre is the ChatGPT is proving him right, that a neural net can build logical grammar, and he is denying it!

noonething 4y ago

are you finding langchain and llama-index similar?

zzleeper 4y ago

Hi Mark, One question out of academic curiosity: I'm exploring ways to use these tools for research projects in econ and am struggling to see a good angle. For instance, suppose I have lots of PDF reports on how firms have evolved on each quarter (10,000 reports or any other number beyond what I can read). Can LLMs be used to spit out variables based on these reports? EG: indicator variables (optimistic-vs-pesimistic), categories, etc. that I can then use as data to test economic models? I tried simple cases with ChatGPT3.5 and got the feeling it's great for outputting narrative text, but felt mediocre for when the output was more narrowly defined into categories (which was surprising).

mark_l_watson 4y ago
Look at the LangChain and Llama-Index (used to be called GPT-Index) projects that make smaller projects that need to use a large amount of text data do-able. There is also support for reading PDF files (and many other data sources), and pre-computing embeddings. If you spend a short while looking at example code in the documentation, find something that is similar to your requirements (e.g., semantic search, conversational chat about a set of documents, etc.), and build on that.
- zzleeper 4y ago
  Amazing; thanks a lot!