4 ms·

> So why quit? Three reasons. First, I’ve felt for a few years now that the startups I’m seeing don’t seem so much like progress as just shopkeeping. This isn’t

by gwern 2y ago

> So why quit? Three reasons. First, I’ve felt for a few years now that the startups I’m seeing don’t seem so much like progress as just shopkeeping. This isn’t a dig: there’s money to be made razoring a thin slice off a huge market, and there’s certainly less risk in that than there is building a market from scratch. But it’s just not that interesting. The excitement I had as a 14-year old in 1979 trying to understand everything there was to know about computers, because this was something that could radically improve people’s lives, I just don’t get anymore. And not because people’s lives, the people who have access to the tech, aren’t better, I think they are (although this, of course, has not been as unalloyed as I imagined as a 14-year old.) But because I think the tech is no longer improving lives, it’s just maintaining the current status quo. It just feels, I don’t know, done. This is remarkable to me because I would say that right now is one of the most exciting times to invest ever. We are in one of the new platforms: VC hasn't been so interesting to me since ~2013, when cryptocurrency offered a whole new greenfield, or 2008, when the iPhone changed everything for mobile, or ~1998, when the WWW began to mature enough to allow greenfield everything. (With an honorable mention to VR - still the new platform of the future...) Just think about LLMs and programming! Programming hasn't been about to change this much, this fundamentally, since the invention of networks or timesharing, perhaps. How can you look at this and write it all off as 'shopkeeping' with no 'building a market from scratch' and 'just not that interesting'? I think maybe the real reason is the next one: they're just older, and no longer have the gumption and energy to try to dive into the Shiny New Thing, and it's time to hang up their hat and call it a day, leaving it to the next generation.

pdimitar 2y ago
> This is remarkable to me because I would say that right now is one of the most exciting times to invest ever. You sidestepped their points and jumped straight to investment opportunities and iPhones and VR and LLMs. He actually said tech is not used to improve people's lives and that I have observed to be true during my 22+ years in the tech area. What he said resonates strongly with many programmers who are working on the ground: the tech is not used for absolutely anything at all that tangibly and positively changes the world, companies just want to build strong moat and keep corporate customers with binding contracts -- and there's almost zero actual technical progress, too. (Also, "older" in this case does not do what you think it does: it means they finally got a realistic view of the situation and are no longer looking through rose-tinted glasses. This is a good thing and shouldn't be a condemning factor, and your comment kind of makes it sound like you use it that way.) --- Let me demonstrate my point. Here's an investment idea: smartphone app that photographs your fridge (could use LIDAR as well, why not) and catalogues everything in it, and uses magic sauce to remind you when to stock back on items X or Y. Difficult, right? Who cares! I'd buy it, hell, I'd even subscribe for such a service. Let's go. Invest in it. Stick to it until it's perfect, no matter how long it takes and no matter how much financial losses it incurs until it's done. Will you do it?
- greesil 2y ago
  Arguably this is easier to do now with multimodal LLMs. I say this as someone who is skeptical of the hype around the utility of LLMs. But it can probably catalog your fridge. Was that worth boiling the oceans?
  - pdimitar 2y ago
    What's your point? That some actual improvements to the lives of people will accelerate climate change, and thus we should not pursue it? If so, OK, let's find another way that doesn't consume 300+ watts per GPU for thousands of GPUs. Again, let's go -- let's see some actual technical progress. But we don't have that. We're stuck. And that's what the original article says and I support its thesis.
    - greesil 2y ago
      What's your point? What's my point?
      - pdimitar 2y ago
        My point was pretty clear I thought: that what @gwern says is not true is actually true.
colejhudson 2y ago
Suspect that a core reason for Jerry's PoV is that the dynamics of the (zeitgeist-related) software industry do not currently favor small teams. That's counter to the ethos of Silicon Valley and it's guy-in-a-garage mythology. Jerry would've been 14 around 1980, plus or minus a couple of years, which would've been in the heyday of personal computing. TRS-80s, Apple IIs, the IBM PC, its clones, etc. Then, and for the next 40-odd years, software was king (read: capital-light). And anybody could write software. LLMs are much less egalitarian. Can anyone write an LLM? Yes. Can anyone train an LLM that customers will want? A few. But most will need a few Ms, maybe even some Bs. Hence the AppAmaGooSoft benefactors and Jerry's complaint about the status-quo. Software-like economic dynamics will return, but it'll take a while, and until they do, it'll be much less morally satisfying to be a VC (in software) than it has over the last 40 years.
- dangus 2y ago
  Nah, you and the parent comment to you are not quite there. It really isn’t exciting because the technological shifts have already happened. The “Industrial Revolution” has ended. There’s no remaining industry that isn’t served by plenty of software and hardware solutions. The Internet is done, it exists. Smartphones are done, computers are done, cloud computing is done, and that’s about as good as it gets. AI is just the next tech industry house of cards that followed previous farces that were meant to stir up investment. When gathering data wasn’t profitable enough, big data came along. When big data didn’t lead to enough profitable insights, AI came along. Each new technology promises the moon but they really onmy prodiundly benefit the companies making digital pickaxes. Now you have every company under sun releasing AI products where they have so little clue what problem they’re supposed to solve that they just call it “[Company Name] AI.” That’s because the problems are solved. There’s a CRUD app for everything. It’s the end. This video sums it up pretty well if you ask me: https://youtu.be/pOuBCk8XMC8 https://youtu.be/pOuBCk8XMC8
  - tsimionescu 2y ago
    As an LLM-skeptic, I still think this is too pessimistic as an outlook for the future. I do think there is a good chance that LLMs as they exist will not change anything nearly as significantly as the hype is predicting. But I think there is scope for the technology, if it continues developing, to significantly help people. The most interesting and clear AI-based product I can imagine is a true personal assistant, that can autonomously handle tasks like "book me a barber's appointment this week" (while taking into account my work days, my meetings, my other personal appointments, etc) or "I want to meet up with Joe, Susan, and Mary for a nice dinner, book us a table" (and your personal assistant would sync up with their personal assistants, or with themselves directly if they don't have one, to arrange this, and suggest some possible venues and so on). And it would help remind you of important upcoming events, help organize your shopping and so on. And of course, it would do all of this semi-interactively ("The best slot I'm seeing for your barber appointment would be on Tuesday at 3 PM, but you have that call with Andrew, can that be moved?"). All of these things seem relatively doable in principle, with existing technologies, as long as AIs improve on their reasoning skills and someone spends the time and effort to connect a single AI to several core technologies (in this case, email, calendars, web search, maybe some reservation apps, and some camera apps to help recognize various things).
bbor 2y ago
I would argue what has changed is structural, not technological. As our age continues to gild itself to further extremes everyday, I just don’t think there’s space to use small groups of optimistic hackers to bring bright new things to humanity through risk-prone startups. There’s too much on the line, too much accumulated power and inertia. LLM-powered cognitive systems are not going to be an iPhone moment that changes some social practices and brings some daily utility, but rather.. idk, a railroad moment, I guess. A fundamental interruption to our society and economy. A new era, for better or for worse. I really don’t think the Silicon Valley playbook will see much effective use in this new context. I applaud him for looking for a way to keep that dream alive in changing times! Hopefully he updates the rest of his when he finds some answers ;)
- Earw0rm 2y ago
  There's something in this. Startup hackers in the 80s through to mid aughts were, on the one hand, playing _for real_, but on stakes that weren't absurdly high. They were all-in, but while on the one hand they didn't have an enormous amount to lose, they weren't that screwed if they lost it. What I'm seeing increasingly the last 15 years or so is hyper-privileged rich kids playing at start-ups, without the same total level of commitment - but then, I can hardly blame them, they've more to lose, and you're a lot more screwed if you lose it all in 2024. The sort of people that would have started companies 30 years ago either can't afford to take the risk at all, or stay at indie/hobby business level and make no attempt to scale.
justanotherjoe 2y ago
Even if programming could be changed drastically by LLM, that still doesn't address the fact that all sectors have reached this 'software optimum' that he suggested.
programjames 2y ago
But most of the AI startups are just razoring a thin slice off a huge market. They're making robo-receptionists for the doctor's office, or integrating LLMs into your IDE. Very few are building new markets from scratch, like photonic neural networks or brain-computer interfaces.
- twoodfin 2y ago
  This is obviously survivor bias in the extreme, but Amazon started out by razoring a slice of the (relatively, at the time) huge market for paper books.
Earw0rm 2y ago
I'll give you 1998 (although it really started to pick up momentum a couple of years earlier - that was just the point on the exponential curve where it began to get noticed). And I'll give you 2008, although while it's partly about the hardware, it's also about the pervasive and universal internet, the end of online as a separate "place" and just becoming part of the fabric of day-to-day life. 2013 and crypto, though? Sure, it's made a few (or maybe not so few) people very rich. In terms of day-to-day functionality though, NFTs were empty hype, and the only vaguely useful activity I've seen it used for was probably illegal (circumventing blocks in the global financial system) even if not necessarily immoral.

by gwern 2y ago

pdimitar 2y ago

> This is remarkable to me because I would say that right now is one of the most exciting times to invest ever. You sidestepped their points and jumped straight to investment opportunities and iPhones and VR and LLMs. He actually said tech is not used to improve people's lives and that I have observed to be true during my 22+ years in the tech area. What he said resonates strongly with many programmers who are working on the ground: the tech is not used for absolutely anything at all that tangibly and positively changes the world, companies just want to build strong moat and keep corporate customers with binding contracts -- and there's almost zero actual technical progress, too. (Also, "older" in this case does not do what you think it does: it means they finally got a realistic view of the situation and are no longer looking through rose-tinted glasses. This is a good thing and shouldn't be a condemning factor, and your comment kind of makes it sound like you use it that way.) --- Let me demonstrate my point. Here's an investment idea: smartphone app that photographs your fridge (could use LIDAR as well, why not) and catalogues everything in it, and uses magic sauce to remind you when to stock back on items X or Y. Difficult, right? Who cares! I'd buy it, hell, I'd even subscribe for such a service. Let's go. Invest in it. Stick to it until it's perfect, no matter how long it takes and no matter how much financial losses it incurs until it's done. Will you do it?

greesil 2y ago
Arguably this is easier to do now with multimodal LLMs. I say this as someone who is skeptical of the hype around the utility of LLMs. But it can probably catalog your fridge. Was that worth boiling the oceans?
- pdimitar 2y ago
  What's your point? That some actual improvements to the lives of people will accelerate climate change, and thus we should not pursue it? If so, OK, let's find another way that doesn't consume 300+ watts per GPU for thousands of GPUs. Again, let's go -- let's see some actual technical progress. But we don't have that. We're stuck. And that's what the original article says and I support its thesis.
  - greesil 2y ago
    What's your point? What's my point?
    - pdimitar 2y ago
      My point was pretty clear I thought: that what @gwern says is not true is actually true.

colejhudson 2y ago

Suspect that a core reason for Jerry's PoV is that the dynamics of the (zeitgeist-related) software industry do not currently favor small teams. That's counter to the ethos of Silicon Valley and it's guy-in-a-garage mythology. Jerry would've been 14 around 1980, plus or minus a couple of years, which would've been in the heyday of personal computing. TRS-80s, Apple IIs, the IBM PC, its clones, etc. Then, and for the next 40-odd years, software was king (read: capital-light). And anybody could write software. LLMs are much less egalitarian. Can anyone write an LLM? Yes. Can anyone train an LLM that customers will want? A few. But most will need a few Ms, maybe even some Bs. Hence the AppAmaGooSoft benefactors and Jerry's complaint about the status-quo. Software-like economic dynamics will return, but it'll take a while, and until they do, it'll be much less morally satisfying to be a VC (in software) than it has over the last 40 years.

dangus 2y ago
Nah, you and the parent comment to you are not quite there. It really isn’t exciting because the technological shifts have already happened. The “Industrial Revolution” has ended. There’s no remaining industry that isn’t served by plenty of software and hardware solutions. The Internet is done, it exists. Smartphones are done, computers are done, cloud computing is done, and that’s about as good as it gets. AI is just the next tech industry house of cards that followed previous farces that were meant to stir up investment. When gathering data wasn’t profitable enough, big data came along. When big data didn’t lead to enough profitable insights, AI came along. Each new technology promises the moon but they really onmy prodiundly benefit the companies making digital pickaxes. Now you have every company under sun releasing AI products where they have so little clue what problem they’re supposed to solve that they just call it “[Company Name] AI.” That’s because the problems are solved. There’s a CRUD app for everything. It’s the end. This video sums it up pretty well if you ask me: https://youtu.be/pOuBCk8XMC8 https://youtu.be/pOuBCk8XMC8
- tsimionescu 2y ago
  As an LLM-skeptic, I still think this is too pessimistic as an outlook for the future. I do think there is a good chance that LLMs as they exist will not change anything nearly as significantly as the hype is predicting. But I think there is scope for the technology, if it continues developing, to significantly help people. The most interesting and clear AI-based product I can imagine is a true personal assistant, that can autonomously handle tasks like "book me a barber's appointment this week" (while taking into account my work days, my meetings, my other personal appointments, etc) or "I want to meet up with Joe, Susan, and Mary for a nice dinner, book us a table" (and your personal assistant would sync up with their personal assistants, or with themselves directly if they don't have one, to arrange this, and suggest some possible venues and so on). And it would help remind you of important upcoming events, help organize your shopping and so on. And of course, it would do all of this semi-interactively ("The best slot I'm seeing for your barber appointment would be on Tuesday at 3 PM, but you have that call with Andrew, can that be moved?"). All of these things seem relatively doable in principle, with existing technologies, as long as AIs improve on their reasoning skills and someone spends the time and effort to connect a single AI to several core technologies (in this case, email, calendars, web search, maybe some reservation apps, and some camera apps to help recognize various things).

bbor 2y ago

I would argue what has changed is structural, not technological. As our age continues to gild itself to further extremes everyday, I just don’t think there’s space to use small groups of optimistic hackers to bring bright new things to humanity through risk-prone startups. There’s too much on the line, too much accumulated power and inertia. LLM-powered cognitive systems are not going to be an iPhone moment that changes some social practices and brings some daily utility, but rather.. idk, a railroad moment, I guess. A fundamental interruption to our society and economy. A new era, for better or for worse. I really don’t think the Silicon Valley playbook will see much effective use in this new context. I applaud him for looking for a way to keep that dream alive in changing times! Hopefully he updates the rest of his when he finds some answers ;)

Earw0rm 2y ago
There's something in this. Startup hackers in the 80s through to mid aughts were, on the one hand, playing _for real_, but on stakes that weren't absurdly high. They were all-in, but while on the one hand they didn't have an enormous amount to lose, they weren't that screwed if they lost it. What I'm seeing increasingly the last 15 years or so is hyper-privileged rich kids playing at start-ups, without the same total level of commitment - but then, I can hardly blame them, they've more to lose, and you're a lot more screwed if you lose it all in 2024. The sort of people that would have started companies 30 years ago either can't afford to take the risk at all, or stay at indie/hobby business level and make no attempt to scale.

justanotherjoe 2y ago

Even if programming could be changed drastically by LLM, that still doesn't address the fact that all sectors have reached this 'software optimum' that he suggested.

programjames 2y ago

But most of the AI startups are just razoring a thin slice off a huge market. They're making robo-receptionists for the doctor's office, or integrating LLMs into your IDE. Very few are building new markets from scratch, like photonic neural networks or brain-computer interfaces.

twoodfin 2y ago
This is obviously survivor bias in the extreme, but Amazon started out by razoring a slice of the (relatively, at the time) huge market for paper books.

Earw0rm 2y ago

I'll give you 1998 (although it really started to pick up momentum a couple of years earlier - that was just the point on the exponential curve where it began to get noticed). And I'll give you 2008, although while it's partly about the hardware, it's also about the pervasive and universal internet, the end of online as a separate "place" and just becoming part of the fabric of day-to-day life. 2013 and crypto, though? Sure, it's made a few (or maybe not so few) people very rich. In terms of day-to-day functionality though, NFTs were empty hype, and the only vaguely useful activity I've seen it used for was probably illegal (circumventing blocks in the global financial system) even if not necessarily immoral.