4 ms·

It's still not clear to me that LLMs are actually useful, but I'm perfectly willing to accept that I'm some sort of neo-luddite (how deliciously ironic a term b

It's still not clear to me that LLMs are actually useful, but I'm perfectly willing to accept that I'm some sort of neo-luddite (how deliciously ironic a term by the way). I wonder, do you think there are examples of companies that have good strategies now? And if so, what utility are they getting out of it?

vertis 2y ago
But are you actually a luddite objecting to the impact that technology (AI in this case) is going to have or just a skeptic?
ReptileMan 2y ago
LLMs are tremendous force multipliers. They don't replace humans but increase the productivity of people quite a bit
- johnnyanmac 2y ago
  Motte: "AI will make humans more efficient and help scale small businesses and creators" Bailey:" AI can replace your entire human labor (the most expensive cost of business) by scraping the content made by laborers on the internet without compensation" Unfortunately, businesses are being sold the Bailey. And the sad part is that some companies aren't even pretending to hide behind the motte. I'm sure some, many businesses won't fall for the hype, but many others will and it will take a huge toll on the workers as a result.
- bugbuddy 2y ago
  That’s not a certainty but an AI salesman’s pitch.
  - brookst 2y ago
    Lived experience for me.
- jordanb 2y ago
  I've seen that claim made but I've not seen it in reality. There are examples of where LLMs are useful for information retrieval in that they can take vague queries and produce good summaries, but they're just as likely to produce reasonable-sounding nonsense that needs to be carefully verified. I've also seen examples where LLMs can produce code that implements well-defined toy problems but when used on a real software system with millions of lines of context they do not produce code that can remain coherent with the system. From what I've seen LLMs have yet to graduate from "impressive demo" and everyone characterizing them as being a revolution are still describing some potential future state with no real evidence that we'll get there with this technology or any time soon.
  - ianbutler 2y ago
    We live in different realities then. I can't talk about details for my previous employer, but I can say there is an implementation involving LLMs saving Data Scientists a lot of time right now. I witnessed impact first hand and advised on the implementation and refinement for the group that built it.
  - photon_collider 2y ago
    I agree that LLMs are less useful for smaller scale use cases. I’ve found success using them for creating mock data, reformatting unstructured text into structured data, and a few other menial tasks that would take me a while to do on my own. I think so much of the hype is about potential larger scale applications but the models just don’t seem reliable enough yet for that.
llm_trw 2y ago
I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially. Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs. Before this I was building a matching engine for online retail which used (then) state of the art multimodal models to decide which products would match your query to the lowest priced item. The dirty secret here is that we've solved search, it just costs too much to provide it for free. I think we'll soon see an online Costco show up which can provide this service to members and filter all the bullshit dropshippers.
- fuzztester 2y ago
  >I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially. so are you saying that you are okay with LLM hallucinations causing your company to make possible huge losses, even if rarely ? I should mention that I am an LLM newbie, or rather a zerobie. all I have is a very high level sketchy overview of the field. but I am a dev with many years background in many areas, though not AI at all, except for reading a little bit now and then about it. feel free to blast me, but preferably with data-backed statements.
- quanto 2y ago
  > Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs. What an interesting comment. So, your thesis is that a few players that come out ahead will be worth substantially more than the biggest players now and they will all be using LLMs? What's driving the consolidation? High barrier to entry for making a foundation model for finance?
- RandomLensman 2y ago
  I see that as a continuation of a long arc of using machines to read things in finance (not just on the buy side, btw.), don't think it will change the competitive dynamics between the more dominant players. (There might be some more concentration in some areas, but the dynamics for that pre-date LLMs or even AI/ML in some cases.)
brookst 2y ago
I honestly don’t know. AI is so young; it’s like asking who had good internet strategies in 1994; in hindsight we know some did, but nobody really knew at the time (I would argue those who say otherwise are speaking from survivorship bias). But take a look at GitHub’s Copilot Workspaces for ideas of how transformative AI will be. I don’t have all the pieces (if I did, I’d be rich, right?), but I think AI will deliver mass customization of goods, software, and content by reducing up front costs to near zero. So if I were looking for good strategies, I’d start with companies that currently spend a lot of money on software or content creation and who are investing in automating that.
HenryBemis 2y ago
From own experience with ChatGPT, I asked it some months ago to go through an EU regulation, and create a checklist/audit programme for IT Security. Work that would have taken me a couple of weeks, it took 'it' a few seconds. I had to go through the text to make sure it's clean and doesn't contain hallucinations. Total work effort 1-2 hours max. This doesn't mean that the human is no longer required, but for certain tasks a company can save a lot of time/money. So I don't know if they dream to drop a dept of 50 people and replace it with a single machine, but there are some tasks that cannot be outsourced to machines (e.g. decisions impacting the lives cannot be made by automation) "the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject." So good luck 'convincing' the regulator that an LLM can replace mortgage-decisions makers (it is considered a significant activity in a person's life).
- RandomLensman 2y ago
  If the regulations are so extensive that you would have needed a couple of weeks to create a checklist, how did you manage to see if the list was complete and correct within a couple of hours?
  - deleted 2y ago
    [deleted]
  - milesvp 2y ago
    This was my first thought? The amount of effort I have to go through verifying what I get from gpt makes me wonder. On the other hand, there I’ve been thinking about how it might help me read datasheets. There are details that are easy to miss when skimming, that may be easy to verify through simple text searches. That said, I know someone who almost took a job writing the actual text of laws, and they were excited about it because, while they didn’t control the laws in general, comma placement allowed them to subtly change the law’s meaning. I currently would‘t trust an LLM to understand subtle comma placement.
- 2y ago
jack_pp 2y ago
I think customer support will be replaced or severely downsized within the next 5 years and that's a pretty big number of jobs.

vertis 2y ago

But are you actually a luddite objecting to the impact that technology (AI in this case) is going to have or just a skeptic?

ReptileMan 2y ago

LLMs are tremendous force multipliers. They don't replace humans but increase the productivity of people quite a bit

johnnyanmac 2y ago
Motte: "AI will make humans more efficient and help scale small businesses and creators" Bailey:" AI can replace your entire human labor (the most expensive cost of business) by scraping the content made by laborers on the internet without compensation" Unfortunately, businesses are being sold the Bailey. And the sad part is that some companies aren't even pretending to hide behind the motte. I'm sure some, many businesses won't fall for the hype, but many others will and it will take a huge toll on the workers as a result.
bugbuddy 2y ago
That’s not a certainty but an AI salesman’s pitch.
- brookst 2y ago
  Lived experience for me.
jordanb 2y ago
I've seen that claim made but I've not seen it in reality. There are examples of where LLMs are useful for information retrieval in that they can take vague queries and produce good summaries, but they're just as likely to produce reasonable-sounding nonsense that needs to be carefully verified. I've also seen examples where LLMs can produce code that implements well-defined toy problems but when used on a real software system with millions of lines of context they do not produce code that can remain coherent with the system. From what I've seen LLMs have yet to graduate from "impressive demo" and everyone characterizing them as being a revolution are still describing some potential future state with no real evidence that we'll get there with this technology or any time soon.
- ianbutler 2y ago
  We live in different realities then. I can't talk about details for my previous employer, but I can say there is an implementation involving LLMs saving Data Scientists a lot of time right now. I witnessed impact first hand and advised on the implementation and refinement for the group that built it.
- photon_collider 2y ago
  I agree that LLMs are less useful for smaller scale use cases. I’ve found success using them for creating mock data, reformatting unstructured text into structured data, and a few other menial tasks that would take me a while to do on my own. I think so much of the hype is about potential larger scale applications but the models just don’t seem reliable enough yet for that.

llm_trw 2y ago

I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially. Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs. Before this I was building a matching engine for online retail which used (then) state of the art multimodal models to decide which products would match your query to the lowest priced item. The dirty secret here is that we've solved search, it just costs too much to provide it for free. I think we'll soon see an online Costco show up which can provide this service to members and filter all the bullshit dropshippers.

fuzztester 2y ago
>I'm in finance and we're using LLMs to summarize a ton of soft data on company performance. We're making a killing and beating the market substantially. so are you saying that you are okay with LLM hallucinations causing your company to make possible huge losses, even if rarely ? I should mention that I am an LLM newbie, or rather a zerobie. all I have is a very high level sketchy overview of the field. but I am a dev with many years background in many areas, though not AI at all, except for reading a little bit now and then about it. feel free to blast me, but preferably with data-backed statements.
quanto 2y ago
> Within the next five years I don't think there will be anyone left standing who isn't worth a trillion dollars or doesn't have all their quants augmented by LLMs. What an interesting comment. So, your thesis is that a few players that come out ahead will be worth substantially more than the biggest players now and they will all be using LLMs? What's driving the consolidation? High barrier to entry for making a foundation model for finance?
RandomLensman 2y ago
I see that as a continuation of a long arc of using machines to read things in finance (not just on the buy side, btw.), don't think it will change the competitive dynamics between the more dominant players. (There might be some more concentration in some areas, but the dynamics for that pre-date LLMs or even AI/ML in some cases.)

brookst 2y ago

I honestly don’t know. AI is so young; it’s like asking who had good internet strategies in 1994; in hindsight we know some did, but nobody really knew at the time (I would argue those who say otherwise are speaking from survivorship bias). But take a look at GitHub’s Copilot Workspaces for ideas of how transformative AI will be. I don’t have all the pieces (if I did, I’d be rich, right?), but I think AI will deliver mass customization of goods, software, and content by reducing up front costs to near zero. So if I were looking for good strategies, I’d start with companies that currently spend a lot of money on software or content creation and who are investing in automating that.

HenryBemis 2y ago

From own experience with ChatGPT, I asked it some months ago to go through an EU regulation, and create a checklist/audit programme for IT Security. Work that would have taken me a couple of weeks, it took 'it' a few seconds. I had to go through the text to make sure it's clean and doesn't contain hallucinations. Total work effort 1-2 hours max. This doesn't mean that the human is no longer required, but for certain tasks a company can save a lot of time/money. So I don't know if they dream to drop a dept of 50 people and replace it with a single machine, but there are some tasks that cannot be outsourced to machines (e.g. decisions impacting the lives cannot be made by automation) "the existence of automated decision-making, including profiling, referred to in Article 22(1) and (4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject." So good luck 'convincing' the regulator that an LLM can replace mortgage-decisions makers (it is considered a significant activity in a person's life).

RandomLensman 2y ago
If the regulations are so extensive that you would have needed a couple of weeks to create a checklist, how did you manage to see if the list was complete and correct within a couple of hours?
- deleted 2y ago
  [deleted]
- milesvp 2y ago
  This was my first thought? The amount of effort I have to go through verifying what I get from gpt makes me wonder. On the other hand, there I’ve been thinking about how it might help me read datasheets. There are details that are easy to miss when skimming, that may be easy to verify through simple text searches. That said, I know someone who almost took a job writing the actual text of laws, and they were excited about it because, while they didn’t control the laws in general, comma placement allowed them to subtly change the law’s meaning. I currently would‘t trust an LLM to understand subtle comma placement.
2y ago

jack_pp 2y ago

I think customer support will be replaced or severely downsized within the next 5 years and that's a pretty big number of jobs.