5 ms·

At our company, there is a mad dash to build things using LLMs, so that we can turn to the board and say "see? we have LLMs" so that the board can turn to inves

At our company, there is a mad dash to build things using LLMs, so that we can turn to the board and say "see? we have LLMs" so that the board can turn to investors and say "see? we have LLMs" Some of the use cases are certainly of dubious benefit to the actual product.

silisili 3y ago
I'm seeing a lot of the same, and I'd say mostly dubious. The use cases I've seen that actually make a ton of sense to me are anything with rules engines with cumbersome steps or syntax. It's so much easier to allow input like 'block traffic from outside the US this weekend from the hours of x to y' or 'junk emails that come from a different smtp server than the domain name indicates' and so on.
- bombolo 3y ago
  Then you need an expert to understand if the output is correct anyway.
  - silisili 3y ago
    Well, you should definitely be able to read the output, agreed on that much. But sometimes it's annoying to create such rules either due to having to click through multiple screens, or not remembering the exact syntax offhand. Locally, something like iptables comes to mind. I've used it hundreds of times, but never regularly enough to remember how to write more complex rules without consulting the manual or Google each time.
    - skybrian 3y ago
      The way I'd do it would be to have the LLM generate a rule in some fairly-readable rules engine language, and show the rule, examples, and counter-examples for the user to confirm. This would be like writing email filters in gmail or smart playlists in iTunes. Possibly, you could ask the LLM for the examples and counter-examples that look most likely to be a bug?
  - Turskarama 3y ago
    I think realistically, you can't use AI for anything where the output absolutely must be precise. AI is great for when close enough is good enough, but even just specifying "close enough" is hard. If you need to know that the output is perfect then you need to check it, and if you need to check it then maybe it's better to just get an expert to do it in the first place.
    - harry8 3y ago
      >then maybe it's better to just get an expert to do it in the first place. Yeah but also maybe an expert gets a shipload more done using an AI? Huge productivity increase? Dunno!
      - bombolo 3y ago
        If you're an expert, it's much easier to do the thing rather than to carefully watch over the shoulder of a noob doing the thing.
      - Turskarama 3y ago
        Again, I'm talking about in situations where you need to be absolutely precise. This isn't for emails explaining simple concepts to a client (which is a good use) but rather a technical document explaining a process to another expert end user.
- dghlsakjg 3y ago
  It seems like both of the examples cited could be done very well with simple code. Is it just a case of saving dev time since you only need to review?
  - silisili 3y ago
    Essentially, yes. A lot of SaaS companies provide their own syntax, workflows, or PL, that you as an end user don't use enough to be perhaps know the exact syntax offhand. Even something like AWS and their policy documents, for example. And don't forget something like AWS/Terraform.
    - dghlsakjg 3y ago
      Gotcha. Saves a day of documentation familiarization, essentially.
Dudester230518 3y ago
> ...there is a mad dash to build things using LLMs... As long as the mad dash translates into healthy 8-hour work days - have fun!
mu53 3y ago
I used to be cynical about doing business this way, but the more I have dealt with it. It's fun for everyone except the customers. Investors get a nice presentation. Executives get to be part of the trend for marketing. Employees learn about the latest technology
noobermin 3y ago
AI isn't going to fix the fundamental problems of companies. Note no one is proposing replacing management with AI, just the rank and file.
peterth3 3y ago
Glad to hear we’re not the only ones having an emergency LangChain hackathon. So far what we’ve been able to build feels brittle. But, LLMs are fun to play with though

silisili 3y ago

I'm seeing a lot of the same, and I'd say mostly dubious. The use cases I've seen that actually make a ton of sense to me are anything with rules engines with cumbersome steps or syntax. It's so much easier to allow input like 'block traffic from outside the US this weekend from the hours of x to y' or 'junk emails that come from a different smtp server than the domain name indicates' and so on.

bombolo 3y ago
Then you need an expert to understand if the output is correct anyway.
- silisili 3y ago
  Well, you should definitely be able to read the output, agreed on that much. But sometimes it's annoying to create such rules either due to having to click through multiple screens, or not remembering the exact syntax offhand. Locally, something like iptables comes to mind. I've used it hundreds of times, but never regularly enough to remember how to write more complex rules without consulting the manual or Google each time.
  - skybrian 3y ago
    The way I'd do it would be to have the LLM generate a rule in some fairly-readable rules engine language, and show the rule, examples, and counter-examples for the user to confirm. This would be like writing email filters in gmail or smart playlists in iTunes. Possibly, you could ask the LLM for the examples and counter-examples that look most likely to be a bug?
- Turskarama 3y ago
  I think realistically, you can't use AI for anything where the output absolutely must be precise. AI is great for when close enough is good enough, but even just specifying "close enough" is hard. If you need to know that the output is perfect then you need to check it, and if you need to check it then maybe it's better to just get an expert to do it in the first place.
  - harry8 3y ago
    >then maybe it's better to just get an expert to do it in the first place. Yeah but also maybe an expert gets a shipload more done using an AI? Huge productivity increase? Dunno!
    - bombolo 3y ago
      If you're an expert, it's much easier to do the thing rather than to carefully watch over the shoulder of a noob doing the thing.
    - Turskarama 3y ago
      Again, I'm talking about in situations where you need to be absolutely precise. This isn't for emails explaining simple concepts to a client (which is a good use) but rather a technical document explaining a process to another expert end user.
dghlsakjg 3y ago
It seems like both of the examples cited could be done very well with simple code. Is it just a case of saving dev time since you only need to review?
- silisili 3y ago
  Essentially, yes. A lot of SaaS companies provide their own syntax, workflows, or PL, that you as an end user don't use enough to be perhaps know the exact syntax offhand. Even something like AWS and their policy documents, for example. And don't forget something like AWS/Terraform.
  - dghlsakjg 3y ago
    Gotcha. Saves a day of documentation familiarization, essentially.

Dudester230518 3y ago

> ...there is a mad dash to build things using LLMs... As long as the mad dash translates into healthy 8-hour work days - have fun!

mu53 3y ago

I used to be cynical about doing business this way, but the more I have dealt with it. It's fun for everyone except the customers. Investors get a nice presentation. Executives get to be part of the trend for marketing. Employees learn about the latest technology

noobermin 3y ago

AI isn't going to fix the fundamental problems of companies. Note no one is proposing replacing management with AI, just the rank and file.

peterth3 3y ago

Glad to hear we’re not the only ones having an emergency LangChain hackathon. So far what we’ve been able to build feels brittle. But, LLMs are fun to play with though