4 ms·

We use LLMs in dozens of different production applications for critical business flows. They allow for a lot of dynamism in our flows that aren’t amenable to di

We use LLMs in dozens of different production applications for critical business flows. They allow for a lot of dynamism in our flows that aren’t amenable to direct quantitative reasoning or structured workflows. Double digit percents of our growth in the last year are entirely due to them. The biggest challenge is tool chain, limits on inference capacity, and developer understanding of the abilities, limits, and techniques for using LLMs effectively. I often see these messages from the community doubting the reality, but LLMs are a powerful tool in the tool chest. But I think most companies are not staffed with skilled enough engineers with a creative enough bent to really take advantage of them yet or be willing to fund basic research and from first principles toolchain creation. That’s ok. But it’s foolish to assume this is all hype like crypto was. The parallels are obvious but the foundations are different.

mvdtnz 2y ago
Yet another post claiming "dozens" of production use cases without listing a single one.
- fnordpiglet 2y ago
  I’ve listed plenty in my comment history. I don’t generally feel compelled to trot them all out all the time - I don’t need to “prove” anything and if you think I’m lying that’s your choice. Finally, many of our uses are trade secrets and a significant competitive advantage so I don’t feel the need to disclose them to the world if our competitors don’t believe in the tech. We can keep eating their lunch.
  - deleted 2y ago
    [deleted]
threeseed 2y ago
No one is saying that all of AI is hype. It clearly isn't. But the facts are that today LLMs are not suitable for use cases that need accurate results. And there is no evidence or research that suggests this is changing anytime soon. Maybe for ever. There are very strong parallels to crypto in that (a) people are starting with the technology and trying to find problems and (b) there is a cult like atmosphere where non-believers are seen as being anti-progress and anti-technology.
- fnordpiglet 2y ago
  Yeah I think a key is LLMs in business are not generally useful alone. They require classical computing techniques to really be powerful. Accurate computation is a generally well established field and you don’t need an LLM to do optimization or math or even deductive logical reasoning. That’s a waste of their power which is typically abstract semantic abductive “reasoning” and natural language processing. Overlaying this with constraints, structure, and augmenting with optimizers, solvers, etc, you get a form of computing that was impossible more than 5 years prior and is only practical in the last 9 months. On the crypto stuff yeah I get it - especially if you’re not in the weeds of its use. A lot of people formed opinions from GPT3.5, Gemini, copilot, and other crappy experiences and haven’t kept up with the state of the art. The rate of change in AI is breathtaking and I think hard to comprehend for most people. Also the recent mess of crypto and the fact grifters grift etc also hurts. But people who doubt -are- stuck in the past. That’s not necessarily their fault and it might not even apply to their career or lives in the present and the flaws are enormous as you point out. But it’s such a remarkably powerful new mode of compute that it in combination with all the other powerful modes of compute is changing everything and will continue too, especially if next generation models keep improving as they seem to be likely to.
  - jeffreygoesto 2y ago
    That text applies to basically every new technology. Point is that you can't predict it's usefulness in 20 years from that. To me it still looks like a hammer made completely from rubber. You can practice to get some good hits, but it is pretty hard to get something reliable. And a beginner will basically just bounce it around. But it is sold as rescue for beginners.
- idf00 2y ago
  I didn't see anything in the article that indicated the authors believed that those who don't see use cases for LLMs are anti-progress or anti-technology. Is that comment related to the authors of this article, or just a general grievance you have unrelated to this article?
TeMPOraL 2y ago
> We use LLMs in dozens of different production applications for critical business flows. They allow for a lot of dynamism in our flows that aren’t amenable to direct quantitative reasoning or structured workflows. Double digit percents of our growth in the last year are entirely due to them. The biggest challenge is tool chain, limits on inference capacity, and developer understanding of the abilities, limits, and techniques for using LLMs effectively. That sounds like corporate buzzword salad. It doesn't tell much as it stands, not without at least one specific example to ground all those relative statements.
- hamelsmu 2y ago
  Hi, Hamel here. I'm one of the co-authors. I'm an independent consultant and not all clients allow me to talk about their work. However, I have two that do, which I've discussed in the article. These are two production use cases that I have supported (which again, are explicitly mentioned in the article): 1. https://www.honeycomb.io/blog/introducing-query-assistant https://www.honeycomb.io/blog/introducing-query-assistant 2. https://www.youtube.com/watch?v=B_DMMlDuJB0 https://www.youtube.com/watch?v=B_DMMlDuJB0 Other co-authors have worked on significant bodies of work: Bryan Bischoff lead the creation of Magic in Hex: https://www.latent.space/p/bryan-bischof https://www.latent.space/p/bryan-bischof Jason Liu created the most popular OSS libraries for structured data called instructor https://github.com/jxnl/instructor https://github.com/jxnl/instructor, and works with some of the leading companies in the space like Limitless and Raycast (https://jxnl.co/services/#current-and-past-clients https://jxnl.co/services/#current-and-past-clients) Eugene Yan works with LLMs extensively at Amazon and uses that to inform his writing: https://eugeneyan.com/writing/ https://eugeneyan.com/writing/ (However he isn't allowed to share specifics about Amazon) I believe you might find these worth looking at.
  - mattmanser 2y ago
    You've linked to a query generator for a custom programming language and a 1 hour video about LLM tools. The cynic in me feels like the former could probably be done by chatgpt off the shelf. But those do not seem to be real world business cases. Can you expand a bit more why you think they are? We don't have hours to spend reading, and you say you've been allowed to talk about them. So can you summarise the business benefits for us, which is what people are asking for, instead of linking to huge articles?

mvdtnz 2y ago

Yet another post claiming "dozens" of production use cases without listing a single one.

fnordpiglet 2y ago
I’ve listed plenty in my comment history. I don’t generally feel compelled to trot them all out all the time - I don’t need to “prove” anything and if you think I’m lying that’s your choice. Finally, many of our uses are trade secrets and a significant competitive advantage so I don’t feel the need to disclose them to the world if our competitors don’t believe in the tech. We can keep eating their lunch.
- deleted 2y ago
  [deleted]

threeseed 2y ago

No one is saying that all of AI is hype. It clearly isn't. But the facts are that today LLMs are not suitable for use cases that need accurate results. And there is no evidence or research that suggests this is changing anytime soon. Maybe for ever. There are very strong parallels to crypto in that (a) people are starting with the technology and trying to find problems and (b) there is a cult like atmosphere where non-believers are seen as being anti-progress and anti-technology.

fnordpiglet 2y ago
Yeah I think a key is LLMs in business are not generally useful alone. They require classical computing techniques to really be powerful. Accurate computation is a generally well established field and you don’t need an LLM to do optimization or math or even deductive logical reasoning. That’s a waste of their power which is typically abstract semantic abductive “reasoning” and natural language processing. Overlaying this with constraints, structure, and augmenting with optimizers, solvers, etc, you get a form of computing that was impossible more than 5 years prior and is only practical in the last 9 months. On the crypto stuff yeah I get it - especially if you’re not in the weeds of its use. A lot of people formed opinions from GPT3.5, Gemini, copilot, and other crappy experiences and haven’t kept up with the state of the art. The rate of change in AI is breathtaking and I think hard to comprehend for most people. Also the recent mess of crypto and the fact grifters grift etc also hurts. But people who doubt -are- stuck in the past. That’s not necessarily their fault and it might not even apply to their career or lives in the present and the flaws are enormous as you point out. But it’s such a remarkably powerful new mode of compute that it in combination with all the other powerful modes of compute is changing everything and will continue too, especially if next generation models keep improving as they seem to be likely to.
- jeffreygoesto 2y ago
  That text applies to basically every new technology. Point is that you can't predict it's usefulness in 20 years from that. To me it still looks like a hammer made completely from rubber. You can practice to get some good hits, but it is pretty hard to get something reliable. And a beginner will basically just bounce it around. But it is sold as rescue for beginners.
idf00 2y ago
I didn't see anything in the article that indicated the authors believed that those who don't see use cases for LLMs are anti-progress or anti-technology. Is that comment related to the authors of this article, or just a general grievance you have unrelated to this article?

TeMPOraL 2y ago

> We use LLMs in dozens of different production applications for critical business flows. They allow for a lot of dynamism in our flows that aren’t amenable to direct quantitative reasoning or structured workflows. Double digit percents of our growth in the last year are entirely due to them. The biggest challenge is tool chain, limits on inference capacity, and developer understanding of the abilities, limits, and techniques for using LLMs effectively. That sounds like corporate buzzword salad. It doesn't tell much as it stands, not without at least one specific example to ground all those relative statements.

hamelsmu 2y ago
Hi, Hamel here. I'm one of the co-authors. I'm an independent consultant and not all clients allow me to talk about their work. However, I have two that do, which I've discussed in the article. These are two production use cases that I have supported (which again, are explicitly mentioned in the article): 1. https://www.honeycomb.io/blog/introducing-query-assistant https://www.honeycomb.io/blog/introducing-query-assistant 2. https://www.youtube.com/watch?v=B_DMMlDuJB0 https://www.youtube.com/watch?v=B_DMMlDuJB0 Other co-authors have worked on significant bodies of work: Bryan Bischoff lead the creation of Magic in Hex: https://www.latent.space/p/bryan-bischof https://www.latent.space/p/bryan-bischof Jason Liu created the most popular OSS libraries for structured data called instructor https://github.com/jxnl/instructor https://github.com/jxnl/instructor, and works with some of the leading companies in the space like Limitless and Raycast (https://jxnl.co/services/#current-and-past-clients https://jxnl.co/services/#current-and-past-clients) Eugene Yan works with LLMs extensively at Amazon and uses that to inform his writing: https://eugeneyan.com/writing/ https://eugeneyan.com/writing/ (However he isn't allowed to share specifics about Amazon) I believe you might find these worth looking at.
- mattmanser 2y ago
  You've linked to a query generator for a custom programming language and a 1 hour video about LLM tools. The cynic in me feels like the former could probably be done by chatgpt off the shelf. But those do not seem to be real world business cases. Can you expand a bit more why you think they are? We don't have hours to spend reading, and you say you've been allowed to talk about them. So can you summarise the business benefits for us, which is what people are asking for, instead of linking to huge articles?