3 ms·

Interesting way to look at it! At least looks like companies are investing with some expected future return that is not yet there today. Other thoughts: 1. Cu

Interesting way to look at it! At least looks like companies are investing with some expected future return that is not yet there today. Other thoughts: 1. Current revenues might be a bit higher than you calculated. E.g. I’m not sure if copilot and azure OpenAI service are fully included in those revenues, and those might be relevant figures at this scale. 2. The 10-100x growth might actually materialize. Corporates are much slower to adopt and scale a technology than people might expect. As a result, many big potential users are only at the very beginning of adopting AI. (I am assuming there are valuable applications for them to use AI for)

chasd00 2y ago
Outside of your usual customer service chat bots I haven’t seen very many real genai applications in enterprises. I led a team that put one in prod that was pretty basic RAG over a knowledge base to answer questions for about 100 specialists internal to my employer. The knowledge base had to be so refined and the system prompt so tuned that any additional content added would blow it up. There was another bot that popped up in our firm meant to answer questions about corporate policy. I’m guessing that team did something similar but over our policy docs. It vanished about 3 months of being online. It probably gave a bad answer and someone called it out to legal. I’m curious if anyone else has seen a real application in an enterprise worthy of the hype.
- hectormalot 2y ago
  We’re automatically logging and summarizing calls. I’m in a regulated industry so it’s not an optional activity and there are quite some requirements to how and what we log. Interesting thing we learned is that agents tend to log during the calls, not afterwards. We now see (qualitative feedback) that they are less busy logging and therefore have more attention for the client, and (quantitatively) we see the calls are getting shorter and people with the tool are doing more calls per day. We do many millions of calls a year, so it sums out to a good number. Similarly, we have processes with 100s analysts with very high standards to their outputs. The traditional way is to have QA teams review and provide feedback for a few rounds. We’re introducing AI for the first round(s) of feedback to shorten the cycle time reduces context switches) and have the QA teams focus on final reviews. But I get your point. RAG knowledge bases for experts are in a hard spot. After a few months of employment, the experts tend to know the general knowledge well. As a result, the RAG-bot mostly gets questions about exceptions and niches, where it doesn’t perform very well, and mistakes might be expensive.
  - chasd00 2y ago
    Thank you for replying. One thing my company (global 700k employees so a big firm) really REALLY doesn’t like is sending corporate IP like code and also competitive things like proposals into an LLM. We use Azure which legal begrudgingly allows but they’d prefer nothing. Remember appliances? I bet there’s a market for a metal box containing an LLM that an enterprise could stick in their data center. “Cloud” is pretty well adopted but something about an LLM API makes enterprises nervous.

chasd00 2y ago

Outside of your usual customer service chat bots I haven’t seen very many real genai applications in enterprises. I led a team that put one in prod that was pretty basic RAG over a knowledge base to answer questions for about 100 specialists internal to my employer. The knowledge base had to be so refined and the system prompt so tuned that any additional content added would blow it up. There was another bot that popped up in our firm meant to answer questions about corporate policy. I’m guessing that team did something similar but over our policy docs. It vanished about 3 months of being online. It probably gave a bad answer and someone called it out to legal. I’m curious if anyone else has seen a real application in an enterprise worthy of the hype.

hectormalot 2y ago
We’re automatically logging and summarizing calls. I’m in a regulated industry so it’s not an optional activity and there are quite some requirements to how and what we log. Interesting thing we learned is that agents tend to log during the calls, not afterwards. We now see (qualitative feedback) that they are less busy logging and therefore have more attention for the client, and (quantitatively) we see the calls are getting shorter and people with the tool are doing more calls per day. We do many millions of calls a year, so it sums out to a good number. Similarly, we have processes with 100s analysts with very high standards to their outputs. The traditional way is to have QA teams review and provide feedback for a few rounds. We’re introducing AI for the first round(s) of feedback to shorten the cycle time reduces context switches) and have the QA teams focus on final reviews. But I get your point. RAG knowledge bases for experts are in a hard spot. After a few months of employment, the experts tend to know the general knowledge well. As a result, the RAG-bot mostly gets questions about exceptions and niches, where it doesn’t perform very well, and mistakes might be expensive.
- chasd00 2y ago
  Thank you for replying. One thing my company (global 700k employees so a big firm) really REALLY doesn’t like is sending corporate IP like code and also competitive things like proposals into an LLM. We use Azure which legal begrudgingly allows but they’d prefer nothing. Remember appliances? I bet there’s a market for a metal box containing an LLM that an enterprise could stick in their data center. “Cloud” is pretty well adopted but something about an LLM API makes enterprises nervous.