5 ms·

I agree with you - about the sentiment around the GenAI megaphone. FWIW, Chronon does serve context within prompts to personalize LLM responses. It is also use

I agree with you - about the sentiment around the GenAI megaphone. FWIW, Chronon does serve context within prompts to personalize LLM responses. It is also used to time-travel new prompts for evaluation.

cactusplant7374 3y ago
> time-travel new prompts for evaluation What does this mean?
- nikhilsimha 3y ago
  Imagine you are building a customer support bot for a food delivery app. The user might say - I need a refund. The bot needs to know contextual information - order details, delivery tracking details etc. Now you have written a prompt template that needs to be rendered with contextual information. This rendered prompt is what the model will use to decide whether to issue a refund or not. Before you deploy this prompt to prod, you want to evaluate its performance - instances where it correctly decided to issue or decline a refund. To evaluate, you can “replay” historical refund requests. The issue is that the information in the context changes with time. You want to instead simulate the value of the context at a historical point in time - or time-travel.
  - jamesblonde 3y ago
    Time-travel evals, nice.
  - jamesblonde 3y ago
    Are you using function calling for the context info?
  - uoaei 3y ago
    In what world is it appropriate or even legal to decide on refunds via LLM? Can you give an example that's not ripe for abuse? This really doesn't sell LLMs as anything useful except insulation from the consequences of bad decisions.
    - nikhilsimha 3y ago
      "Imagine" is the operative word :-)
    - throwaway2037 3y ago
      Don't think of LLM as completely replacing the support agent here; rather augmenting. A lot of customer service is setting/finding context: customer name, account, order, item, etc. If an LLM chatbot can do all of that, then handoff to a human support agent, there is real cost savings to be had, without reducing the quality of service.
      - uoaei 3y ago
        I'd love for others to think that way. I am a very vocal (in my own bubble) advocate for human-in-the-loop ML.
    - amenhotep 3y ago
      Have you requested a refund off Amazon lately? They have an automated system where, iirc, a wizard will ask you a few questions and then process it, presumably inspecting your customer history and so on. If the system thinks your request looks genuine and it's within whatever parameters they've set, it'll accept instantly, refund you, sometimes without even asking you to send the item back. If it's less sure, it will pass the request on to a human agent to be dealt with like it would have been in the Before Times. I can see no reason why it would be illegal or inappropriate to use an LLM as part of the initial flow there. In fact I see no reason why it would be illegal for Amazon to simply flip a coin to decide whether to immediately accept your refund. (Appropriateness is another matter!) I guess you're assuming the LLM would be the only point of contact with no recourse if it rejects you? Which strikes me as very pessimistic, unless you live in a very poorly regulated country.
  - deleted 3y ago
    [deleted]

cactusplant7374 3y ago

> time-travel new prompts for evaluation What does this mean?

nikhilsimha 3y ago
Imagine you are building a customer support bot for a food delivery app. The user might say - I need a refund. The bot needs to know contextual information - order details, delivery tracking details etc. Now you have written a prompt template that needs to be rendered with contextual information. This rendered prompt is what the model will use to decide whether to issue a refund or not. Before you deploy this prompt to prod, you want to evaluate its performance - instances where it correctly decided to issue or decline a refund. To evaluate, you can “replay” historical refund requests. The issue is that the information in the context changes with time. You want to instead simulate the value of the context at a historical point in time - or time-travel.
- jamesblonde 3y ago
  Time-travel evals, nice.
- jamesblonde 3y ago
  Are you using function calling for the context info?
- uoaei 3y ago
  In what world is it appropriate or even legal to decide on refunds via LLM? Can you give an example that's not ripe for abuse? This really doesn't sell LLMs as anything useful except insulation from the consequences of bad decisions.
  - nikhilsimha 3y ago
    "Imagine" is the operative word :-)
  - throwaway2037 3y ago
    Don't think of LLM as completely replacing the support agent here; rather augmenting. A lot of customer service is setting/finding context: customer name, account, order, item, etc. If an LLM chatbot can do all of that, then handoff to a human support agent, there is real cost savings to be had, without reducing the quality of service.
    - uoaei 3y ago
      I'd love for others to think that way. I am a very vocal (in my own bubble) advocate for human-in-the-loop ML.
  - amenhotep 3y ago
    Have you requested a refund off Amazon lately? They have an automated system where, iirc, a wizard will ask you a few questions and then process it, presumably inspecting your customer history and so on. If the system thinks your request looks genuine and it's within whatever parameters they've set, it'll accept instantly, refund you, sometimes without even asking you to send the item back. If it's less sure, it will pass the request on to a human agent to be dealt with like it would have been in the Before Times. I can see no reason why it would be illegal or inappropriate to use an LLM as part of the initial flow there. In fact I see no reason why it would be illegal for Amazon to simply flip a coin to decide whether to immediately accept your refund. (Appropriateness is another matter!) I guess you're assuming the LLM would be the only point of contact with no recourse if it rejects you? Which strikes me as very pessimistic, unless you live in a very poorly regulated country.
- deleted 3y ago
  [deleted]