3 ms·

There's evidence that you can get LLMs to write chain-of-thought explanations that are consistent with the instructions in the given text, including quantitativ

by cs702 4y ago

There's evidence that you can get LLMs to write chain-of-thought explanations that are consistent with the instructions in the given text, including quantitative data, cohort performance, governance imperatives, qualitative considerations, etc. The models can even be given directions to write conditional approvals if necessary. To get a sense of what is and will be possible, take a look at the ReAct paper: https://arxiv.org/abs/2210.03629 https://arxiv.org/abs/2210.03629 and some of the LangChain tutorials that use it: https://langchain.readthedocs.io/en/latest/modules/agents/getting_started.html https://langchain.readthedocs.io/en/latest/modules/agents/ge... https://langchain.readthedocs.io/en/latest/modules/agents/implementations/react.html?highlight=zero-shot%20react https://langchain.readthedocs.io/en/latest/modules/agents/im...

jsemrau 4y ago
Thank you for sharing the python code. While I am aware of the paper, I have not seen the code. Let me run a demo with this. Yet overall, this is not only a tech problem, but a compliance / regulatory problem that includes a time differential of sometimes many years. Also, I am not saying its impossible. Mainly because I was the one pushing this type of innovation for a long time and faced headwinds from Credit Operations for many years. Quality prevailed. One comment on the SerpAPI and compliance tools = load_tools(["serpapi", "llm-math"], llm=llm) With this innocuously looking line one integrates Google Search API (through SERPAPI) into the credit approval flow. I.e., you have no control where your customer data might end up with. Second comment: SERPAI sign-up requires email+phone. Why?
- mark_l_watson 4y ago
  The SerpAPI has backends for a few search engines. And, maybe substitute an ElasticSearch or Nutch search engine? You make a good point.
- cs702 4y ago
  > Yet overall, this is not only a tech problem, but a compliance / regulatory problem that includes a time differential of sometimes many years. Hmmm... I see what you mean. You're likely right that regulators won't take kindly to this at first. Adoption could take a long while. The difference this time is that executives will be pushing for it!

by cs702 4y ago

jsemrau 4y ago

Thank you for sharing the python code. While I am aware of the paper, I have not seen the code. Let me run a demo with this. Yet overall, this is not only a tech problem, but a compliance / regulatory problem that includes a time differential of sometimes many years. Also, I am not saying its impossible. Mainly because I was the one pushing this type of innovation for a long time and faced headwinds from Credit Operations for many years. Quality prevailed. One comment on the SerpAPI and compliance tools = load_tools(["serpapi", "llm-math"], llm=llm) With this innocuously looking line one integrates Google Search API (through SERPAPI) into the credit approval flow. I.e., you have no control where your customer data might end up with. Second comment: SERPAI sign-up requires email+phone. Why?

mark_l_watson 4y ago
The SerpAPI has backends for a few search engines. And, maybe substitute an ElasticSearch or Nutch search engine? You make a good point.
cs702 4y ago
> Yet overall, this is not only a tech problem, but a compliance / regulatory problem that includes a time differential of sometimes many years. Hmmm... I see what you mean. You're likely right that regulators won't take kindly to this at first. Adoption could take a long while. The difference this time is that executives will be pushing for it!