4 ms·

I think a lot of these libraries are made by people who don't really design systems. At its core, what chatgpt and the like really are, are ways to transform n

I think a lot of these libraries are made by people who don't really design systems. At its core, what chatgpt and the like really are, are ways to transform natural language into structured data and vice versa. All you really need is an interface that allows you to plug in these APIs to _normal_ workflow engines that lots of companies are using every day to do work, both fully automated workflow engines like airflow or whatever, and also just plugging them into the _exact same systems_ that human engineers use -- it's perfectly possible to get chatgpt to use Jira and submit PRs or whatever.

dd36 3y ago
How?
- ComodoHacker 3y ago
  Is OData still a thing?
- empath75 3y ago
  This is going to sound like a smart-ass answer, but "programming". Jira has an api, openai has an api. You need some script that takes structured json and uses it to modify a jira ticket, you need some other script that asks openai to generate that json based on some prompt and context, and that's all there is to it. I wrote a script right after I got access to the API that could open and refine jira tickets based on conversations in a chat window.
nomel 3y ago
> are ways to transform natural language into structured data I've seen quite the opposite, because at their core they are next word predictors. Telling it to structure an output biases the context into data that it saw in a structured format. From what I've seen, this always leads to lower quality reasoning/output. I get much better results if I leave the data unstructured, to maximize reasoning, then add a later step to structure that output.
- mafuku 3y ago
  You mean like an intermediate "thoughts" or "observations" output step?
  - nomel 3y ago
    My use case has always been: 1. take some input 2. run it through the llm to get some output 3. make that output useful to my non-english-speaking software by structuring it. Forcing step three to happen in step 2 means you're in a very unusual/unrealistic "head space"/context for next word completion. Leaving structuring as a later step means the human like output/reasoning can be maintained for step 2, with the inhuman like structuring being more of a reformatting than reasoning+reformatting. Leaving the "context" as small and precise as possible (which might me large for a given task) gets the best output, from what I've seen. Being a next word predictor, this makes sense to me. I suppose this is just "chain of thought". This is pretty easy to benchmark. Have it put the answer into json format, after producing the answer. Then, have it come up with an answer in json format, as one step. It's a night and day difference, at least for my tasks.
lmeyerov 3y ago
Agreed that it feels like a cowboy show. We get 2 main uses out: 1. Inverting the control structure. Traditional orchestration frameworks blackbox simple calls, like asking an LLM to analyze a patch. Chaining is about an LLM-based planning agent + ensuring all tools etc are abstracted so the planner can orchestrate them. Airflow era tools leave too many integration gaps here. 2. Plugin interfaces. Related to 1, it puts in patterns for keeping vendors easy to swap out. (In theory, practice takes more bc tuning, but at least the shape is right.) Orchestration frameworks and new LLM ones have a lot plugin ground to cover here. We had to do a lot of work around langchain to make prod ready for louie.ai, and just as we have been figuring out patterns we need, was clear so was langchain. Don't get me started on threading and eval..

dd36 3y ago

How?

ComodoHacker 3y ago
Is OData still a thing?
empath75 3y ago
This is going to sound like a smart-ass answer, but "programming". Jira has an api, openai has an api. You need some script that takes structured json and uses it to modify a jira ticket, you need some other script that asks openai to generate that json based on some prompt and context, and that's all there is to it. I wrote a script right after I got access to the API that could open and refine jira tickets based on conversations in a chat window.

nomel 3y ago

> are ways to transform natural language into structured data I've seen quite the opposite, because at their core they are next word predictors. Telling it to structure an output biases the context into data that it saw in a structured format. From what I've seen, this always leads to lower quality reasoning/output. I get much better results if I leave the data unstructured, to maximize reasoning, then add a later step to structure that output.

mafuku 3y ago
You mean like an intermediate "thoughts" or "observations" output step?
- nomel 3y ago
  My use case has always been: 1. take some input 2. run it through the llm to get some output 3. make that output useful to my non-english-speaking software by structuring it. Forcing step three to happen in step 2 means you're in a very unusual/unrealistic "head space"/context for next word completion. Leaving structuring as a later step means the human like output/reasoning can be maintained for step 2, with the inhuman like structuring being more of a reformatting than reasoning+reformatting. Leaving the "context" as small and precise as possible (which might me large for a given task) gets the best output, from what I've seen. Being a next word predictor, this makes sense to me. I suppose this is just "chain of thought". This is pretty easy to benchmark. Have it put the answer into json format, after producing the answer. Then, have it come up with an answer in json format, as one step. It's a night and day difference, at least for my tasks.

lmeyerov 3y ago

Agreed that it feels like a cowboy show. We get 2 main uses out: 1. Inverting the control structure. Traditional orchestration frameworks blackbox simple calls, like asking an LLM to analyze a patch. Chaining is about an LLM-based planning agent + ensuring all tools etc are abstracted so the planner can orchestrate them. Airflow era tools leave too many integration gaps here. 2. Plugin interfaces. Related to 1, it puts in patterns for keeping vendors easy to swap out. (In theory, practice takes more bc tuning, but at least the shape is right.) Orchestration frameworks and new LLM ones have a lot plugin ground to cover here. We had to do a lot of work around langchain to make prod ready for louie.ai, and just as we have been figuring out patterns we need, was clear so was langchain. Don't get me started on threading and eval..