4 ms·

I don't like the way LLM papers are written. LLMs receive inputs and produce outputs that are best represented as plaintext with some special characters. Simply

by golol 2y ago

I don't like the way LLM papers are written. LLMs receive inputs and produce outputs that are best represented as plaintext with some special characters. Simply showing a few examples of the agent's core LLM text continuation job would explain the architecture much better than figures. I can't help but feel that the authors which do this are intentionally obfuscating things.

exe34 2y ago
it's a delaying tactic, isn't it, if you're working on version 2, you don't want version 1 to be too obvious that somebody might scoop your version 2. I just wish the reviewers would clamp down on this kind of thing.
ukuina 2y ago
I suspect they do this to give the paper more weight than a mere prompt deserves. As an example: > Given a task u \in \mathcal{U} and a human-designed action set \mathcal{A}u with R \in \mathcal{A}u , at time step t , we sample a thought-action pair (h_t, a_t) \sim \pi\theta(a_t \mid \mathcal{A}u, u, c{t-1}) following the ReAct framework (Yao et al., 2023b). Here, c{t-1} = \{(h_1, a_1, o_1), \dots, (h_{t-1}, a_{t-1}, o_{t-1})\} represents the interaction history up to time t-1 . The action a_t is executed, and an observation o_t is returned from the environment, updating the context to c_t = c_{t-1} \cup \{(h_t, a_t, o_t)\} . If a_t contains a new function not present in \mathcal{A}_{t-1}^g , we update the generated action set by setting \mathcal{A}t^g = \mathcal{A}{t-1}^g \cup f(a_t) , where f(a_t) denotes the set of functions defined in action a_t . This is a roundabout way to say: "We pick an action based on what’s happened so far, do it, see the result, and update the history. If it’s something new, we add it to the list of actions we can use."
- MattPalmer1086 2y ago
  Hah, I had a paper published this year. My co-authors are academics but I am not. Honestly, I couldn't understand the first version of the paper we wrote, despite inventing the algorithm it described! There is definitely a certain language and a precise mathematical approach which is needed to pass review for academic papers. It isn't nonsense, but does obfuscate obvious meanings.
- NitpickLawyer 2y ago
  The funny thing is that we're getting closer to being able to give that paragraph to an LLM and have it spit out the simpler explanation. This is what chatgpt gave me for the prompt "can you explain this in two sentences". It's pretty close to what you wrote. > The system follows the ReAct framework to decide on a thought and action at each step based on the task, available actions, and interaction history, updating its context with the results of the action. If the action introduces new functions, the system expands its action set to include these new capabilities.
  - ukuina 2y ago
    Isn't that still too wordy? It has that inescapable "LLM-ness" to it.
lbeurerkellner 2y ago
Typically these logs are available, but hard to read or just dumped in some JSON file. However, they have been efforts like https://explorer.invariantlabs.ai/benchmarks/ https://explorer.invariantlabs.ai/benchmarks/ that try to make agents more transparent in that way (show interaction logs).
PunchTornado 2y ago
I agree. And I view it as intellectual imposture. Instead of saying something really simple which can give good results, you obfuscate it a lot to make it sound more intelligent. Reviewers shouldn't accept these kind of papers and I'm thinking that we need a Sokal moment in AI research.

by golol 2y ago

exe34 2y ago

it's a delaying tactic, isn't it, if you're working on version 2, you don't want version 1 to be too obvious that somebody might scoop your version 2. I just wish the reviewers would clamp down on this kind of thing.

ukuina 2y ago

I suspect they do this to give the paper more weight than a mere prompt deserves. As an example: > Given a task u \in \mathcal{U} and a human-designed action set \mathcal{A}u with R \in \mathcal{A}u , at time step t , we sample a thought-action pair (h_t, a_t) \sim \pi\theta(a_t \mid \mathcal{A}u, u, c{t-1}) following the ReAct framework (Yao et al., 2023b). Here, c{t-1} = \{(h_1, a_1, o_1), \dots, (h_{t-1}, a_{t-1}, o_{t-1})\} represents the interaction history up to time t-1 . The action a_t is executed, and an observation o_t is returned from the environment, updating the context to c_t = c_{t-1} \cup \{(h_t, a_t, o_t)\} . If a_t contains a new function not present in \mathcal{A}_{t-1}^g , we update the generated action set by setting \mathcal{A}t^g = \mathcal{A}{t-1}^g \cup f(a_t) , where f(a_t) denotes the set of functions defined in action a_t . This is a roundabout way to say: "We pick an action based on what’s happened so far, do it, see the result, and update the history. If it’s something new, we add it to the list of actions we can use."

MattPalmer1086 2y ago
Hah, I had a paper published this year. My co-authors are academics but I am not. Honestly, I couldn't understand the first version of the paper we wrote, despite inventing the algorithm it described! There is definitely a certain language and a precise mathematical approach which is needed to pass review for academic papers. It isn't nonsense, but does obfuscate obvious meanings.
NitpickLawyer 2y ago
The funny thing is that we're getting closer to being able to give that paragraph to an LLM and have it spit out the simpler explanation. This is what chatgpt gave me for the prompt "can you explain this in two sentences". It's pretty close to what you wrote. > The system follows the ReAct framework to decide on a thought and action at each step based on the task, available actions, and interaction history, updating its context with the results of the action. If the action introduces new functions, the system expands its action set to include these new capabilities.
- ukuina 2y ago
  Isn't that still too wordy? It has that inescapable "LLM-ness" to it.

lbeurerkellner 2y ago

Typically these logs are available, but hard to read or just dumped in some JSON file. However, they have been efforts like https://explorer.invariantlabs.ai/benchmarks/ https://explorer.invariantlabs.ai/benchmarks/ that try to make agents more transparent in that way (show interaction logs).

PunchTornado 2y ago

I agree. And I view it as intellectual imposture. Instead of saying something really simple which can give good results, you obfuscate it a lot to make it sound more intelligent. Reviewers shouldn't accept these kind of papers and I'm thinking that we need a Sokal moment in AI research.