3 ms·

Fold clothes -> clothes are folded. Take children to school -> they safely arrive on time. Autonomous driving -> arrive at destination without crashing. Call

Fold clothes -> clothes are folded. Take children to school -> they safely arrive on time. Autonomous driving -> arrive at destination without crashing. Call centre -> customers are happy.

TeMPOraL 2y ago
Those don't look like rewards, or at least don't get processed as such for many people (myself included). Or maybe there is some art to finding happiness in simple things like having folded clothes or surviving the commute?
- IshKebab 2y ago
  In RL rewards can be anything you want. They don't have to be things that humans like.
  - TeMPOraL 2y ago
    Fair enough! I guess you can always find some well-specified, measurable goal/reward, but then that choice limits the performance of your model. It's fine when you're building a very specialized system; it gets more difficult the more general you're trying to be. For a general system meant to operate in human environment, the goal ends up approaching "things that humans like". Case in point, that's what the overall LLM goal function is - continuations that make sense to humans, in fully-general meaning of that.
deleted 2y ago
[deleted]
YeGoblynQueenne 2y ago
>> Fold clothes -> clothes are folded. >> Take children to school -> they safely arrive on time. >> Autonomous driving -> arrive at destination without crashing. >> Call centre -> customers are happy. Define a) "folded", b) "safely", c) "destination", d) "happy". Also define the reward functions for each of the four objectives above.
- IshKebab 2y ago
  Safely -> no crashes Destination -> Like, close to the destination? I don't see how that's hard. Happy -> you can use customer feedback for this Folded -> this is indeed the trickiest one, but I think well within the capabilities of modern vision models.
  - YeGoblynQueenne 2y ago
    >> Safely -> no crashes Really? What about fires? Falling off cliffs? Causing others to crash? Your "examples" are all hand-wavy and vague and no good to train an RL agent. You've also not provided a reward function.

Fold clothes -> clothes are folded. Take children to school -> they safely arrive on time. Autonomous driving -> arrive at destination without crashing. Call centre -> customers are happy.

TeMPOraL 2y ago

Those don't look like rewards, or at least don't get processed as such for many people (myself included). Or maybe there is some art to finding happiness in simple things like having folded clothes or surviving the commute?

IshKebab 2y ago
In RL rewards can be anything you want. They don't have to be things that humans like.
- TeMPOraL 2y ago
  Fair enough! I guess you can always find some well-specified, measurable goal/reward, but then that choice limits the performance of your model. It's fine when you're building a very specialized system; it gets more difficult the more general you're trying to be. For a general system meant to operate in human environment, the goal ends up approaching "things that humans like". Case in point, that's what the overall LLM goal function is - continuations that make sense to humans, in fully-general meaning of that.

deleted 2y ago

[deleted]

YeGoblynQueenne 2y ago

>> Fold clothes -> clothes are folded. >> Take children to school -> they safely arrive on time. >> Autonomous driving -> arrive at destination without crashing. >> Call centre -> customers are happy. Define a) "folded", b) "safely", c) "destination", d) "happy". Also define the reward functions for each of the four objectives above.

IshKebab 2y ago
Safely -> no crashes Destination -> Like, close to the destination? I don't see how that's hard. Happy -> you can use customer feedback for this Folded -> this is indeed the trickiest one, but I think well within the capabilities of modern vision models.
- YeGoblynQueenne 2y ago
  >> Safely -> no crashes Really? What about fires? Falling off cliffs? Causing others to crash? Your "examples" are all hand-wavy and vague and no good to train an RL agent. You've also not provided a reward function.