4 ms·

That’s a really interesting way of looking at the difficulty of the problem that’s being solved. I’m curious, how do you arrive at the number 10^9?

hervature 5y ago
You have 3 joints with 3 dimensions. So 9 variables you can control. If each variable has 10 choices, and you need to make a choice for each, that is 10^9 possible combinations.
- whimsicalism 5y ago
  "3 joints" -> What are the 3 dimensions that joints move along? I naively would have thought one or two (ie. how much spin on one axis and how much spin on the other axis) just by naively looking at my elbow for a second. For an arm of fixed length, polar coordinates have 2 dof.
  - hervature 5y ago
    Yes, there are multiple types of joints. In this case, you could think of a ball joint like a shoulder or hip. They move "up-down" and "left-right" but also rotate.
phunehehe0 5y ago
I presume it's 3 dimensions for each joint, so 10^3 * 10^3 * 10^3
- liuzhy71 5y ago
  I think the problem is exaggerated. Even with three ball joints, the action space is not that large since there are constraints on the velocity of the joints. They have to move gradually. So the actual action space is a lot smaller. A lot RL problem has similar continuity constraints, cuz in real world we are dealing with time series signals. I am not a expert in the RL domain yet, so I open myself to any opinion.
throwawaysea 5y ago
The GP specified a robot with three joints in his illustrative example. Each joint has three axes (physical dimensions) along which it can move. Each movement has an arbitrary scale of movement of 0-10, since the GP discretized continuous space into 10 buckets arbitrarily for sake of illustration. So 10 choices for each of 3 joints for each of 3 dimensions. That comes out to 10 choices for Joint 1 Dimension 1 x 10 choices for Joint 1 Dimension 2 x … 10 choices for Joint 3 Dimension 3 which comes out to 10^9.

hervature 5y ago

You have 3 joints with 3 dimensions. So 9 variables you can control. If each variable has 10 choices, and you need to make a choice for each, that is 10^9 possible combinations.

whimsicalism 5y ago
"3 joints" -> What are the 3 dimensions that joints move along? I naively would have thought one or two (ie. how much spin on one axis and how much spin on the other axis) just by naively looking at my elbow for a second. For an arm of fixed length, polar coordinates have 2 dof.
- hervature 5y ago
  Yes, there are multiple types of joints. In this case, you could think of a ball joint like a shoulder or hip. They move "up-down" and "left-right" but also rotate.

phunehehe0 5y ago

I presume it's 3 dimensions for each joint, so 10^3 * 10^3 * 10^3

liuzhy71 5y ago
I think the problem is exaggerated. Even with three ball joints, the action space is not that large since there are constraints on the velocity of the joints. They have to move gradually. So the actual action space is a lot smaller. A lot RL problem has similar continuity constraints, cuz in real world we are dealing with time series signals. I am not a expert in the RL domain yet, so I open myself to any opinion.

throwawaysea 5y ago

The GP specified a robot with three joints in his illustrative example. Each joint has three axes (physical dimensions) along which it can move. Each movement has an arbitrary scale of movement of 0-10, since the GP discretized continuous space into 10 buckets arbitrarily for sake of illustration. So 10 choices for each of 3 joints for each of 3 dimensions. That comes out to 10 choices for Joint 1 Dimension 1 x 10 choices for Joint 1 Dimension 2 x … 10 choices for Joint 3 Dimension 3 which comes out to 10^9.