4 ms·

Author of the paper here. Ask me anything!

YeGoblynQueenne 6y ago
Hi and congratulations for the article on IEEE. The article makes the point that your technique will give an advantage to academic teams that don't have the resourcers of big corporations. To me it seems that your technique optimises the use of available resources, but the amount of available resources remains the deciding advantage. That is to say, both large corporate teams and smaller academic teams can improve their use of resources using your proposed approach, but large corporate teams have more of those than the smaller academic teams. So the large corporate teams will still come up ahead and the smaller academic teams will still be left "in the dust" as the article puts it. What do you think?
- highfrequency 6y ago
  The key is that large corporations/labs achieve scale through many distributed machines. This paper explores optimizations that are particular to a single multi-core machine. These optimizations exploit low-latency shared memory between threads on one machine, and thus cannot be replicated on a distributed cluster.
highfrequency 6y ago
Hi Alex, cool stuff! A clarification from Section 3.2 on coordinating between rollout workers and policy workers: you say that we want k/2 > t_inf / t_env for maximal performance. If t_env is defined as the time required for one thread to simulate one step, it seems like we want k/2 > N * t_inf / t_env, where N is the number of cores/threads on the machine. For example, say t_inf = 10 microseconds, t_env = 1 microsecond and we are training on a 30 core machine. When k=600, we batch 300 inference steps in 10 mics, and complete 300 simulation steps on the other half of the rollout workers in 10 mics. Both cohorts of rollout workers finish at the same time, achieving optimal performance. Am I thinking about this correctly? Also, this equation assumes that we can batch up k/2 inference jobs on the GPU without increasing t_inf correct?

Hi and congratulations for the article on IEEE. The article makes the point that your technique will give an advantage to academic teams that don't have the resourcers of big corporations. To me it seems that your technique optimises the use of available resources, but the amount of available resources remains the deciding advantage. That is to say, both large corporate teams and smaller academic teams can improve their use of resources using your proposed approach, but large corporate teams have more of those than the smaller academic teams. So the large corporate teams will still come up ahead and the smaller academic teams will still be left "in the dust" as the article puts it. What do you think?

highfrequency 6y ago
The key is that large corporations/labs achieve scale through many distributed machines. This paper explores optimizations that are particular to a single multi-core machine. These optimizations exploit low-latency shared memory between threads on one machine, and thus cannot be replicated on a distributed cluster.

highfrequency 6y ago

Hi Alex, cool stuff! A clarification from Section 3.2 on coordinating between rollout workers and policy workers: you say that we want k/2 > t_inf / t_env for maximal performance. If t_env is defined as the time required for one thread to simulate one step, it seems like we want k/2 > N * t_inf / t_env, where N is the number of cores/threads on the machine. For example, say t_inf = 10 microseconds, t_env = 1 microsecond and we are training on a 30 core machine. When k=600, we batch 300 inference steps in 10 mics, and complete 300 simulation steps on the other half of the rollout workers in 10 mics. Both cohorts of rollout workers finish at the same time, achieving optimal performance. Am I thinking about this correctly? Also, this equation assumes that we can batch up k/2 inference jobs on the GPU without increasing t_inf correct?