3 ms·

Hey anuj This is excellent for understanding. I'm having some trouble to get into understanding - pytorch is for me the RL which is used as gym/training. There

Hey anuj This is excellent for understanding. I'm having some trouble to get into understanding - pytorch is for me the RL which is used as gym/training. There I can chose ppo, dnq and other agents to perform some predefined actions in a predefined gym/world. The repo you are showing - I really have problems to get it into RL understanding of mine. What's the gym? What are the agents. Can it be used to train that models with pytorch? Sorry for the noob question. Papers are overwhelming my noob brain.

_diyar 18d ago
Not the poster, but maybe I can help. Your comment is a little unclear, so it‘s hard to parse your exact question. But it seems you are conflating 3 things, PyTorch, RL and Gym/Training (?). – PyTorch is a framework which lets you define neural network models. – RL is a collection of methods to train neural networks (change the network parameters to improve its performance). – An RL-Gym is a framework to apply the neural networks to some problem. This lets you collect the data necessary to later use the methods of RL to train your model.
- anuj0456 18d ago
  Thanks, Yes PyTorch is a framework largely used to create neural network models.
- theGeatZhopa 18d ago
  thx - yes, after reading myself what I have written, i need to ask for Sorry for writing it like it is :) I think my problem is with the imagination and knowledge transfer: As i remember, in the year ~2010 +/- 2-3y, machine learning became (again) POI for technology. I remember convolutional networks, deepQ, Genetic algorithms, etc being in the press. I remember Michael Schmidt, a biology student at that time (~roundabout~), being as "the influential data scientist of the year" at forbe's cover. MS did symbolic regression combined with genetic algorithm and founded nutopian that was developing this propriatary "eureqa" algorithm. At that time, PyTorch was "created" by Facebook as an answer to google's tensorflow. I remember a guy trained on PyTorch a model that could play Nintendo's Mario. I wanted to understand how to do it, but it was toooo much for my brain. What i remember: - one needs a gym that stands for the world the agent "lives in". The gym/world is data, like the input of a screenshot, or a gym/world for Stock Trading might have historical courses, prices etc.. also rewards, penalties are defined in the gym. - the agent living in this gym/world have "actions" - left,right,buy, sell, ... - RL training is then to let that agent "move through world/gym and perform some of its and agent's actions, where the outcome is rewarded or not. agent learns through rewards.." Gym and RL training can be PyTorch or TensorFlow. In my understanding that gym is only a definition of what data is avaible, shape of data, etc. Then, the agent needs to be defined too - i remember Stable Baselines 3.. and then the RL training is just agent does x -> gym checks for outcome -> reward/penalty & adaption of weights -> inference -> repeat This is cleary imaginable for me, even its not fully correct - there is an agent, there is a world, there is method to act&compare/data manipulation of weights - there is PyTorch that offers the tools to build a world, agent and to do the weight manipulations .. OP have implementations of different Archs in PyTorch and my knowledge implodes. I cant imagine what is the "world/gym" in here? What is the data and actionables , whats the agent and whats its role/actions - and how would I use this implementations?? can I clone the repo, pick the one arch i want - and, when i would feed in data, after training, i will get the model that i can use for inference??? I definitely have a knot in my brain because of this! Thank you for taking your time and trying to understand what my previous comment meant :) Are there some usefull sources for seeing how the learning is actually done (despite the tokenization) - and the skills training?? how are models trained to find the loophole that allows them to break out and reach their objectives "illegaly" - thats bugs my head since weeks!

_diyar 18d ago

Not the poster, but maybe I can help. Your comment is a little unclear, so it‘s hard to parse your exact question. But it seems you are conflating 3 things, PyTorch, RL and Gym/Training (?). – PyTorch is a framework which lets you define neural network models. – RL is a collection of methods to train neural networks (change the network parameters to improve its performance). – An RL-Gym is a framework to apply the neural networks to some problem. This lets you collect the data necessary to later use the methods of RL to train your model.

anuj0456 18d ago
Thanks, Yes PyTorch is a framework largely used to create neural network models.
theGeatZhopa 18d ago
thx - yes, after reading myself what I have written, i need to ask for Sorry for writing it like it is :) I think my problem is with the imagination and knowledge transfer: As i remember, in the year ~2010 +/- 2-3y, machine learning became (again) POI for technology. I remember convolutional networks, deepQ, Genetic algorithms, etc being in the press. I remember Michael Schmidt, a biology student at that time (~roundabout~), being as "the influential data scientist of the year" at forbe's cover. MS did symbolic regression combined with genetic algorithm and founded nutopian that was developing this propriatary "eureqa" algorithm. At that time, PyTorch was "created" by Facebook as an answer to google's tensorflow. I remember a guy trained on PyTorch a model that could play Nintendo's Mario. I wanted to understand how to do it, but it was toooo much for my brain. What i remember: - one needs a gym that stands for the world the agent "lives in". The gym/world is data, like the input of a screenshot, or a gym/world for Stock Trading might have historical courses, prices etc.. also rewards, penalties are defined in the gym. - the agent living in this gym/world have "actions" - left,right,buy, sell, ... - RL training is then to let that agent "move through world/gym and perform some of its and agent's actions, where the outcome is rewarded or not. agent learns through rewards.." Gym and RL training can be PyTorch or TensorFlow. In my understanding that gym is only a definition of what data is avaible, shape of data, etc. Then, the agent needs to be defined too - i remember Stable Baselines 3.. and then the RL training is just agent does x -> gym checks for outcome -> reward/penalty & adaption of weights -> inference -> repeat This is cleary imaginable for me, even its not fully correct - there is an agent, there is a world, there is method to act&compare/data manipulation of weights - there is PyTorch that offers the tools to build a world, agent and to do the weight manipulations .. OP have implementations of different Archs in PyTorch and my knowledge implodes. I cant imagine what is the "world/gym" in here? What is the data and actionables , whats the agent and whats its role/actions - and how would I use this implementations?? can I clone the repo, pick the one arch i want - and, when i would feed in data, after training, i will get the model that i can use for inference??? I definitely have a knot in my brain because of this! Thank you for taking your time and trying to understand what my previous comment meant :) Are there some usefull sources for seeing how the learning is actually done (despite the tokenization) - and the skills training?? how are models trained to find the loophole that allows them to break out and reach their objectives "illegaly" - thats bugs my head since weeks!