9 ms·

I recently received a response to one of my comments saying that GPT-3 can't predict the future. From a naive understanding of the basic way it works, if it can

I recently received a response to one of my comments saying that GPT-3 can't predict the future. From a naive understanding of the basic way it works, if it can generate subsequent words and phrases that follow the rules of a language based on tokens, couldn't it be trained on larger concepts if those are represented as tokens? I understand that simulations are ultimately constrained by their assumptions, eg how the ultimate stable state of Sim City games seems to be some sort of authoritarian police-state. [1] Couldn't we create a simulation of human history, etc, and iterate on the rules by backtesting, ie doing something similar to the way stock trading indicators are tested? I do fear that AI could be used for nefarious purposes in that regard. "I want to see what needs to occur and what I need to do for this company to gain a monopoly in this industry." I'd be surprised if this isn't already occurring. If anyone has any links they can share about how AI is being applied to social spheres, that would be awesome. 1. https://www.polygon.com/videos/2021/4/1/22352583/simcity-hidden-politics-ideology-urban-dynamics https://www.polygon.com/videos/2021/4/1/22352583/simcity-hid...

simion314 5y ago
One problem is if you can train your ML correctly and not on garbage data. To predict the world you need a giant input and a ton of data. And you also have chaos, a little input divergence can cause a big output difference, I am not an ML-ANN dev but chaos feels to me impossible to approximate/interpolate with ANN, I would love to know if I am wrong and you can do any meaningful stuff on chaotic systems.
tsimionescu 5y ago
You're making an unwarranted assumption: that the methods used to predict the next token in human language would work for predicting the next token in a simulation of SimCity or history or weather etc. As far as we've seen, each of DeepMind's and OpenAI's successes have been a different network architecture, it's not like we have one network architecture that is good at learning anything you throw at it. Not to mention, the huge problem with weather and history is the limited amount of data, compared to human language, for which there are immense troves.
whimsicalism 5y ago
This is a good question and you are being unfairly downvoted. I think a big challenge is that there is no intuitive sense of "location"/position in the architecture. But yes, sequential architectures can be scaled to lots of things beyond just words.
- nefitty 5y ago
  Thanks for saying that. Sometimes HN makes me feel like I'm a lot dumber than I realize, but I'm so dumb that it's obvious to everyone else here lol

simion314 5y ago

One problem is if you can train your ML correctly and not on garbage data. To predict the world you need a giant input and a ton of data. And you also have chaos, a little input divergence can cause a big output difference, I am not an ML-ANN dev but chaos feels to me impossible to approximate/interpolate with ANN, I would love to know if I am wrong and you can do any meaningful stuff on chaotic systems.

tsimionescu 5y ago

You're making an unwarranted assumption: that the methods used to predict the next token in human language would work for predicting the next token in a simulation of SimCity or history or weather etc. As far as we've seen, each of DeepMind's and OpenAI's successes have been a different network architecture, it's not like we have one network architecture that is good at learning anything you throw at it. Not to mention, the huge problem with weather and history is the limited amount of data, compared to human language, for which there are immense troves.

whimsicalism 5y ago

This is a good question and you are being unfairly downvoted. I think a big challenge is that there is no intuitive sense of "location"/position in the architecture. But yes, sequential architectures can be scaled to lots of things beyond just words.

nefitty 5y ago
Thanks for saying that. Sometimes HN makes me feel like I'm a lot dumber than I realize, but I'm so dumb that it's obvious to everyone else here lol