5 ms·
So, this is pretty exciting. I can how this can already be used to generate realistic physics approximations in a game engine. You create a bunch of snippets o
by marcyb5st 2y ago
So, this is pretty exciting.
I can how this can already be used to generate realistic physics approximations in a game engine. You create a bunch of snippets of gameplay using a much heavier and realistic physics engine (perhaps even CGI). The model learn to approximate the physics and boom, now you have a lightweight physics engine. Perhaps you can even have several that are specialized (e.g. one for smoke dynamics, one for explosions, ...). Even if it allucinates, wouldn't be worse than the physics bugs that are so common in games.
- hobs 2y agoA physics bug would be a consistent problem you can fix. There's no such guarantee about an ML model. This would likely only be ok in the context of a game specifically made to be janky.
- kqr 2y agoThis obsession people have with determinism! I'd much rather take a low rate of weird bugs than common consistent ones. I don't believe reproducibility of bugs makes for better gameplay generally.
- paulryanrogers 2y agoReproducibility does make bugs more likely to be fixed, or at least fixable. Also, games introduce randomness in a controlled way so users don't get frustrated by it appearing in unexpected places. I don't want characters to randomly appear and disappear. It's fine if bullet trajectory varies more randomly as they get further away.
- skydhash 2y agoAlso most engines have been worked on for years. So more often than not, core elements like audio, physics, input,... are very stable and the remaining bugs are either "can't fix" or "won't fix".
- NotMichaelBay 2y agoIt might be fine for casual players, but it would prevent serious and pro players from getting into the game. In Counter-Strike, for example, pro players (and other serious players) practice specific grenade throws so they can use them reliably in matches.
- kqr 2y agoI'm not saying one can make specifically Counter-Strike on a non-deterministic engine -- that seems like strawmanning my argument. People play and enjoy many games with varying levels of randomness as a fundamental component, some even professionally (poker, stock market). This could be made such a game.
- monsieurbanana 2y agoEither the physics engine matter, in which case you want a deterministic engine as you said, or it doesn't like in a poker game and you don't want to spend much resources (manpower, computer cycles) into it. Which also means an off-the-shelf deterministic engine.
- mrob 2y agoThe whole hobby of speedrunning relies heavily on exploiting deterministic game bugs.
- hobs 2y agoMake a fun game with this as a premise and I will try it, but it sounds just an annoying concept.
- FridgeSeal 2y agoGosh can you imagine how frustrating a game would be when you can _barely_ do anything reliably? Wanted to jump on the platform? Wow too bad, model had a little day-dream and now you’re floating somewhere. Wanted to peak a corner? Oops too bad, model had a moment and now the wall curves differently. Trying to pick a shot-well you’d definitely hit the other player if the model wasn’t making physics go out the window. Inconsistently.
- dartos 2y agoYou don’t play a lot of games, huh? Consistent bugs you can anticipate and play/work around, random ones you can’t. Just look at pretty much any speed running community for games before 1995. Say goodbye to any real competitive scene with random unfixable potentially one off bugs.
- siev 2y ago[dead]
- fullstackwife 2y agoThis is one of the fallacies of current AI research space: they don't focus on the end-user too much. In this case the end-user would be the gamer, and while playing games you expect a valid gameplay, so those kind of hallucinations are not acceptable, while I'm pretty sure they give the AI research authors a strong dopamine trigger. We have a hammer and now we are looking for a nail, while you should ask a question first: what is the problem we are trying to solve here? Real world usage will be probably different, and maybe even unexpected by the authors of this research.
- jsheard 2y ago> This is one of the fallacies of current AI research space: they don't focus on the end-user too much. In this case the end-user would be the gamer Or from another angle the end-user is a game developer trying to actually work with this kind of technology, which is just a nightmarish prospect. Nobody in the industry is asking for a game engine that runs entirely on vibes and dream logic, gamedev is already chaotic enough when everything is laid out in explicit code and data.
- badpun 2y agoThe whole purpose of academia is literally to nerd out on cool, impractical things, which will ocasionally turn out to have some real-life relevance years or decades later. This (hallucinated CS) is still more relevant to real world than 99% of what happens in academic research.
- dartos 2y agoYes to the first part, no to the random “99% useless” number you made up. I’m no fan of academia, but it undeniably produces useful and meaningful knowledge regularly.
- badpun 2y agoYou made up the "99% useless" straw man, I never used the word useless in my post. I merely said that the AI research is more relevant than 99% of research, which includes things like gender studies, studies of Rennaisance German theatre, discussions of which exact villages were pillaged and in what order during the march of some army 300 years ago etc. etc.
- monsieurbanana 2y ago> Even if it allucinates, wouldn't be worse than the physics bugs that are so common in games. I don't know about that. Physic bugs are common, but you can prioritize and fix the worst (gamebreaking) ones. If you have a blackbox model, it becomes much harder to do that.
- Thorrez 2y agoWould that work for multiplayer? If it's a visual effect only, I guess it would be ok. But if it affects gameplay, wouldn't different players get different results?
- killerstorm 2y agoWell, it doesn't make sense to use this exact model - this is just demonstration that it can learn world model from pixels. An obvious next step towards a more playable game is to add state vector to the inputs of the model: it is easier to learn to render the world from pixels + state vectors than from pixels alone. Then it depends what we want to do. If we want normal Counter Strike gameplay but with new graphics, we can keep existing CS game server and train only the rendering part. If you want to make Dream-Counter-Strike where rules are more bendable then you might want to train state update model...
- twic 2y agoDo you think that inference on a thirteen million parameter neural network is more lightweight than running a conventional physics engine?
- epolanski 2y agoEvery software that can be implemented in a JavaScript, ehm, LLM, will eventually be implemented in an LLM.
- kendalf89 2y agoAre you predicting node.llm right now?
- procgen 2y agoConvincing liquid physics (e.g. surf interacting with a beach, rocks, the player character) might be a good candidate.
- tiagod 2y agoIn some cases, the model will be lighter. There is no need for 14M parameters for physics simulations, and there's a lot of promising work in that area.
- amelius 2y ago> boom, now you have a lightweight physics engine lightweight, but producing several hundred watts of heat.
- marcyb5st 2y agoNot true. If you just need a model to learn the physics and not all the pixels I am sure you can get a much smaller model. If you look at the metnet papers similar approaches have been successful for weather prediction where models have better results and are order of magnitude faster at inference time compared to numerical models. Of course, training will be time consuming and resources intensive, but it's something you can do once and then ship it to users when they install / update the game.
- bobsomers 2y agoWhat makes you think the network inference is less expensive? Newtonian physics is already extremely well known and pretty computationally efficient to compute. How would a "function approximation" of Newtonian physics, with billions of parameters, be cheaper to compute? It seems like this would both be more expensive and less correct than a proper physics simulation.
- crackalamoo 2y agoBasic Newtonian physics is pretty efficient to compute, but afaik some more complex physics like fluids is faster with network inference. There are probably a lot of cases where network inference physics is faster.
- crazygringo 2y agoYeah, I definitely wouldn't trust it to replace basic physics of running, jumping, bullets, objects shattering, etc. But it seems extremely promising for fiery explosions, smoke, and especially water. Anything with dynamics that are essentially complex. Also for lighting -- both to get things like skin right with subsurface scattering, as well as global ray-traced lighting. You can train specific lightweight models for these things, and they important thing is that their output is correct at the macro level. E.g., a tree should be casting a shadow that looks like the right shadow at the right angle for that type of tree and its types of leaves and general shape. Nobody cares if each individual leaf shadow corresponds to an individual leaf 10 feet above or is just hallucinated.
- BriggyDwiggs42 2y agoI was thinking it would be especially effective for situations where one needed to predict the behavior of a chaotic system, not so much accurately but convincingly. Turbulence flow would be an irreducible system right?
- slashdave 2y agoDefine "lightweight".