10 ms·

Google Stadia uses Style Transfer ML to change video game art in real time

143 points by pavel_tiunov 8y ago

deleted 8y ago
[deleted]
lwb 8y ago
To me this was the least interesting part of the Stadia reveal. State share was the real innovation imo. Would any serious game studio use style transfer ml to actually replace their visual artists? Didn't seem like the quality was there. That said, it is an awesome PoC. Just not something I see being practically applicable.
- kirkules 8y ago
  Honestly seems more like a user-end personalization than something a developer would use. In a game settings menu, you choose an image that maybe you uploaded to use as the game style? Definitely something people would use. Especially if they figure out how to make it selectively apply to only certain textures/objects or parts of the screen, which seems likely.
  - uxp100 8y ago
    Street Fighter 4 had a feature kinda like that (with predefined selectable styles). AFAIK it was not popular. Sometimes my friends would turn it on as a joke.
- xiphias2 8y ago
  I would have been much more interested in using deep learning to make human faces more realistic. Somebody was experimenting with it in the past quite successfully IIRC, but I'd love to try it on a real game.
- dragonwriter 8y ago
  > Would any serious game studio use style transfer ml to actually replace their visual artists? No, a serious (AAA) studio would not. OTOH, the platform might intend to support producers that aren't AAA studios, and whose limited pre-release resources are focussed on gameplay more than art assets, and those producers might view it very differently than AAA studios.
  - libraryatnight 8y ago
    Indeed, I'd be curious to see what some indie devs do with it.
    - joshuamorton 8y ago
      Imagine a game where you step into a portrait and are suddenly in the art style of the image. You suddenly walk into a countryside that feels american-gothic style, and then time-jump to 1940s New York in the style of Nighthawks. (different) Stylization as a medium in games isn't that explored, and the opportunity for indie devs to play with that is, IMO, really cool.
      - NathanKP 8y ago
        The Witcher Blood and Wine expansion actually did a great job with this. There is a main storyline quest where you enter a haunted painting
  - ghostbrainalpha 8y ago
    Would it help in game development? For example giving developers better looking playable prototypes while the artists are still working out their designs?
  - cwyers 8y ago
    You still have to do all the work of building the geometries for the art assets, are you really saving that much on not having to color/texture those assets?
- heyitsguay 8y ago
  I think the more interesting applications will come when you get tools like this used by visual artists, complementing handmade design. The demo is cool but uses style transfer very bluntly, applying one style to the whole image with high intensity. I imagine the future of this is something more like neural "shaders", able to apply real-time pixel alterations to scenes or objects within scenes. Used carefully, they could increase the range of creative choices a visual artist can make.
  - Lerc 8y ago
    This is where I think the value lies. I have been thinking about a process where a procedurally generated template is completed by artists and subsequently used as a base for training an AI to apply to further procedural generations. I also wonder if you can do micro style adjustments to unify artistic style, train a net on one particular artists work, the rest of the team copy the general style but use a final pass of a style transfer to make it a closer match.
- AnIdiotOnTheNet 8y ago
  Actually, I wonder if there's potential in something like this for accessibility purposes. If you have one of the many variations of color-blindness, for instance, could such a restyling be helpful? Could other neurological disabilities be similarly aided by dynamically restyling visuals?
  - cycrutchfield 8y ago
    There are many low-tech solutions that can be applied instead of shoehorning in neural networks for this purpose.
    - AnIdiotOnTheNet 8y ago
      Yes, but they also require active participation from the developer and many developers are unwilling or unable to spend the time to implement them, especially when there's a rather large range of disabilities to account for. It's possible that tech like this could offer a user-definable solution to these issues.
- tanilama 8y ago
  Exactly...This is a cool gimmick to try once, but it will seriously distort the intended art style. It is another example of forced intrusion of DL techniques creeping into places where it is not needed in the first place.
  - tenaciousDaniel 8y ago
    Gimmick is the perfect way to describe it. I really don't understand why they even included it.
- pducks32 8y ago
  One incredible use is actually in network training for things like AR and autonomous driving. You want a slightly fake world and you could easily try out various network approaches using a free fake world found in game engines. Microsoft has something for this but cannot find a link or the name right now.
- m3kw9 8y ago
  Maybe used as a player entering dream like equivalent after eating magic mushrooms. Other than that you’d be hard pressed to see this in any game in memory
- Fellshard 8y ago
  Hypothesis: the rough and relatively unpredictable results are not so much a good fit for finished products as it is for _rapid visual prototyping_. A good art director could use this to quickly settle on a rough goal around which the rest of the game would be targeted. I'm aware the pipeline usually goes from art director -> development, but this could help, say, a competent dev team with a gameplay direction but no artist yet, or a long-standing developer that wants to use its existing engines and mechanics to draft new, distinctive franchises.
  - ender7 8y ago
    This is the use case they highlighted in the keynote. It doesn't appear to be a system designed for shipped games.
- anonytrary 8y ago
  I was baffled with the style transfer announcement. It seems arbitrary, random, and out-of-place. Style transfer (applied to art) is really neat, but I can't see it being used to re-clothe games for production. Before I get downvoted, yes I do think it could be useful for prototyping and conceptualization during development, but I doubt anyone would actually ship a complete game with the entire camera view transformed by style transfer.
- pier25 8y ago
  > Would any serious game studio use style transfer ml to actually replace their visual artists? I doubt that was the point. IMO it is more of a palette testing tool.
vernie 8y ago
y tho
- shultays 8y ago
  They need to show something shiny to hide the fact that the system has 100ms of latency on a test environment. I wonder how much lag the style transfer would add
  - j-c-hewitt 8y ago
    100ms+ feels worse in this kind of thing compared to 100ms ping to a multiplayer server because even if you have bad ping to a server the game still records your inputs locally and feels responsive. More realistically it will be more like 180ms+ on a home connection. I was an OnLive user when it existed earlier in the decade and I would say that it felt generally fine in games with forgiving timers that were not all that hard. In any game that is challenging, this kind of lag even in single player games can be very frustrating, and you might as well not even think about multiplayer because it will be a much worse experience than playing on a 56K modem in the late 1990s. The old Assassin's Creed games were a great fit for the technology because nothing required a reaction time of under 1000 ms and movement did not require a lot of fine adjustments. Anything turn based is also a great fit for this kind of technology. Even Steam Link wireless in the same house over a good router has annoying input lag that may not be immediately noticeable until you compare it to a wired connection or using the PC itself. Another big issue with this in the US is that only rich people are going to have the quality of connection you would need to enjoy this at good quality. And if you are rich, spending like $1k on a budget gaming PC or a few hundred bucks on a console are going to be pretty reasonable. No one would be dumb enough to spend $300/month+ on a high bandwidth connection so that they can stream games to potato hardware. On the other hand, if high bandwidth connections were ubiquitous in the US as they are in some other developed countries at an affordable price, the calculus would be totally different.
    - ghostbrainalpha 8y ago
      You are totally right about the turn based games. I used a game streaming service called Shadow, and it works amazing for Sim City, Frost, and Civilization type games. I really hope Google focused on putting out quality titles in the city builder genres. Hell it might even make sense if they outright bought the Civilization franchise.
adamnemecek 8y ago
This seems like a solution looking for a problem.
roywiggins 8y ago
Fun, but nobody will ever do it for more than 5 seconds before getting a headache and turning it off again.
theknight 8y ago
this can potentially be a "mod" play... where individual gamers can change game "skin" and stream videos on youtube...
andreygrehov 8y ago
It could be that they are throwing it out so that the community try to find a practical application of it. It is obvious that they understand that what they are demoing is not the end product so to speak, but just the best way to represent general idea.
newobj 8y ago
There is nothing more Googley than building something that's technically impressive with absolutely no - or even antithetical - market fit
- caprese 8y ago
  Onlive - required other people's infrastructure and a hardware play, and a software client. Playstation Now - required Sony to buy/use a cloud service, maintain it, and expensive Sony hardware just as a force of habit. Google - negligible instance running on their existing infrastructure, maintained under existing processes, playable on chromecasts already in hotels and homes everywhere, alongside browsers and mobile devices. If Google doesn't shut it down within the 5 years for people to notice this is even an option, they could focus more on titles and developers more than any other entrant to this market. And perhaps the gaming is just the application to get people's attention.
  - WrtCdEvrydy 8y ago
    The market is dead because Nvidia only want their high tier graphics cards to be deployed... can you imagine if instead of deploying 30,000 dollar GPUs, you could drop quad Titan X's or Titan V's. The datacenter license is cancer!
  - heavenlyblue 8y ago
    Are you implying Google doesn’t need to maintain an expensive cloud infrastructure for it as opposed to Playstation? The only reason they can do that today without specialised hardware a-la OnLive is because Nvidia and co. now have specialised GPU circuits allowing one to stream the encoded video output to memory. When OnLive started they had to do it themselves. I don’t believe you have any idea what you are speaking about.
    - BeefySwain 8y ago
      > Nvidia and co. now have specialised GPU circuits allowing one to stream the encoded video output to memory Can you give a little more info on this? Is this standard to consumer level GPU's? What is the tech called?
kerng 8y ago
I'm already excited to see what Microsoft will show at E3 with Xcloud streaming service that they have been working on for a couple of years- lots of great progress in that area right now.
eof 8y ago
Consensus in this thread seems to be that this is a neat gimmick but nothing else. Maybe because I have a friend working on a video game as a solo developer, and putting a /lot/ of energy/money/time into art--I really see the potential in this (type of thing). This is not really for the player's benefit to put on whatever mods they want. This is for fast iteration for artists and creators to automate a huge (sometimes necessary evil) burden.
- kllrnohj 8y ago
  > Maybe because I have a friend working on a video game as a solo developer, and putting a /lot/ of energy/money/time into art--I really see the potential in this (type of thing). For that usage Nvidia's thing was way more interesting: https://blogs.nvidia.com/blog/2019/03/18/gaugan-photorealistic-landscapes-nvidia-research/ https://blogs.nvidia.com/blog/2019/03/18/gaugan-photorealist... That gets you usable texture assets super quickly, which is probably what your solo developer friend spends the most time on. That or models. Neither of which Stadia's style transfer helps with. They probably aren't spending much time tweaking a shader-applied post-process effect, but hey, maybe they are, and this style transfer ML thing is useful. Very, very unlikely, but maybe.
  - mikewhy 8y ago
    > For that usage Nvidia's thing was way more interesting If you're building a photorealistic game. And there's already lots of tools on the market to generate these sorts of textures. I could see this Nvidia thing used for much better looking mockups / concept art. But it doesn't seem suited for final asset generation.
    - kllrnohj 8y ago
      > But it doesn't seem suited for final asset generation. Well yes, of course, and neither is Google's Style Transfer ML. These are both just prototyping tools, not a magic wand to make production-quality assets.
ohthanks 8y ago
This is not a consumer product at all. I don't know how useful it will be but it's designed to be development tool for early art direction visualization. Even for 2d visualization it could be very useful if you could drop images into your website mockup files and have it intelligently apply color palette and texture information to your existing content. It won't look good, but it might look good enough to save you from having to manually tweak the design 20-30 times to explore a range of styles.
andy_ppp 8y ago
Can't wait for ads to be inserted into games with this...
- fileeditview 8y ago
  A few days ago I commented on the Stadia on HN and basically predicted the same thing. Does not seem to be a popular opinion however..
  - andy_ppp 8y ago
    I do find HN to be against sarcasm of any kind, which in it's own way is slightly ironic. I enjoyed your comment [1] and consequently upvoted it! [1] https://news.ycombinator.com/item?id=19433773 https://news.ycombinator.com/item?id=19433773
    - babuskov 8y ago
      That was sarcasm? I though he was serious and upvoted. I can imagine walking down the street in Call of Duty game or something like Cyberpunk 2077 and seeing ads overlaid on wall textures of buildings.
      - penagwin 8y ago
        A while ago PUBG plastered ads for their Berlin Invitational. While they were ads related to the game, the incrementally added them EVERYWHERE! First they're on a bus-stop wall, that's thematic okay. Then they're on random walls - getting annoying. Then the loading screen and the parachutes, nows it's annoying. Like, guys I'm from the US and I don't want to go to your stupid competition!
smaili 8y ago
Is anyone else concerned this will be yet another Google product that goes by the wayside in a few years? This has essentially become my biggest fear now with any new product they announce/release.
- robotmay 8y ago
  It's pretty much a certainty with this project, in my opinion. Until everybody somehow has gigabit lines to their home and to their phones, I don't see this becoming popular. Like most Google projects, their actual success period is most likely 10+ years away, and the project will be axed long before then.
  - chatmasta 8y ago
    5G deployments have already started rolling out. 5G has average latency in the range of 1-2ms. Google controls a huge swath of the backbone after that. They can realistically achieve < 5ms latency on 5G devices in the next 1-2 years. IMO, a more valid argument against its success would be that any consumer who is in a position to ensure they have fast enough internet for this service, is also in a position to buy a gaming console. But the counterargument to that is that there is no reason someone cannot have a gaming console and also use Stadia. I'm normally very skeptical of Google's motivations, respectful of their cloud technology, and doubtful about many of their products. But I think this one will be a success, and shortly followed by competing products from Amazon and Microsoft.
    - robotmay 8y ago
      Seeing as I live just on the outskirts of a capital city (Cardiff), and I can't even get 3G signal inside my house, and no more than 80/20 broadband; I'll remain skeptical about the possibility of success for this outside of very specific cities. There's also a lot to be said for reliability and the requirement to always have a fast connection: one of the benefits to many videogames is the fact that they are not online. Isn't 5G on a wavelength that has poorer penetration than 4G? It might work in the USA but signal is a constant issue in stone housing here in the UK (though it may be better where you are based). I had to get a femtocell from EE in order to get signal in my house :\
SubiculumCode 8y ago
I've seen a number of these impressive looking demos..for example: https://www.theverge.com/2018/12/3/18121198/ai-generated-video-game-graphics-nvidia-driving-demo-neurips https://www.theverge.com/2018/12/3/18121198/ai-generated-vid... Impressive sense of realism in level of detail variety..show the potential...BUT consistency, sharpness and realism of object boundaries would not survive close inspection.
summerlight 8y ago
I don't think there will be heavy users of style transfer among high budget games anytime soon, but the important part is that it's done in real time. With an assumption that it has a reasonable level of latency, probably AA, super resolution or any other neural net based quality improvement can be a real thing.
currymj 8y ago
this is certainly just a neat gimmick, but it's interesting to me for a particular reason: the Stadia box uses an AMD GPU, and per the Google announcement, all the style transfer is done in real time. this suggests that we may soon have better AMD support in TensorFlow.
- make3 8y ago
  Tensorflow support on AMD is supposedly not awful anymore. https://medium.com/tensorflow/amd-rocm-gpu-support-for-tensorflow-33c78cc6a6cf https://medium.com/tensorflow/amd-rocm-gpu-support-for-tenso... I wish someone who had >actual< practical experience in the domain would comment. It's an important question.
  - currymj 8y ago
    I haven't personally tried to do anything on an AMD GPU, but AMD actually has a very active GitHub repository https://github.com/ROCmSoftwarePlatform/tensorflow-upstream https://github.com/ROCmSoftwarePlatform/tensorflow-upstream . If you go through the issues you can get a sense of what it's like from people who do have practical experience.
    - make3 8y ago
      this is like bad selection bias though. people never write issues to say things are going well. the general Tensorflow issues are also about only bad things, by definition
  - avinium 8y ago
    I've been following ROCm development closely, and everything I read is pointing towards it being a first class competitor to CUDA. That being said, I don't have any personal experience because, well, I don't have AMD GPUs to play around with.
- Sephr 8y ago
  Style transfer is most likely not running on the GPU in this case. It's probably running on Google TPUs, given that the GPUs are likely already under high load in order to play AAA games at 4K HDR 60fps.
andjd 8y ago
Did anyone else see this headline and think: Why would they use a customized XML for this in 2019?
_bxg1 8y ago
A neat tech demo, but as with so many things coming out of The Valley these days, I don't think anybody actually wants this. No actual game is so bland that you'd want the ability to overlay your own "themes" on it after the fact. AI has lots of potential for assisting artists, and maybe part of that will include post-processing effects, but putting such things in the player's hands is pointless.
b_tterc_p 8y ago
Silent hill concepts could be fun with this. Otherwise eh. I would like to see Jojo palette swaps occur to highlight tension but I don’t see that being implemented with something like this.
alkonaut 8y ago
The useful application for AI in streamed gaming seems like it would be for predicting the next 30ms to avoid input lag. This seems gimmicky at best. Prediction has to take place on the client though - so the challenge is making it cheap.
- nabergh 8y ago
  This is really cool to think about. In practice though I would imagine every incorrect prediction to be incredibly jarring. Additionally my intuition is that latency is easier to decrease than predicting the next 30ms. If there is some level of latency that is very difficult to reduce (more so than training an AI to predict the next 30ms with very high accuracy), I'd be curious to know why.
  - alkonaut 8y ago
    Between e.g. central and west US you'd have a physical ping of ~10ms so 20 round trip just based on the speed of light (i.e. very hard to get rid of). Meanwhile the input lag you want is one or two frames so on the same order. But this lag is additive, when the game receives the new state it has to compute and render the world, the display has its lag before it shows up etc. So the new frame has to start computing as soon as the input is registered in order to hit reasonable input lag. Predicting 2-3 frames should't beyond what we can do at least roughly. Especially for some important inputs such as turning in first person view it should basically just start moving the screen. I belive John Carmack tried some kind of rotational prediction (Not based on AI) for Oculus.
- chris_mc 8y ago
  I am not a game developer, but couldn't you make a multiplayer game more fair in this situation by adding artificial lag into everyone's controls to match the highest lag person? Just limit it to a certain amount, like no more than 200-300ms.
  - alkonaut 8y ago
    I wasn’t considering fairness here (that’s a whole different can of worms). Even for a game where everyone’s lag is the same, X milliseconds of input lag and Y milliseconds of network latency is painful for some values of X and Y. For fast paced games I’d say X is probably just a couple of frames (15-30ms) and Y is about 100ms.
deleted 8y ago
[deleted]

143 points by pavel_tiunov 8y ago

deleted 8y ago

[deleted]

lwb 8y ago

To me this was the least interesting part of the Stadia reveal. State share was the real innovation imo. Would any serious game studio use style transfer ml to actually replace their visual artists? Didn't seem like the quality was there. That said, it is an awesome PoC. Just not something I see being practically applicable.

kirkules 8y ago
Honestly seems more like a user-end personalization than something a developer would use. In a game settings menu, you choose an image that maybe you uploaded to use as the game style? Definitely something people would use. Especially if they figure out how to make it selectively apply to only certain textures/objects or parts of the screen, which seems likely.
- uxp100 8y ago
  Street Fighter 4 had a feature kinda like that (with predefined selectable styles). AFAIK it was not popular. Sometimes my friends would turn it on as a joke.
xiphias2 8y ago
I would have been much more interested in using deep learning to make human faces more realistic. Somebody was experimenting with it in the past quite successfully IIRC, but I'd love to try it on a real game.
dragonwriter 8y ago
> Would any serious game studio use style transfer ml to actually replace their visual artists? No, a serious (AAA) studio would not. OTOH, the platform might intend to support producers that aren't AAA studios, and whose limited pre-release resources are focussed on gameplay more than art assets, and those producers might view it very differently than AAA studios.
- libraryatnight 8y ago
  Indeed, I'd be curious to see what some indie devs do with it.
  - joshuamorton 8y ago
    Imagine a game where you step into a portrait and are suddenly in the art style of the image. You suddenly walk into a countryside that feels american-gothic style, and then time-jump to 1940s New York in the style of Nighthawks. (different) Stylization as a medium in games isn't that explored, and the opportunity for indie devs to play with that is, IMO, really cool.
    - NathanKP 8y ago
      The Witcher Blood and Wine expansion actually did a great job with this. There is a main storyline quest where you enter a haunted painting
- ghostbrainalpha 8y ago
  Would it help in game development? For example giving developers better looking playable prototypes while the artists are still working out their designs?
- cwyers 8y ago
  You still have to do all the work of building the geometries for the art assets, are you really saving that much on not having to color/texture those assets?
heyitsguay 8y ago
I think the more interesting applications will come when you get tools like this used by visual artists, complementing handmade design. The demo is cool but uses style transfer very bluntly, applying one style to the whole image with high intensity. I imagine the future of this is something more like neural "shaders", able to apply real-time pixel alterations to scenes or objects within scenes. Used carefully, they could increase the range of creative choices a visual artist can make.
- Lerc 8y ago
  This is where I think the value lies. I have been thinking about a process where a procedurally generated template is completed by artists and subsequently used as a base for training an AI to apply to further procedural generations. I also wonder if you can do micro style adjustments to unify artistic style, train a net on one particular artists work, the rest of the team copy the general style but use a final pass of a style transfer to make it a closer match.
AnIdiotOnTheNet 8y ago
Actually, I wonder if there's potential in something like this for accessibility purposes. If you have one of the many variations of color-blindness, for instance, could such a restyling be helpful? Could other neurological disabilities be similarly aided by dynamically restyling visuals?
- cycrutchfield 8y ago
  There are many low-tech solutions that can be applied instead of shoehorning in neural networks for this purpose.
  - AnIdiotOnTheNet 8y ago
    Yes, but they also require active participation from the developer and many developers are unwilling or unable to spend the time to implement them, especially when there's a rather large range of disabilities to account for. It's possible that tech like this could offer a user-definable solution to these issues.
tanilama 8y ago
Exactly...This is a cool gimmick to try once, but it will seriously distort the intended art style. It is another example of forced intrusion of DL techniques creeping into places where it is not needed in the first place.
- tenaciousDaniel 8y ago
  Gimmick is the perfect way to describe it. I really don't understand why they even included it.
pducks32 8y ago
One incredible use is actually in network training for things like AR and autonomous driving. You want a slightly fake world and you could easily try out various network approaches using a free fake world found in game engines. Microsoft has something for this but cannot find a link or the name right now.
m3kw9 8y ago
Maybe used as a player entering dream like equivalent after eating magic mushrooms. Other than that you’d be hard pressed to see this in any game in memory
Fellshard 8y ago
Hypothesis: the rough and relatively unpredictable results are not so much a good fit for finished products as it is for _rapid visual prototyping_. A good art director could use this to quickly settle on a rough goal around which the rest of the game would be targeted. I'm aware the pipeline usually goes from art director -> development, but this could help, say, a competent dev team with a gameplay direction but no artist yet, or a long-standing developer that wants to use its existing engines and mechanics to draft new, distinctive franchises.
- ender7 8y ago
  This is the use case they highlighted in the keynote. It doesn't appear to be a system designed for shipped games.
anonytrary 8y ago
I was baffled with the style transfer announcement. It seems arbitrary, random, and out-of-place. Style transfer (applied to art) is really neat, but I can't see it being used to re-clothe games for production. Before I get downvoted, yes I do think it could be useful for prototyping and conceptualization during development, but I doubt anyone would actually ship a complete game with the entire camera view transformed by style transfer.
pier25 8y ago
> Would any serious game studio use style transfer ml to actually replace their visual artists? I doubt that was the point. IMO it is more of a palette testing tool.

vernie 8y ago

y tho

shultays 8y ago
They need to show something shiny to hide the fact that the system has 100ms of latency on a test environment. I wonder how much lag the style transfer would add
- j-c-hewitt 8y ago
  100ms+ feels worse in this kind of thing compared to 100ms ping to a multiplayer server because even if you have bad ping to a server the game still records your inputs locally and feels responsive. More realistically it will be more like 180ms+ on a home connection. I was an OnLive user when it existed earlier in the decade and I would say that it felt generally fine in games with forgiving timers that were not all that hard. In any game that is challenging, this kind of lag even in single player games can be very frustrating, and you might as well not even think about multiplayer because it will be a much worse experience than playing on a 56K modem in the late 1990s. The old Assassin's Creed games were a great fit for the technology because nothing required a reaction time of under 1000 ms and movement did not require a lot of fine adjustments. Anything turn based is also a great fit for this kind of technology. Even Steam Link wireless in the same house over a good router has annoying input lag that may not be immediately noticeable until you compare it to a wired connection or using the PC itself. Another big issue with this in the US is that only rich people are going to have the quality of connection you would need to enjoy this at good quality. And if you are rich, spending like $1k on a budget gaming PC or a few hundred bucks on a console are going to be pretty reasonable. No one would be dumb enough to spend $300/month+ on a high bandwidth connection so that they can stream games to potato hardware. On the other hand, if high bandwidth connections were ubiquitous in the US as they are in some other developed countries at an affordable price, the calculus would be totally different.
  - ghostbrainalpha 8y ago
    You are totally right about the turn based games. I used a game streaming service called Shadow, and it works amazing for Sim City, Frost, and Civilization type games. I really hope Google focused on putting out quality titles in the city builder genres. Hell it might even make sense if they outright bought the Civilization franchise.

adamnemecek 8y ago

This seems like a solution looking for a problem.

roywiggins 8y ago

Fun, but nobody will ever do it for more than 5 seconds before getting a headache and turning it off again.

theknight 8y ago

this can potentially be a "mod" play... where individual gamers can change game "skin" and stream videos on youtube...

andreygrehov 8y ago

It could be that they are throwing it out so that the community try to find a practical application of it. It is obvious that they understand that what they are demoing is not the end product so to speak, but just the best way to represent general idea.

newobj 8y ago

There is nothing more Googley than building something that's technically impressive with absolutely no - or even antithetical - market fit

caprese 8y ago
Onlive - required other people's infrastructure and a hardware play, and a software client. Playstation Now - required Sony to buy/use a cloud service, maintain it, and expensive Sony hardware just as a force of habit. Google - negligible instance running on their existing infrastructure, maintained under existing processes, playable on chromecasts already in hotels and homes everywhere, alongside browsers and mobile devices. If Google doesn't shut it down within the 5 years for people to notice this is even an option, they could focus more on titles and developers more than any other entrant to this market. And perhaps the gaming is just the application to get people's attention.
- WrtCdEvrydy 8y ago
  The market is dead because Nvidia only want their high tier graphics cards to be deployed... can you imagine if instead of deploying 30,000 dollar GPUs, you could drop quad Titan X's or Titan V's. The datacenter license is cancer!
- heavenlyblue 8y ago
  Are you implying Google doesn’t need to maintain an expensive cloud infrastructure for it as opposed to Playstation? The only reason they can do that today without specialised hardware a-la OnLive is because Nvidia and co. now have specialised GPU circuits allowing one to stream the encoded video output to memory. When OnLive started they had to do it themselves. I don’t believe you have any idea what you are speaking about.
  - BeefySwain 8y ago
    > Nvidia and co. now have specialised GPU circuits allowing one to stream the encoded video output to memory Can you give a little more info on this? Is this standard to consumer level GPU's? What is the tech called?

kerng 8y ago

I'm already excited to see what Microsoft will show at E3 with Xcloud streaming service that they have been working on for a couple of years- lots of great progress in that area right now.

eof 8y ago

Consensus in this thread seems to be that this is a neat gimmick but nothing else. Maybe because I have a friend working on a video game as a solo developer, and putting a /lot/ of energy/money/time into art--I really see the potential in this (type of thing). This is not really for the player's benefit to put on whatever mods they want. This is for fast iteration for artists and creators to automate a huge (sometimes necessary evil) burden.

kllrnohj 8y ago
> Maybe because I have a friend working on a video game as a solo developer, and putting a /lot/ of energy/money/time into art--I really see the potential in this (type of thing). For that usage Nvidia's thing was way more interesting: https://blogs.nvidia.com/blog/2019/03/18/gaugan-photorealistic-landscapes-nvidia-research/ https://blogs.nvidia.com/blog/2019/03/18/gaugan-photorealist... That gets you usable texture assets super quickly, which is probably what your solo developer friend spends the most time on. That or models. Neither of which Stadia's style transfer helps with. They probably aren't spending much time tweaking a shader-applied post-process effect, but hey, maybe they are, and this style transfer ML thing is useful. Very, very unlikely, but maybe.
- mikewhy 8y ago
  > For that usage Nvidia's thing was way more interesting If you're building a photorealistic game. And there's already lots of tools on the market to generate these sorts of textures. I could see this Nvidia thing used for much better looking mockups / concept art. But it doesn't seem suited for final asset generation.
  - kllrnohj 8y ago
    > But it doesn't seem suited for final asset generation. Well yes, of course, and neither is Google's Style Transfer ML. These are both just prototyping tools, not a magic wand to make production-quality assets.

ohthanks 8y ago

This is not a consumer product at all. I don't know how useful it will be but it's designed to be development tool for early art direction visualization. Even for 2d visualization it could be very useful if you could drop images into your website mockup files and have it intelligently apply color palette and texture information to your existing content. It won't look good, but it might look good enough to save you from having to manually tweak the design 20-30 times to explore a range of styles.

andy_ppp 8y ago

Can't wait for ads to be inserted into games with this...

fileeditview 8y ago
A few days ago I commented on the Stadia on HN and basically predicted the same thing. Does not seem to be a popular opinion however..
- andy_ppp 8y ago
  I do find HN to be against sarcasm of any kind, which in it's own way is slightly ironic. I enjoyed your comment [1] and consequently upvoted it! [1] https://news.ycombinator.com/item?id=19433773 https://news.ycombinator.com/item?id=19433773
  - babuskov 8y ago
    That was sarcasm? I though he was serious and upvoted. I can imagine walking down the street in Call of Duty game or something like Cyberpunk 2077 and seeing ads overlaid on wall textures of buildings.
    - penagwin 8y ago
      A while ago PUBG plastered ads for their Berlin Invitational. While they were ads related to the game, the incrementally added them EVERYWHERE! First they're on a bus-stop wall, that's thematic okay. Then they're on random walls - getting annoying. Then the loading screen and the parachutes, nows it's annoying. Like, guys I'm from the US and I don't want to go to your stupid competition!

smaili 8y ago

Is anyone else concerned this will be yet another Google product that goes by the wayside in a few years? This has essentially become my biggest fear now with any new product they announce/release.

robotmay 8y ago
It's pretty much a certainty with this project, in my opinion. Until everybody somehow has gigabit lines to their home and to their phones, I don't see this becoming popular. Like most Google projects, their actual success period is most likely 10+ years away, and the project will be axed long before then.
- chatmasta 8y ago
  5G deployments have already started rolling out. 5G has average latency in the range of 1-2ms. Google controls a huge swath of the backbone after that. They can realistically achieve < 5ms latency on 5G devices in the next 1-2 years. IMO, a more valid argument against its success would be that any consumer who is in a position to ensure they have fast enough internet for this service, is also in a position to buy a gaming console. But the counterargument to that is that there is no reason someone cannot have a gaming console and also use Stadia. I'm normally very skeptical of Google's motivations, respectful of their cloud technology, and doubtful about many of their products. But I think this one will be a success, and shortly followed by competing products from Amazon and Microsoft.
  - robotmay 8y ago
    Seeing as I live just on the outskirts of a capital city (Cardiff), and I can't even get 3G signal inside my house, and no more than 80/20 broadband; I'll remain skeptical about the possibility of success for this outside of very specific cities. There's also a lot to be said for reliability and the requirement to always have a fast connection: one of the benefits to many videogames is the fact that they are not online. Isn't 5G on a wavelength that has poorer penetration than 4G? It might work in the USA but signal is a constant issue in stone housing here in the UK (though it may be better where you are based). I had to get a femtocell from EE in order to get signal in my house :\

SubiculumCode 8y ago

I've seen a number of these impressive looking demos..for example: https://www.theverge.com/2018/12/3/18121198/ai-generated-video-game-graphics-nvidia-driving-demo-neurips https://www.theverge.com/2018/12/3/18121198/ai-generated-vid... Impressive sense of realism in level of detail variety..show the potential...BUT consistency, sharpness and realism of object boundaries would not survive close inspection.

summerlight 8y ago

I don't think there will be heavy users of style transfer among high budget games anytime soon, but the important part is that it's done in real time. With an assumption that it has a reasonable level of latency, probably AA, super resolution or any other neural net based quality improvement can be a real thing.

currymj 8y ago

this is certainly just a neat gimmick, but it's interesting to me for a particular reason: the Stadia box uses an AMD GPU, and per the Google announcement, all the style transfer is done in real time. this suggests that we may soon have better AMD support in TensorFlow.

make3 8y ago
Tensorflow support on AMD is supposedly not awful anymore. https://medium.com/tensorflow/amd-rocm-gpu-support-for-tensorflow-33c78cc6a6cf https://medium.com/tensorflow/amd-rocm-gpu-support-for-tenso... I wish someone who had >actual< practical experience in the domain would comment. It's an important question.
- currymj 8y ago
  I haven't personally tried to do anything on an AMD GPU, but AMD actually has a very active GitHub repository https://github.com/ROCmSoftwarePlatform/tensorflow-upstream https://github.com/ROCmSoftwarePlatform/tensorflow-upstream . If you go through the issues you can get a sense of what it's like from people who do have practical experience.
  - make3 8y ago
    this is like bad selection bias though. people never write issues to say things are going well. the general Tensorflow issues are also about only bad things, by definition
- avinium 8y ago
  I've been following ROCm development closely, and everything I read is pointing towards it being a first class competitor to CUDA. That being said, I don't have any personal experience because, well, I don't have AMD GPUs to play around with.
Sephr 8y ago
Style transfer is most likely not running on the GPU in this case. It's probably running on Google TPUs, given that the GPUs are likely already under high load in order to play AAA games at 4K HDR 60fps.

andjd 8y ago

Did anyone else see this headline and think: Why would they use a customized XML for this in 2019?

_bxg1 8y ago

A neat tech demo, but as with so many things coming out of The Valley these days, I don't think anybody actually wants this. No actual game is so bland that you'd want the ability to overlay your own "themes" on it after the fact. AI has lots of potential for assisting artists, and maybe part of that will include post-processing effects, but putting such things in the player's hands is pointless.

b_tterc_p 8y ago

Silent hill concepts could be fun with this. Otherwise eh. I would like to see Jojo palette swaps occur to highlight tension but I don’t see that being implemented with something like this.

alkonaut 8y ago

The useful application for AI in streamed gaming seems like it would be for predicting the next 30ms to avoid input lag. This seems gimmicky at best. Prediction has to take place on the client though - so the challenge is making it cheap.

nabergh 8y ago
This is really cool to think about. In practice though I would imagine every incorrect prediction to be incredibly jarring. Additionally my intuition is that latency is easier to decrease than predicting the next 30ms. If there is some level of latency that is very difficult to reduce (more so than training an AI to predict the next 30ms with very high accuracy), I'd be curious to know why.
- alkonaut 8y ago
  Between e.g. central and west US you'd have a physical ping of ~10ms so 20 round trip just based on the speed of light (i.e. very hard to get rid of). Meanwhile the input lag you want is one or two frames so on the same order. But this lag is additive, when the game receives the new state it has to compute and render the world, the display has its lag before it shows up etc. So the new frame has to start computing as soon as the input is registered in order to hit reasonable input lag. Predicting 2-3 frames should't beyond what we can do at least roughly. Especially for some important inputs such as turning in first person view it should basically just start moving the screen. I belive John Carmack tried some kind of rotational prediction (Not based on AI) for Oculus.
chris_mc 8y ago
I am not a game developer, but couldn't you make a multiplayer game more fair in this situation by adding artificial lag into everyone's controls to match the highest lag person? Just limit it to a certain amount, like no more than 200-300ms.
- alkonaut 8y ago
  I wasn’t considering fairness here (that’s a whole different can of worms). Even for a game where everyone’s lag is the same, X milliseconds of input lag and Y milliseconds of network latency is painful for some values of X and Y. For fast paced games I’d say X is probably just a couple of frames (15-30ms) and Y is about 100ms.

deleted 8y ago

[deleted]