6 ms·

That input latency is why I hate browsers when making games. Great job though!

by Jyaif 2y ago

DonHopkins 2y ago
If you refactor and structure your code in the right way, you can get it to perform pretty well. What will kill you is stacking up layers of emulation, so you're feeding browser input into a Windows or Mac emulation layer, and even an x86 instruction set emulator, plus a useless obsolete operating system "middle ware", because the way input and animation works on old desktop guis is a lot different and much less efficient that how it works in the browser. It's certainly possible to run the original version of Mac or Windows SimCity in the browser inside a Mac or Windows emulator inside a WebAssembly module, but not only is the user interface itself terrible, klunky, and awkward, it's extremely slow and flakey because it's emulating all that obsolete operating system crap and instruction set underneath all the web browser crap between your mouse clicks and the game engine. Since the web browser is so much better and more flexible at user interface and graphics stuff, you want to totally strip all of the user interface and graphics and sound out of the game, implement an efficient API and callback mechanism (that doesn't spend a lot of time thunking and marshalling and unmarshalling parameters, and pass simple primitive data types, sending what you need all at once, instead of ping-ponging back and forth with proxy objects), and implement all of the UI in the browser (especially the animation timers and input handlers), calling back to the simulator only when necessary. One thing I did was to use shared memory between the WebAssembly module and the WebGL tile renderer, and write a custom shader that understands the native 16 bit unsigned int column major SimCity tile format, so WebGL only has to draw two triangles, and there is zero copying to draw the tiles.
- pton_xd 2y ago
  > and implement all of the UI in the browser (especially the animation timers and input handlers) I may have misunderstood you, but in my experience using the DOM for UI is a bad idea if performance is a concern. DOM updates in the browser are incredibly slow, especially for something like a game which wants to modify UI elements every frame. Meticulously avoiding relayouts, style recalculations, and other slowdowns is a constant headache. EDIT: run a few second performance profile of your site. Now do it again while waving the mouse back and forth over the UI links. Notice all the recalculate style and hit test calls that take more time than your entire WASM update! And that's with a UI that's doing nothing!
  - lukevp 2y ago
    I think you did misunderstand their point. Browser native doesn’t just mean DOM, there’s also Canvas and directly reading mouse offsets and various other approaches. What they mean is don’t transit a mouse click through an OS emulation layer through a hardware emulation layer to the simulator, just directly read input from the browser and skip all the intermediate layers and inject the value directly to the code. Imagine that the mouse is simulating a PS2 hardware device with input polling and interrupts on top of a whole simulated OS and all that crap. The approach suggested is assuming you have access to the code to modify how it gets input data, or you’re willing to update the memory directly when input happens.
  - DonHopkins 2y ago
    Since SimCity was originally designed to run on the C64, I don't think performance is too much of a concern on modern hardware, since it runs more than fast enough, and definitely wasn't intended to run at 120 frames per second running 50 simulation ticks per frame (which is what you get when you press the "9" key, and the user interface is still quite responsive on my M1 Mac). Although running it that fast is great for fast-forwarding time, but not practical for actually trying to build the city with the editing tools while it's running at such high speeds. Even 30 FPS with one tick per frame is too fast for most people to keep up with. When I developed it on a SparcStation 2 under the NeWS window system in 1991, the user interface was written completely in PostScript, which was orders of magnitude slower than JavaScript and WebAssembly, since it was interpreted, not compiled or JIT'ed or optimized in any way. But it still worked just fine, orders of magnitude faster than a C64, and could easily run at many years per second. The pie menus (which I'll implement soon for Micropolis) are able to reliably support quick gestural mouse-ahead interactions, even when the computer lags behind. And they made it a lot easier to play the game at high speeds (or when the computer is busy), because you didn't have to waste your time moving back and forth between the map and the tool palette, and you can quickly and easily change tools with swift reliable mouse-ahead gestures. When I showed it to user interface critic Don Norman, who makes a strong case for solving problems instead of focusing on technology, he actually complained about pie menus making SimCity too easy to build a city really quickly without even thinking about it. https://www.youtube.com/watch?v=5GCPQxJttf0 https://www.youtube.com/watch?v=5GCPQxJttf0 Norman: "And then when we saw SimCity, we saw how the pop-up menu that they were doing used pie menus, made it very easy to quickly select the various tools we needed to add to the streets and bulldoze out fires, and change the voting laws, etc. Somehow I thought this was a brilliant solution to the wrong problems. Yes it was much easier to now to plug in little segments of city or put wires in or bulldoze out the fires. But why were fires there in the first place? Along the way, we had a nuclear meltdown. He said "Oops! Nuclear meltdown!" and went merrily on his way." Hopkins: "Linear menus caused the meltdown. But the round menus put the fires out." Norman: "What caused the meltdown?" Hopkins: "It was the linear menus." Norman: "The linear menus?" Hopkins: "The traditional pull down menus caused the meltdown." Norman: "Don't you think a major cause of the meltdown was having a nuclear power plant in the middle of the city?" (laughter) Hopkins: "The good thing about the pie menus is that they make it really easy to build a city really fast without thinking about it." (laughter) Hopkins: "Don't laugh! I've been living in Northern Virginia!" Norman: "Ok. Isn't the whole point of SimCity how you think? The whole point of SimCity is that you learn the various complexities of controlling a city." X11 SimCity Demo -- Demo of Pie Menus in SimCity for X11. Ported to Unix and demonstrated by Don Hopkins: https://www.youtube.com/watch?v=Jvi98wVUmQA https://www.youtube.com/watch?v=Jvi98wVUmQA Even the Flash client / Python server networked version runs plenty fast over the internet, and also supports pie menus: https://www.youtube.com/watch?v=8snnqQSI0GE https://www.youtube.com/watch?v=8snnqQSI0GE Micropolis: Constructionist Educational Open Source SimCity: https://donhopkins.medium.com/har-2009-lightning-talk-transcript-constructionist-educational-open-source-simcity-by-don-3a9e010bf305 https://donhopkins.medium.com/har-2009-lightning-talk-transc... >The NeWS and X11 version both supported advanced user interface features like multiple animated maps and editors, as well as pie menus. The X11 version supported multiple player collaboration, shared text chat and drawing, and voting dialogs. And Unix workstations could run the simulation orders of magnitude faster than home computers, which transformed SimCity into a fast action twitch game! Of course it's fun to optimize just for the sake of optimization, but I think it's more interesting spending the time taking advantage of browser technologies like WebGL, HTML, CSS, Canvas, SVG, and other tools and libraries, to make a rich beautiful user interfaces with useful visualizations (like publishing analytic data into a time series database, and rendering it in tools like Grafana, or using D3 for data visualization). https://news.ycombinator.com/item?id=40065764 https://news.ycombinator.com/item?id=40065764 I want to push as much of the user interface out of the engine and into JavaScript/HTML/Canvas/SvelteKit as possible, since it's so much easier to make much better more advanced user interfaces with a modern stack than trying to deal with a crufty Windows emulation layer, and make improvements to that kind of code. One thing I want to do is to make both Micropolis and Tower able to export lots of raw and cooked data, telemetry, and events, so I can use d3, Grafana, and other off-the-shelf data visualization tools to analyze and display the game state and history. A couple decades ago an Earth Science professor Upmanu Lall at Columbia University proposed a great idea about developing an educational version of SimCity to use in his classes aimed at engaging students from other departments and getting them interested in data analysis and science. He wanted SimCity to simply be able to export spreadsheets of data, and have the students perform experiments and analyze the data using standard tools like Excel. A game like SimCity or SimTower would be an engaging way to pique the student's interest, that they could relate to, and motivate and teach a general literacy and understanding of spreadsheets and data analysis and visualization tools! Upmanu Lall: https://www.columbia.edu/~ula2/ https://www.columbia.edu/~ula2/ Here's the Educational SimCity proposal I wrote, but it didn't go anywhere until many years later when we finally talked EA into relicensing SimCity under GPL-3 for the One Laptop per Child project. https://web.archive.org/web/20050403103131/http://www.donhopkins.com/blog/categories/gameDesign/ https://web.archive.org/web/20050403103131/http://www.donhop...
wffurr 2y ago
Pointerrawupdate events: https://w3c.github.io/pointerevents/#the-pointerrawupdate-event https://w3c.github.io/pointerevents/#the-pointerrawupdate-ev... would help with that. If tearing is Ok, the canvas can be desynchronized and updated immediately. We can get 10 msec inking latency with those on a 60 Hz display.
- CyberDildonics 2y ago
  Is inking latency the new pixel to pixel latency or input to pixel latency? I like to keep up with the new names for old things.
  - tonmoy 2y ago
    I’m guessing “inking” is a autocorrect of a typo that was meant to be “input”
    - dleeftink 2y ago
      From hereon it shall be inking [0]: https://en.wikipedia.org/wiki/Inking https://en.wikipedia.org/wiki/Inking
    - wffurr 2y ago
      Actually I was referring to drawing with a stylus on a drawing app like Google Keep drawing notes or https://canvas.apps.chrome/ https://canvas.apps.chrome/.
  - modeless 2y ago
    The features he's talking about were added to Chrome for ChromeOS/Win8 tablet stylus drawing support, which they refer to as "inking". They're kind of a specific fast path for that use case. They were not tested for any other use case and as a result tend not to be usable in practice due to various caveats, unfortunately.
- modeless 2y ago
  In the past I have experimented with pointerrawupdate and desynchronized, and I was unable to reproduce any latency benefit in practice. These features have a lot of caveats and are not reliable, in my experience, which is probably why practically nobody uses them. It's way too easy to fall off the fast path, if the fast path is even supported on your platform at all. Also, native apps have a lot of options to reduce latency without any risk of tearing artifacts, and it's unfortunate that web apps don't have access to those options. The web has a long way to go to match native on the latency front and there really hasn't been meaningful progress in that direction. I would like to experiment with pointerrawupdate and desynchronized again. But I'm currently relying on Emscripten's SDL implementation for input and drawing, and replacing or modifying that is daunting. After all, this is just a fun side project for me. A much easier and more widely supported latency fix would be to use a blocking API like WebGL's ReadPixels to synchronize the content process and GPU process and defeat the deep frame pipelining that the browser usually falls into, at some cost to performance. I plan to try that first.

That input latency is why I hate browsers when making games. Great job though!

by Jyaif 2y ago

That input latency is why I hate browsers when making games. Great job though!

If you refactor and structure your code in the right way, you can get it to perform pretty well. What will kill you is stacking up layers of emulation, so you're feeding browser input into a Windows or Mac emulation layer, and even an x86 instruction set emulator, plus a useless obsolete operating system "middle ware", because the way input and animation works on old desktop guis is a lot different and much less efficient that how it works in the browser. It's certainly possible to run the original version of Mac or Windows SimCity in the browser inside a Mac or Windows emulator inside a WebAssembly module, but not only is the user interface itself terrible, klunky, and awkward, it's extremely slow and flakey because it's emulating all that obsolete operating system crap and instruction set underneath all the web browser crap between your mouse clicks and the game engine. Since the web browser is so much better and more flexible at user interface and graphics stuff, you want to totally strip all of the user interface and graphics and sound out of the game, implement an efficient API and callback mechanism (that doesn't spend a lot of time thunking and marshalling and unmarshalling parameters, and pass simple primitive data types, sending what you need all at once, instead of ping-ponging back and forth with proxy objects), and implement all of the UI in the browser (especially the animation timers and input handlers), calling back to the simulator only when necessary. One thing I did was to use shared memory between the WebAssembly module and the WebGL tile renderer, and write a custom shader that understands the native 16 bit unsigned int column major SimCity tile format, so WebGL only has to draw two triangles, and there is zero copying to draw the tiles.

pton_xd 2y ago
> and implement all of the UI in the browser (especially the animation timers and input handlers) I may have misunderstood you, but in my experience using the DOM for UI is a bad idea if performance is a concern. DOM updates in the browser are incredibly slow, especially for something like a game which wants to modify UI elements every frame. Meticulously avoiding relayouts, style recalculations, and other slowdowns is a constant headache. EDIT: run a few second performance profile of your site. Now do it again while waving the mouse back and forth over the UI links. Notice all the recalculate style and hit test calls that take more time than your entire WASM update! And that's with a UI that's doing nothing!
- lukevp 2y ago
  I think you did misunderstand their point. Browser native doesn’t just mean DOM, there’s also Canvas and directly reading mouse offsets and various other approaches. What they mean is don’t transit a mouse click through an OS emulation layer through a hardware emulation layer to the simulator, just directly read input from the browser and skip all the intermediate layers and inject the value directly to the code. Imagine that the mouse is simulating a PS2 hardware device with input polling and interrupts on top of a whole simulated OS and all that crap. The approach suggested is assuming you have access to the code to modify how it gets input data, or you’re willing to update the memory directly when input happens.
- DonHopkins 2y ago
  Since SimCity was originally designed to run on the C64, I don't think performance is too much of a concern on modern hardware, since it runs more than fast enough, and definitely wasn't intended to run at 120 frames per second running 50 simulation ticks per frame (which is what you get when you press the "9" key, and the user interface is still quite responsive on my M1 Mac). Although running it that fast is great for fast-forwarding time, but not practical for actually trying to build the city with the editing tools while it's running at such high speeds. Even 30 FPS with one tick per frame is too fast for most people to keep up with. When I developed it on a SparcStation 2 under the NeWS window system in 1991, the user interface was written completely in PostScript, which was orders of magnitude slower than JavaScript and WebAssembly, since it was interpreted, not compiled or JIT'ed or optimized in any way. But it still worked just fine, orders of magnitude faster than a C64, and could easily run at many years per second. The pie menus (which I'll implement soon for Micropolis) are able to reliably support quick gestural mouse-ahead interactions, even when the computer lags behind. And they made it a lot easier to play the game at high speeds (or when the computer is busy), because you didn't have to waste your time moving back and forth between the map and the tool palette, and you can quickly and easily change tools with swift reliable mouse-ahead gestures. When I showed it to user interface critic Don Norman, who makes a strong case for solving problems instead of focusing on technology, he actually complained about pie menus making SimCity too easy to build a city really quickly without even thinking about it. https://www.youtube.com/watch?v=5GCPQxJttf0 https://www.youtube.com/watch?v=5GCPQxJttf0 Norman: "And then when we saw SimCity, we saw how the pop-up menu that they were doing used pie menus, made it very easy to quickly select the various tools we needed to add to the streets and bulldoze out fires, and change the voting laws, etc. Somehow I thought this was a brilliant solution to the wrong problems. Yes it was much easier to now to plug in little segments of city or put wires in or bulldoze out the fires. But why were fires there in the first place? Along the way, we had a nuclear meltdown. He said "Oops! Nuclear meltdown!" and went merrily on his way." Hopkins: "Linear menus caused the meltdown. But the round menus put the fires out." Norman: "What caused the meltdown?" Hopkins: "It was the linear menus." Norman: "The linear menus?" Hopkins: "The traditional pull down menus caused the meltdown." Norman: "Don't you think a major cause of the meltdown was having a nuclear power plant in the middle of the city?" (laughter) Hopkins: "The good thing about the pie menus is that they make it really easy to build a city really fast without thinking about it." (laughter) Hopkins: "Don't laugh! I've been living in Northern Virginia!" Norman: "Ok. Isn't the whole point of SimCity how you think? The whole point of SimCity is that you learn the various complexities of controlling a city." X11 SimCity Demo -- Demo of Pie Menus in SimCity for X11. Ported to Unix and demonstrated by Don Hopkins: https://www.youtube.com/watch?v=Jvi98wVUmQA https://www.youtube.com/watch?v=Jvi98wVUmQA Even the Flash client / Python server networked version runs plenty fast over the internet, and also supports pie menus: https://www.youtube.com/watch?v=8snnqQSI0GE https://www.youtube.com/watch?v=8snnqQSI0GE Micropolis: Constructionist Educational Open Source SimCity: https://donhopkins.medium.com/har-2009-lightning-talk-transcript-constructionist-educational-open-source-simcity-by-don-3a9e010bf305 https://donhopkins.medium.com/har-2009-lightning-talk-transc... >The NeWS and X11 version both supported advanced user interface features like multiple animated maps and editors, as well as pie menus. The X11 version supported multiple player collaboration, shared text chat and drawing, and voting dialogs. And Unix workstations could run the simulation orders of magnitude faster than home computers, which transformed SimCity into a fast action twitch game! Of course it's fun to optimize just for the sake of optimization, but I think it's more interesting spending the time taking advantage of browser technologies like WebGL, HTML, CSS, Canvas, SVG, and other tools and libraries, to make a rich beautiful user interfaces with useful visualizations (like publishing analytic data into a time series database, and rendering it in tools like Grafana, or using D3 for data visualization). https://news.ycombinator.com/item?id=40065764 https://news.ycombinator.com/item?id=40065764 I want to push as much of the user interface out of the engine and into JavaScript/HTML/Canvas/SvelteKit as possible, since it's so much easier to make much better more advanced user interfaces with a modern stack than trying to deal with a crufty Windows emulation layer, and make improvements to that kind of code. One thing I want to do is to make both Micropolis and Tower able to export lots of raw and cooked data, telemetry, and events, so I can use d3, Grafana, and other off-the-shelf data visualization tools to analyze and display the game state and history. A couple decades ago an Earth Science professor Upmanu Lall at Columbia University proposed a great idea about developing an educational version of SimCity to use in his classes aimed at engaging students from other departments and getting them interested in data analysis and science. He wanted SimCity to simply be able to export spreadsheets of data, and have the students perform experiments and analyze the data using standard tools like Excel. A game like SimCity or SimTower would be an engaging way to pique the student's interest, that they could relate to, and motivate and teach a general literacy and understanding of spreadsheets and data analysis and visualization tools! Upmanu Lall: https://www.columbia.edu/~ula2/ https://www.columbia.edu/~ula2/ Here's the Educational SimCity proposal I wrote, but it didn't go anywhere until many years later when we finally talked EA into relicensing SimCity under GPL-3 for the One Laptop per Child project. https://web.archive.org/web/20050403103131/http://www.donhopkins.com/blog/categories/gameDesign/ https://web.archive.org/web/20050403103131/http://www.donhop...

wffurr 2y ago

Pointerrawupdate events: https://w3c.github.io/pointerevents/#the-pointerrawupdate-event https://w3c.github.io/pointerevents/#the-pointerrawupdate-ev... would help with that. If tearing is Ok, the canvas can be desynchronized and updated immediately. We can get 10 msec inking latency with those on a 60 Hz display.

CyberDildonics 2y ago
Is inking latency the new pixel to pixel latency or input to pixel latency? I like to keep up with the new names for old things.
- tonmoy 2y ago
  I’m guessing “inking” is a autocorrect of a typo that was meant to be “input”
  - dleeftink 2y ago
    From hereon it shall be inking [0]: https://en.wikipedia.org/wiki/Inking https://en.wikipedia.org/wiki/Inking
  - wffurr 2y ago
    Actually I was referring to drawing with a stylus on a drawing app like Google Keep drawing notes or https://canvas.apps.chrome/ https://canvas.apps.chrome/.
- modeless 2y ago
  The features he's talking about were added to Chrome for ChromeOS/Win8 tablet stylus drawing support, which they refer to as "inking". They're kind of a specific fast path for that use case. They were not tested for any other use case and as a result tend not to be usable in practice due to various caveats, unfortunately.
modeless 2y ago
In the past I have experimented with pointerrawupdate and desynchronized, and I was unable to reproduce any latency benefit in practice. These features have a lot of caveats and are not reliable, in my experience, which is probably why practically nobody uses them. It's way too easy to fall off the fast path, if the fast path is even supported on your platform at all. Also, native apps have a lot of options to reduce latency without any risk of tearing artifacts, and it's unfortunate that web apps don't have access to those options. The web has a long way to go to match native on the latency front and there really hasn't been meaningful progress in that direction. I would like to experiment with pointerrawupdate and desynchronized again. But I'm currently relying on Emscripten's SDL implementation for input and drawing, and replacing or modifying that is daunting. After all, this is just a fun side project for me. A much easier and more widely supported latency fix would be to use a blocking API like WebGL's ReadPixels to synchronize the content process and GPU process and defeat the deep frame pipelining that the browser usually falls into, at some cost to performance. I plan to try that first.