7 ms·

Window.js is an open-source JavaScript runtime for desktop graphics programming

163 points by EntICOnc 5y ago

udbhavs 5y ago
Implementing an identical canvas API on skia is an interesting idea. I wonder how the performance is compared to regular web canvas.
- joaodasilvaz 5y ago
  Note that Chrome also builds on Skia, via the Blink rendering engine. So Skia is very well aligned with the needs of the Canvas API. As for performance, Window.js has a bit less overhead since it doesn't have a DOM and doesn't implement web security. On the other hand, Chrome is widely used and has a super talented team that knows systems programming, GPU programming, etc in a lot of depth and have spent a ton of time optimizing web APIs. So I wouldn't be surprised if Chrome and Electron have better performance in benchmarks -- this is definitely something to measure! As an example, this p5js example runs very slowly on Window.js: https://p5js.org/examples/simulate-smokeparticles.html https://p5js.org/examples/simulate-smokeparticles.html The reason is that it's doing something like a getPixel(x, y) call in a loop, which Chrome somehow optimizes; in Window.js this is going to be a roundtrip to the GPU for each pixel, which is extremely expensive. (IMO, the right way to implement getPixel based on the Canvas API would be to read all the pixels once via canvas.getImageData)
  - Matheus28 5y ago
    I don't remember very much, but have you tried making a SkImage snapshot on the first call to getImageData (and calls to drawImage that use that canvas), and just reusing that? I think it might do its own caching when you read pixels from it Also invalidate the snapshot on any calls that invalidate the contents of the snapshot of course (stroke, fill, drawImage, fillRect, strokeRect... probably a couple others). I have also reimplemented the canvas 2d api using skia for my own stuff.
    - Matheus28 5y ago
      Ah also I think you don't need to flush before makeImageSnapshot
gnabgib 5y ago
The documentation website: https://windowjs.org/ https://windowjs.org/ is quite a bit more helpful (especially since almost all of the github links are to that domain). But that would be a dupe of https://news.ycombinator.com/item?id=29865107 https://news.ycombinator.com/item?id=29865107 (2 days ago, 2pts, 0 comments)
nevir 5y ago
Awesome to see projects like this! Browsers have a large amount of overlap with operating systems - and with efforts to move more of web tech into userland (web components/etc all starting to rationalize HTML and other layers as being built on top of other more fundamental web APIs, etc)
joaodasilvaz 5y ago
Hi HN, author of windowjs here. I was planning to have the project in a more complete state before announcing to HN :-) It was mostly a fun project to put together v8, GLFW and Skia, and I hope others find it useful and fun to work on too. If you find this interesting then consider getting in touch via GitHub Discussions or the mailing list and helping out with the next features: WebGL 2 API, networking (fetch) API, sound, etc. I don't have much experience collaborating in opensource projects so any feedback is very welcome. Thanks for having a look!
- tylerchilds 5y ago
  Very cool! I'm a web dev and I've wanted to get into game dev. From my cursory glance, this seems like exactly what I'd like to build on top of: just the essentials for a performant interactive application. My one question: Would it be possible to integrate with SDL2? https://www.libsdl.org/ https://www.libsdl.org/
  - joaodasilvaz 5y ago
    libsdl and GLFW are very similar! I ended up using GLFW because it's closer to what I wanted to do with Skia and WebGL, and I didn't need the 2D functionality of SDL. Note that Window.js does not expose GLFW directly; it exposes APIs that are more similar to the web, for familiarity to web devs. For example, GLFW has this for mouse input: https://www.glfw.org/docs/latest/input_guide.html#input_mouse_button https://www.glfw.org/docs/latest/input_guide.html#input_mous... But Window.js wraps that in window.addEventListener and "mousedown" events: https://windowjs.org/doc/window#window.addEventListener https://windowjs.org/doc/window#window.addEventListener https://windowjs.org/doc/window#event-mousedown https://windowjs.org/doc/window#event-mousedown If you give it a try to build a game then keep in mind that it doesn't have a sound API yet, though that's part of the future plans. Finally, is there something specific in libsdl that you'd like to have and Window.js doesn't support?
    - tylerchilds 5y ago
      Awesome! I hadn't heard of GLFW before. My main use case is wanting gamepad support which it looks like GLFW has. From all my user research, it looks like nothing beats up, down, left, right, confirmation and cancellation actions, so that's all I really need. Are those gamepad events accessible through addEventListener?
AshleysBrain 5y ago
It looks like a cool project, but... why not use lightweight webviews (WebView2/WKWebView)? Or even the heavyweight Electron/NW.js? These brings you full web API compatibility, extensibility points, and webviews have a small footprint too. What does this project aim to do differently? I don't mean to criticise a hobby project if that's what this is, it's all well and good, I'm just wondering what the thinking is.
- joaodasilvaz 5y ago
  Electron is a lot more feature-complete and mature, and that's what should be used for any serious projects indeed :-) Window.js is a subset; it doesn't include the DOM nor the CSS APIs, nor a lot of web APIs. Window.js is a tool I wish I had for exploratory 2D (and 3D) programming, without having to install multiple tools, SDKs and runtimes, and without the restrictions that web pages have when it comes to desktop applications. I wanted something like Processing but with more control, or like <canvas> but without all the browser cruft around it, and in a native window. I think it could be a very useful tool to learn WebGL via webglfundamental.org, for example, and having something that runs on the desktop quickly. A goal of Window.js is to eventually run Processing (p5.js), Pixi.js, Phaser and Threejs programs, as long as they don't use the DOM too much.
  - softfalcon 5y ago
    I am curious how you intend to get ThreeJS (or similar) working without a canvas DOM element? Isn’t that a lot of work? Or is there an obvious way to hook it up that I’m missing?
    - joaodasilvaz 5y ago
      Three.js should load and run as long as its usage of window, document, etc works as it expects. So the plan is to have a small loader that prepares the environment that Three.js expects before loading it. This has already been done for Processing/p5.js: https://windowjs.org/about/processing https://windowjs.org/about/processing After the initial load, all of the p5.js APIs make CanvasRendering2DContext calls, which Window.js implements natively. Almost all of the p5.js examples work with this approach, except those that use DOM elements like buttons and sliders. Those elements could also be faked, with some patience :-) I think it's pretty cool that p5.js scripts can run on a desktop window this way, outside of the browser! As an aside, I also originally intended to implement an HTML renderer on something like Window.js, and make a "browser from scratch" based on it. Maybe for another day...
deleted 5y ago
[deleted]
evmar 5y ago
How does this compare to http://www.plask.org/ http://www.plask.org/ ?
- joaodasilvaz 5y ago
  From a quick look at plask.org: * seems like Plask is only for macOS (Window.js also supports Windows and Linux) * Plask exposes the Skia APIs directly to Javascript; Window.js exposes a <canvas>-alike API instead (but also based on Skia) * Plask supports video playback with AVPlayer; Window.js doesn't have video support (for now :-)
easrng 5y ago
This is really cool! Have you thought about exposing a11y APIs?
- joaodasilvaz 5y ago
  What kind of APIs do you have in mind? Ideally yes, Window.js should make it possible (and easy!) to build fully accessible programs. I don't have much experience in this area though, so I'd have to look for help building this :-)
  - easrng 5y ago
    There's AccessKit which would work great if we were a few years in the future and they'd finished their bindings and non-windows implementations. Other than that I don't know of anything cross-platform other than Qt which is rather large.
T3RMINATED 5y ago
deleted 5y ago
[deleted]
marwis 5y ago
Can this be ported to node as a module?
- joaodasilvaz 5y ago
  Not quite, as this is a native program and not a Javascript module. Window.js is like Node itself, but has different APIs. The main difference is that it gives you a desktop window and the Canvas 2D API, and doesn't have any of Node's networking APIs. Other differences to Node: * every source is imported to Window.js as an ES6 module. * Window.js does not support NPM and doesn't aim to. * Node uses callbacks for async APIs, Window.js uses Promises * Node is widely tested and used, Window.js is quite new and experimental :-) Other projects have built bindings for GLFW for Node. Have a look at the GLFW documentation and search for "node": https://www.glfw.org/community.html#bindings https://www.glfw.org/community.html#bindings
deleted 5y ago
[deleted]
noduerme 5y ago
I'm really glad you mentioned PixiJS - that would be a very compelling use case for me! I formerly built a lot of casual 2D / 2.5D games in AIR. Pixi is my favorite screen graph library now by far. The biggest thing I think the JS/canvas/WebGL ecosystem is really lacking, sorry to say, is the ability to compile to .ipa and .apk, with something like AIR Native Extensions (which worked along the lines of Phonegap plugins to centralize access to Android and iOS APIs in a common library). Cordova is dead and there's no replacement. And Cordova was never really meant for writing games. Something like this with WebGL support that just opened a full screen phone app running PixiJS - even with just minimal connectivity APIs - would be a great option for quickly building indie games.
- melony 5y ago
  React native has Skia support https://shopify.github.io/react-native-skia/ https://shopify.github.io/react-native-skia/
  - noduerme 5y ago
    Yeah... I looked into that a little, but I've always avoided React like the plague. Looking at JSX just drives me up a wall, and I'm not into the idea of the React Native pseudo-DOM. Oddly it feels a little like Flex/MXML which also annoyed me to no end.
apatheticonion 5y ago
After seeing how pretty Windows 11 applications are, I've been trying to create Windows desktop applications using Go or Rust but I have no idea what the Windows GUI API looks like and am unable to find/create bindings. Linux's GTK4 is pretty easy to use, it's like vanilla JS and it's easy to get started. Microsoft made it really hard to get started, and I gave up not long after I started.
- EMM_386 5y ago
  You will likely need to use the Windows UI Library (WinUI). Documentation is here: https://docs.microsoft.com/en-us/windows/apps/winui/winui2/ https://docs.microsoft.com/en-us/windows/apps/winui/winui2/
  - apatheticonion 5y ago
    Thanks! Off I go again!
  - starik36 5y ago
    I think WinUI2 is either deprecated or soon will be. There is WinUI 3. https://docs.microsoft.com/en-us/windows/apps/winui/winui3/ https://docs.microsoft.com/en-us/windows/apps/winui/winui3/
    - EMM_386 5y ago
      Correct, noted on the top of the doc page I sent with links to it .... but yes, 3.0.
- lostmsu 5y ago
  The best place to start for you is https://github.com/microsoft/windows-samples-rs https://github.com/microsoft/windows-samples-rs UI example is: https://github.com/microsoft/windows-samples-rs/tree/master/xaml_app https://github.com/microsoft/windows-samples-rs/tree/master/...
apineda 5y ago
This is cool. I'm writing something in three / electron presently.
DonnyV 5y ago
I think including a BabylonJS sample would work really well with this. BabylonJS does everything in canvas. https://www.babylonjs.com/ https://www.babylonjs.com/

163 points by EntICOnc 5y ago

Implementing an identical canvas API on skia is an interesting idea. I wonder how the performance is compared to regular web canvas.

joaodasilvaz 5y ago
Note that Chrome also builds on Skia, via the Blink rendering engine. So Skia is very well aligned with the needs of the Canvas API. As for performance, Window.js has a bit less overhead since it doesn't have a DOM and doesn't implement web security. On the other hand, Chrome is widely used and has a super talented team that knows systems programming, GPU programming, etc in a lot of depth and have spent a ton of time optimizing web APIs. So I wouldn't be surprised if Chrome and Electron have better performance in benchmarks -- this is definitely something to measure! As an example, this p5js example runs very slowly on Window.js: https://p5js.org/examples/simulate-smokeparticles.html https://p5js.org/examples/simulate-smokeparticles.html The reason is that it's doing something like a getPixel(x, y) call in a loop, which Chrome somehow optimizes; in Window.js this is going to be a roundtrip to the GPU for each pixel, which is extremely expensive. (IMO, the right way to implement getPixel based on the Canvas API would be to read all the pixels once via canvas.getImageData)
- Matheus28 5y ago
  I don't remember very much, but have you tried making a SkImage snapshot on the first call to getImageData (and calls to drawImage that use that canvas), and just reusing that? I think it might do its own caching when you read pixels from it Also invalidate the snapshot on any calls that invalidate the contents of the snapshot of course (stroke, fill, drawImage, fillRect, strokeRect... probably a couple others). I have also reimplemented the canvas 2d api using skia for my own stuff.
  - Matheus28 5y ago
    Ah also I think you don't need to flush before makeImageSnapshot

gnabgib 5y ago

The documentation website: https://windowjs.org/ https://windowjs.org/ is quite a bit more helpful (especially since almost all of the github links are to that domain). But that would be a dupe of https://news.ycombinator.com/item?id=29865107 https://news.ycombinator.com/item?id=29865107 (2 days ago, 2pts, 0 comments)

nevir 5y ago

Awesome to see projects like this! Browsers have a large amount of overlap with operating systems - and with efforts to move more of web tech into userland (web components/etc all starting to rationalize HTML and other layers as being built on top of other more fundamental web APIs, etc)

joaodasilvaz 5y ago

Hi HN, author of windowjs here. I was planning to have the project in a more complete state before announcing to HN :-) It was mostly a fun project to put together v8, GLFW and Skia, and I hope others find it useful and fun to work on too. If you find this interesting then consider getting in touch via GitHub Discussions or the mailing list and helping out with the next features: WebGL 2 API, networking (fetch) API, sound, etc. I don't have much experience collaborating in opensource projects so any feedback is very welcome. Thanks for having a look!

tylerchilds 5y ago
Very cool! I'm a web dev and I've wanted to get into game dev. From my cursory glance, this seems like exactly what I'd like to build on top of: just the essentials for a performant interactive application. My one question: Would it be possible to integrate with SDL2? https://www.libsdl.org/ https://www.libsdl.org/
- joaodasilvaz 5y ago
  libsdl and GLFW are very similar! I ended up using GLFW because it's closer to what I wanted to do with Skia and WebGL, and I didn't need the 2D functionality of SDL. Note that Window.js does not expose GLFW directly; it exposes APIs that are more similar to the web, for familiarity to web devs. For example, GLFW has this for mouse input: https://www.glfw.org/docs/latest/input_guide.html#input_mouse_button https://www.glfw.org/docs/latest/input_guide.html#input_mous... But Window.js wraps that in window.addEventListener and "mousedown" events: https://windowjs.org/doc/window#window.addEventListener https://windowjs.org/doc/window#window.addEventListener https://windowjs.org/doc/window#event-mousedown https://windowjs.org/doc/window#event-mousedown If you give it a try to build a game then keep in mind that it doesn't have a sound API yet, though that's part of the future plans. Finally, is there something specific in libsdl that you'd like to have and Window.js doesn't support?
  - tylerchilds 5y ago
    Awesome! I hadn't heard of GLFW before. My main use case is wanting gamepad support which it looks like GLFW has. From all my user research, it looks like nothing beats up, down, left, right, confirmation and cancellation actions, so that's all I really need. Are those gamepad events accessible through addEventListener?

AshleysBrain 5y ago

It looks like a cool project, but... why not use lightweight webviews (WebView2/WKWebView)? Or even the heavyweight Electron/NW.js? These brings you full web API compatibility, extensibility points, and webviews have a small footprint too. What does this project aim to do differently? I don't mean to criticise a hobby project if that's what this is, it's all well and good, I'm just wondering what the thinking is.

joaodasilvaz 5y ago
Electron is a lot more feature-complete and mature, and that's what should be used for any serious projects indeed :-) Window.js is a subset; it doesn't include the DOM nor the CSS APIs, nor a lot of web APIs. Window.js is a tool I wish I had for exploratory 2D (and 3D) programming, without having to install multiple tools, SDKs and runtimes, and without the restrictions that web pages have when it comes to desktop applications. I wanted something like Processing but with more control, or like <canvas> but without all the browser cruft around it, and in a native window. I think it could be a very useful tool to learn WebGL via webglfundamental.org, for example, and having something that runs on the desktop quickly. A goal of Window.js is to eventually run Processing (p5.js), Pixi.js, Phaser and Threejs programs, as long as they don't use the DOM too much.
- softfalcon 5y ago
  I am curious how you intend to get ThreeJS (or similar) working without a canvas DOM element? Isn’t that a lot of work? Or is there an obvious way to hook it up that I’m missing?
  - joaodasilvaz 5y ago
    Three.js should load and run as long as its usage of window, document, etc works as it expects. So the plan is to have a small loader that prepares the environment that Three.js expects before loading it. This has already been done for Processing/p5.js: https://windowjs.org/about/processing https://windowjs.org/about/processing After the initial load, all of the p5.js APIs make CanvasRendering2DContext calls, which Window.js implements natively. Almost all of the p5.js examples work with this approach, except those that use DOM elements like buttons and sliders. Those elements could also be faked, with some patience :-) I think it's pretty cool that p5.js scripts can run on a desktop window this way, outside of the browser! As an aside, I also originally intended to implement an HTML renderer on something like Window.js, and make a "browser from scratch" based on it. Maybe for another day...

deleted 5y ago

[deleted]

evmar 5y ago

How does this compare to http://www.plask.org/ http://www.plask.org/ ?

joaodasilvaz 5y ago
From a quick look at plask.org: * seems like Plask is only for macOS (Window.js also supports Windows and Linux) * Plask exposes the Skia APIs directly to Javascript; Window.js exposes a <canvas>-alike API instead (but also based on Skia) * Plask supports video playback with AVPlayer; Window.js doesn't have video support (for now :-)

easrng 5y ago

This is really cool! Have you thought about exposing a11y APIs?

joaodasilvaz 5y ago
What kind of APIs do you have in mind? Ideally yes, Window.js should make it possible (and easy!) to build fully accessible programs. I don't have much experience in this area though, so I'd have to look for help building this :-)
- easrng 5y ago
  There's AccessKit which would work great if we were a few years in the future and they'd finished their bindings and non-windows implementations. Other than that I don't know of anything cross-platform other than Qt which is rather large.

T3RMINATED 5y ago

deleted 5y ago

[deleted]

marwis 5y ago

Can this be ported to node as a module?

joaodasilvaz 5y ago
Not quite, as this is a native program and not a Javascript module. Window.js is like Node itself, but has different APIs. The main difference is that it gives you a desktop window and the Canvas 2D API, and doesn't have any of Node's networking APIs. Other differences to Node: * every source is imported to Window.js as an ES6 module. * Window.js does not support NPM and doesn't aim to. * Node uses callbacks for async APIs, Window.js uses Promises * Node is widely tested and used, Window.js is quite new and experimental :-) Other projects have built bindings for GLFW for Node. Have a look at the GLFW documentation and search for "node": https://www.glfw.org/community.html#bindings https://www.glfw.org/community.html#bindings

deleted 5y ago

[deleted]

noduerme 5y ago

I'm really glad you mentioned PixiJS - that would be a very compelling use case for me! I formerly built a lot of casual 2D / 2.5D games in AIR. Pixi is my favorite screen graph library now by far. The biggest thing I think the JS/canvas/WebGL ecosystem is really lacking, sorry to say, is the ability to compile to .ipa and .apk, with something like AIR Native Extensions (which worked along the lines of Phonegap plugins to centralize access to Android and iOS APIs in a common library). Cordova is dead and there's no replacement. And Cordova was never really meant for writing games. Something like this with WebGL support that just opened a full screen phone app running PixiJS - even with just minimal connectivity APIs - would be a great option for quickly building indie games.

melony 5y ago
React native has Skia support https://shopify.github.io/react-native-skia/ https://shopify.github.io/react-native-skia/
- noduerme 5y ago
  Yeah... I looked into that a little, but I've always avoided React like the plague. Looking at JSX just drives me up a wall, and I'm not into the idea of the React Native pseudo-DOM. Oddly it feels a little like Flex/MXML which also annoyed me to no end.

apatheticonion 5y ago

After seeing how pretty Windows 11 applications are, I've been trying to create Windows desktop applications using Go or Rust but I have no idea what the Windows GUI API looks like and am unable to find/create bindings. Linux's GTK4 is pretty easy to use, it's like vanilla JS and it's easy to get started. Microsoft made it really hard to get started, and I gave up not long after I started.

EMM_386 5y ago
You will likely need to use the Windows UI Library (WinUI). Documentation is here: https://docs.microsoft.com/en-us/windows/apps/winui/winui2/ https://docs.microsoft.com/en-us/windows/apps/winui/winui2/
- apatheticonion 5y ago
  Thanks! Off I go again!
- starik36 5y ago
  I think WinUI2 is either deprecated or soon will be. There is WinUI 3. https://docs.microsoft.com/en-us/windows/apps/winui/winui3/ https://docs.microsoft.com/en-us/windows/apps/winui/winui3/
  - EMM_386 5y ago
    Correct, noted on the top of the doc page I sent with links to it .... but yes, 3.0.
lostmsu 5y ago
The best place to start for you is https://github.com/microsoft/windows-samples-rs https://github.com/microsoft/windows-samples-rs UI example is: https://github.com/microsoft/windows-samples-rs/tree/master/xaml_app https://github.com/microsoft/windows-samples-rs/tree/master/...

apineda 5y ago

This is cool. I'm writing something in three / electron presently.

DonnyV 5y ago

I think including a BabylonJS sample would work really well with this. BabylonJS does everything in canvas. https://www.babylonjs.com/ https://www.babylonjs.com/