5 ms·
For me the main advantage of WebXR is that one can create complex 3D objects at runtime/on the fly without havjbg to load them as assets or compiling them into
by hoc 3y ago
For me the main advantage of WebXR is that one can create complex 3D objects at runtime/on the fly without havjbg to load them as assets or compiling them into the binary.
Any thougts on the best way to do this in classic runtimes like Unity or Unreal (which scripting plugins, which base construction objects/assets?)?
The main issue I have with WebXR, though, is it being canvas-based instead of being page/HTML-based, resulting in graphic-oriented products instead of structure-oriented ones (e.g. 3D web pages) which would allow placing all the dynamic and complex HTML objects that were developed over the last few years. Any good known approach to solve this?
- andybak 3y agoNot sure what you're asking, exactly? Unity (and I presume Unreal) can do the same procedural generation stuff today your can do in js. There's many libraries/tutorials etc
- hoc 3y agoBasically i'd like to add, edit, store and load code during runtime.
- andybak 3y agoOh, code? I assumed you meant geometry and/or materials. Yes, it's possible. I can think of several ways to approach it. Not quite as cleanly as with javaScript, mind.
- hoc 3y agoHeh. What would be your favourite thoughts/combo here? Sorry for not being clear on the code part. I assumed USD and the like would be the norm now. "Complex objects" is not really helpful in this context, though.
- andybak 3y agoUSD is fairly widely used. But it's not "code" in the sense that I mean it. It's (mostly) declarative (like HTML) so I lump it in with obj, fbx and gltf. I am assuming code means "executable code" - I know the boundaries are fuzzy here and some file formats are Turing complete - but I guess what I'm saying is that I'm still not sure what you're asking!
- hoc 3y agoCode that I can create during runtime. Ideally interactively.
- Uehreka 3y agoJust a heads up, when folks say “complex 3d objects”, it’s usually assumed that they’re talking about things like GLTF files, which contain geometry, materials, lighting elements, animations, etc, and which Unity/Unreal can absolutely load/unload at runtime or even stream from a media server online (so they don’t have to be in the binary). However, once you cross over into scripting code, you are in fact crossing a barrier, and it does become harder to hot-reload the way you can in a JS app. Like, the fact that Unity scripts are written in C# and have to be compiled and linked is a pretty huge impediment that you don’t face when loading almost any other kind of asset. I’m right there with you: Hot-reloading and 3D is a thrilling combination, and tools like react-three-fiber are miles ahead of anything I’ve seen in Unity-land in terms of laying foundations for stuff like this. I’d recommend emphasizing “hot-reloading code” when drawing this distinction with Unity/Unreal folks to better get across the magnitude of what you’re looking for.
- hoc 3y agoThanks for both hints. I recently had more contact with gaming engines than I originally wanted and there's always this kind of community-specific vocabulary whenever you cross borders. Oh no,like in real life.
- gregfagan 3y agoI wrote up a post on my R3F based WebXR hot reloading setup here: https://github.com/gregfagan/blog/blob/main/xr-remote-display/README.md https://github.com/gregfagan/blog/blob/main/xr-remote-displa... You don’t actually need this if you’re just working on graphics, as usually a flat projection is fine, but it is quite nice to work inside your own XR app while iterating on the interaction.
- hoc 3y agoNice. I also started looking into possible WebRTC sources and servers since at least this could be integrated quite well into WebXR. Unfortunately I find myself cycling thru needed components for interactivity these day and being stuck between competing SW and HW constraints. Will give it a closer read next week.
- WesSouza 3y agoCSS supports many 3D transforms, I wonder if VR browsers implement that in any sort of 3D plane.
- hoc 3y agoCool. If I could xr-dive into a css-3d-transformed world of objects instead of having them projected onto the 2d window right away, that would be what I was looking for in the second part of the comment above. Not sure if WebXR perspective and a 3d space is provided right now at the same level these css transforms are used.
- Arathorn 3y agoyou can calculate 2D CSS transforms which match the equivalent transforms of your WebGL scene in WebXR - as an efficient but hacky way to (for instance) do live video overlays in 3D without having to mess around importing the video texture into WebGL (assuming you don’t need occlusion or environmental effects etc). we’re toying with this as an approach for video overlays in https://thirdroom.io https://thirdroom.io, especially for underpowered devices.
- hoc 3y agoYes, I've seen similar examples. Reminded me of HW video card overlays and I hoped we would have a better solution soon. But currently probably it what's there. Would this work as well for other HTML content, like dynamically projecting a whole DIV into/over a canvas scene that I would then be able to have positioned over some surface in VR? Love the product/concept, btw.
- ImHereToVote 3y agoI don't think people realize how dog shit slow the DOM is. There is a reason webXR uses the canvas.
- doctorpangloss 3y agoYou can stream it. Here’s an example using Unity: https://appmana.com/watch/virtualtestdrive https://appmana.com/watch/virtualtestdrive For runtime loading in such an architecture, you can using an asset like TriLib. There are many streaming ideas. Most were poorly executed. I wouldn’t generalize based on those. It is definitely the future of delivery for many reasons besides technology.
- hoc 3y agoYes. I'd love to load and re-use existing js libs as well as add code to a project and rules at runtime. I'd have loved to see WebXR being an extension of HTML instead of it being an extension to the canvas.
- jayd16 3y agoFor Unity, I think you can just dynamically load DLLs as long as you're not on a plane form that requires AOT (iOS). Even then, for both Unreal and Unity there are many options to use scripting languages like js or lua or what have you.
- hoc 3y agoOk. will look into scripting plugins then for now, probably bridging some introspected interfaces. Would be nice if one could add/integrate existing JS libs but that's outside the Unity context.
- andybak 3y agoI'm keeping a close eye on WASM as a way to hotload code in Unity and as a way to access libraries from a variety of ecosystems.
- hoc 3y agoGood point. So many great exec environments already for WASM. Is there already something available for Unity? Maybe even with a source language and compiler built in? My focus currently is on creating code inside a running app (by the user) with references to selected Unity objects and their methods. While WebXR is great for this, the first (and sometimes only) SDK implementations for certain devices are usually in Unity. WASM probably ist great for hotloading but would need a more complex toolset available at runtime to edit and create. Interactivity and incremental code creation would also be better in a scripting envirnment.
- andybak 3y agoI went with Lua personally. I have mixed feelings about some aspects of the language but the ease of integration was hard to fault. I'm using Moonsharp which is interpreted rather than jitted but Blua looks promising if performance on the Lua side is critical.