8 ms·

Here's a thought I had watching the bit with 2D man: Actually, the 3D view that 2D man does not understand but which we do understand is... still 2D. My screen

Here's a thought I had watching the bit with 2D man: Actually, the 3D view that 2D man does not understand but which we do understand is... still 2D. My screen is flat. You can use a 2D viewport to render a 3D scene in a way that is natural and easy to understand for us humans: A human watching the 2D scene can very quickly surmise from a glance at the viewport: Which objects are in the scene, and where are they located, in _ALL_ 3 dimensions? This raises the question: Can you render a 4D scene onto a 3D viewport such that us humans are pretty good at understanding where every object in the scene is, in all 4 dimensions? I assume the answer is 'yeah, you can do that'. I wonder what that would look like. It's complicated of course; our eyeballs are involved and they kinda work in 2D and not 3D; where we humans can casually glance at a 2D viewport rendering a 3D scene for a second and know what's happening, we'd have to walk around the 3D screen rendering a 4D scene in order to even see everything.

anotheryou 8y ago
I asked the same question a few seconds before you :) found this: https://youtu.be/S-yRYmdsnGs?t=252 https://youtu.be/S-yRYmdsnGs?t=252 and even better, this: https://youtu.be/dy_MUfBuq2I https://youtu.be/dy_MUfBuq2I
- dm319 8y ago
  Yes, and I think this just shows how a 3d representation on 2d gives our brain the _illusion_ of 3d. This doesn't work for 4d, because our brain doesn't have an internal understanding of 4d objects.
  - anotheryou 8y ago
    I think once we have it interactively our brain will adapt.
    - ianai 8y ago
      It’s t least worth a shot.
  - notyourtypical 8y ago
    >our brain doesn't have an internal understanding of 4d objects I agree and I think it will never have. We can certainly speculate but ultimately it'd be like an animal with monochromatic vision trying to see colours.
joshumax 8y ago
Unfortunately, it's not as simple as that (although I wish it were). While you could generate a 3D perspective on each 2D "slice" of a 3D viewport, it wouldn't be that much more different than what's going on in here. The main issue arises from trying to render a 4-polytope in a universe where we ourselves can only sense 3 dimensions "natively."
- myfonj 8y ago
  Sad thing is that what we sense (see) is in fact just a poor projection into 2D (aka perspective) with some largely learnt concept of depth. If only we could sense space entirely with nothing "behind", not just from single point...
  - ozzmotik 8y ago
    i think it's great that you represent it as a projection because that is exactly what it is: a representation of another, perhaps more "objective" (though I'm not sure at that level that attributes like that apply) data set. I think it's super easy for us to assume what we visually perceive has a 1:1 correspondence with the intrinsic nature of the data it is constructed from. and i think it's also important to realize that that construct is also informed by other avenues of sensory input and that extra context tends to take it even further away from the nature of the raw data it is constructed from
    - myfonj 8y ago
      Perhaps it is determined by the fact senses are bound to body and body is limitary. Our 2d vision could grasp entire Flatland (if it fitted into single view), but sight of similarly finite Flatland inhabitant could grasp just his immediate surroundings, i.e. the sum of closest opaque objects in his 1D sight.
MAXPOOL 8y ago
Yes. there are many ways to do 4d to 3d projections, including perspective projections. You could also make a 3d 'camera' view with 4d lens and 3d projection plane. Here is a good intro to 4D Visualization http://eusebeia.dyndns.org/4d/vis/vis http://eusebeia.dyndns.org/4d/vis/vis
simias 8y ago
The problem is that while everything you see on your (2D) monitor is effectively 2D our brain is good at taking 3D cues (shadows, perspective, parallax etc...) to reconstitute 3D "data" from it. For instance look at 3D rotating cube on your monitor: https://www.youtube.com/watch?v=5E10zYKln3g https://www.youtube.com/watch?v=5E10zYKln3g Even though there's no shading, a bad resolution and zero context anchoring it in a familiar setting you immediately see it as a 3D object when effectively that's just 3 2D quadrilaterals changing colors and shape. We really need very little to be able to extrapolate depth information from a very limited 2D-only display. If you did that with 4D cues in 3D space (say in VR or something) I think it wouldn't work. It'd just look like a 3D object changing shape, not a projection of a higher dimension (the same way an hypothetical 2D being would see 3 quadrilaterals changing shape in the above video, not a solid 3D object that would be an abstract concept to them, not a familiar reality). Could we teach our brain to "think in 4D" and interpret these cues differently? To a certain extent probably, maybe if you spent hours and hours and hours in a VR simulation with 4D objects you'd start getting a feel for it. I doubt you'd ever get as good as with 3D objects since we've been exposed to these literally every waking moment since birth, but maybe I underestimate the plasticity of our brain. That actually leads me to an other question: is 3D somewhat hardcoded in our brains or is it purely learned? If we were mad scientists and took a baby brain into a 4D "plato's cave" style simulation, could it grow into being able to perceive 4D as intuitively and effectively as we do with 3D space? Also unrelated question: does anybody have a baby I could borrow?
- scrollaway 8y ago
  > Could we teach our brain to "think in 4D" and interpret these cues differently? To a certain extent probably, maybe if you spent hours and hours and hours in a VR simulation with 4D objects you'd starting getting a feel for it. I doubt you'd ever get as good as with 3D objects since we've been exposed to these literally every waking moment since we're born.* Good lord! Has anyone tried? This reminds me of the backwards bicycle video on youtube (https://www.youtube.com/watch?v=MFzDaBzBlL0 https://www.youtube.com/watch?v=MFzDaBzBlL0). I... really want to try what you're suggesting.
  - ianai 8y ago
dan_mctree 8y ago
I think you can't. The problem is that we don't actually see 3D, we basically see 2D twice. This gives our senses enough info to reconstruct a 3D mental image, but the information we gain isn't truly 3D. Actual 3D-sight might include us seeing the inside of objects, what's behind them, etc. We're aided in converting our 2x 2D sight into 3D mental models through the use of color and parallax. In projections on images, it's easy to just use color and get rid of parallax, and still get sort of close. Suppose we tried to make a 3D projection of a 4D world, we don't actually get any significant new tools. We're still going to be limited by having to use parallax and color. This doesn't allow us to add enough information to add that 4D projection into 3D as neatly as the 3D into 2D works. At best you could try to distribute the information somehow. For example by giving information about the third dimension through brightness and the fourth dimension through hue. It certainly wouldn't work nearly as well though.
- bunderbunder 8y ago
  > The problem is that we don't actually see 3D, we basically see 2D twice. Most of us do. But people who only have sight in one eye have no difficulty perceiving three dimensions. And some fairly large percentage of people with binocular vision are stereoblind, typically without even realizing it. If anything, I'd say it's not that > We're aided in converting our 2x 2D sight into 3D mental models through the use of color and parallax. so much as that we're aided in converting our 2D sight into 3D mental models through the use of stuff like color and parallax through the use of stereopsis.
  - lostlogin 8y ago
    As you mention, binocular vision is lacking in a surprising number of people and until recently this was thought to be uncorrectable if not treated while the child was still young. That turned out to be not entirely accurate. Those without binocular vision can have a fairly large range of non-specific symptoms and perception problems. There is an interesting discussion of the change in understanding in this link. http://www.strabismus.org/all_about_strabismus.html http://www.strabismus.org/all_about_strabismus.html
- 8y ago
machiavelli1024 8y ago
You can’t reconstruct a 3D scene from a 2D projection unless you take educated guesses. E.g. When you look at a picture with something that looks like a chair, you assume that it’s indeed a chair, and then you can estimate its size/pose/etc. But there are infinitely many non-chair shapes that would produce the exact same projection. It’s just that you won’t encounter them in real life, except maybe in trickshots like this: https://youtu.be/SKpOKXAVjGo https://youtu.be/SKpOKXAVjGo
gumby 8y ago
If you start from the understandings (such as we have)of physiology and evolutionary psychology I think the answer is "no" (perhaps there's a trick someone can figure out via a different path -- I hope so). It seems pretty clear that looking at 2D images (and in particular, still ones) is a learned task, like reading, rather than an innate one. Both appear to tie into deep structures in the brain, but both are very recent inventions. From the perspective of developmental stages described by Piaget, children learn to view in three space primarily with objects within reach (parallax pretty much peters out around the ends of your arms. Once the child becomes mobile, she is able to use semantic understanding to estimate the size of distant objects and get a rough idea of distance. The whole human process of seeing is very different from the way, say, a NN is trained on an image: the whole thing isn't gulped in at once, but we foveate on various parts of the image and assemble / confabulate a whole. You can see this in the structure of Chinese classical painting or pre-persepctive European paintings: distant objects aren't sized in any way proportional to their apparent size. This really maps more to how much attention you pay to various objects in the scene. You then learn to map that into a 3D model which I believe (but am not digging up refs this instance, sorry) has hardware support. Thus the 2D->3D process exploits a lot of learned and innate knowledge and technique that you have already developed. With one exception you haven't any 4D experience. That one exception is temporal data -- we can easily extrapolate from, say, a sphere shrinking and growing. Apart from that, there isn't much to work with.
csomar 8y ago
I don't think the problem is rendering but understanding. Your brain reads a cube, as eh, a cube. If I give you this [2,4,4,6]+ and tell you rzwitserloot that is a cube in the space. You can still imagine that. So a 4D rendering is just more information in a 2D flat screen. Your brain can't process all of this information. So it "can't see it". But it is there. +dunno if these coordinates translates to a cube or just some 2-3D object but I hope you get the point.
essayist 8y ago
Study from 45 years ago: Heinz von Foerster’s 1970-1971 experiment at the Biological Computing Laboratory for apprehending the fourth dimension is unique ... combining four dimensional geometry, stereoscopic vision, and joystick manipulation of objects on the screen. ... The fourth dimension was chosen as the knowledge to be acquired because there was no chance that any subjects would have attempted such knowledge before the experiment. By allowing the physical “grasping” of the visual object, where one hand coordinated movement on three axes in the 3rd dimension, while the other similarly controlled three axes of movement in the 4th dimension, subjects were able to intuitively figure out that the strange succession of transforming 3D objects they were seeing (with 3D glasses) were cross-sections of a single 4D object. http://psychedelicsandlanguage.com/dimensionality/ http://psychedelicsandlanguage.com/dimensionality/ Disclaimer: I studied with von Foerster years ago, and he'd mentioned this. I only today found this online reference. I want those 4D Toys. Steam, here I come.
- romwell 8y ago
  The next thing to come from the same person, Miegakure[1], is going to be even more epic and mind-blowing. But it will take a while for it to be done. [1]http://miegakure.com/ http://miegakure.com/
amakuri 8y ago
Wow, I had the same idea a while back and thought about writing some kind of engine which does this - but I didn't even begin to research whether someone already did something similar. Now I see this on HN :) So I searched and found this: http://www.urticator.net/maze/ http://www.urticator.net/maze/ It seems to be exactly what you propose, a 4D world rendered into 3D space. It even has a kind of "stereo" mode, enabling a 3D experience on a 2D moniter. And to expand on the idea: Once we become familiar with 4D, we can continue and use it to explore 5D, can't we?
modeless 8y ago
> Can you render a 4D scene onto a 3D viewport [...] I wonder what that would look like. It would look like VR. 4D toys supports VR already. I tried it but it did not advance my understanding further than the 2D version, although using 3D tracked controllers is definitely a big improvement over a 2D mouse for manipulating 3D objects or 3D projections of 4D objects.
nikanj 8y ago
If you use time for 4th dimension, you can easily do a 4d-scene on a 2d-screen
mukuro 8y ago
There are YouTube videos where people attempt to render 4D scenes, usually by slicing a mesh along the fourth component of its position, then drawing it as 3D. But since we lack the intuition for 4D space it just looks like a bunch of objects popping in and out of existence.
grawprog 8y ago
Has anybody tried any of the 4d maze games? https://en.m.wikipedia.org/wiki/List_of_four-dimensional_games https://en.m.wikipedia.org/wiki/List_of_four-dimensional_gam... I've tried a maze game and a 4D space shooter before and I could never wrap my head around them. I don't know if it was just poor representations or if it was just because my brain is incapable of understanding.

anotheryou 8y ago

I asked the same question a few seconds before you :) found this: https://youtu.be/S-yRYmdsnGs?t=252 https://youtu.be/S-yRYmdsnGs?t=252 and even better, this: https://youtu.be/dy_MUfBuq2I https://youtu.be/dy_MUfBuq2I

dm319 8y ago
Yes, and I think this just shows how a 3d representation on 2d gives our brain the _illusion_ of 3d. This doesn't work for 4d, because our brain doesn't have an internal understanding of 4d objects.
- anotheryou 8y ago
  I think once we have it interactively our brain will adapt.
  - ianai 8y ago
    It’s t least worth a shot.
- notyourtypical 8y ago
  >our brain doesn't have an internal understanding of 4d objects I agree and I think it will never have. We can certainly speculate but ultimately it'd be like an animal with monochromatic vision trying to see colours.

joshumax 8y ago

Unfortunately, it's not as simple as that (although I wish it were). While you could generate a 3D perspective on each 2D "slice" of a 3D viewport, it wouldn't be that much more different than what's going on in here. The main issue arises from trying to render a 4-polytope in a universe where we ourselves can only sense 3 dimensions "natively."

myfonj 8y ago
Sad thing is that what we sense (see) is in fact just a poor projection into 2D (aka perspective) with some largely learnt concept of depth. If only we could sense space entirely with nothing "behind", not just from single point...
- ozzmotik 8y ago
  i think it's great that you represent it as a projection because that is exactly what it is: a representation of another, perhaps more "objective" (though I'm not sure at that level that attributes like that apply) data set. I think it's super easy for us to assume what we visually perceive has a 1:1 correspondence with the intrinsic nature of the data it is constructed from. and i think it's also important to realize that that construct is also informed by other avenues of sensory input and that extra context tends to take it even further away from the nature of the raw data it is constructed from
  - myfonj 8y ago
    Perhaps it is determined by the fact senses are bound to body and body is limitary. Our 2d vision could grasp entire Flatland (if it fitted into single view), but sight of similarly finite Flatland inhabitant could grasp just his immediate surroundings, i.e. the sum of closest opaque objects in his 1D sight.

MAXPOOL 8y ago

Yes. there are many ways to do 4d to 3d projections, including perspective projections. You could also make a 3d 'camera' view with 4d lens and 3d projection plane. Here is a good intro to 4D Visualization http://eusebeia.dyndns.org/4d/vis/vis http://eusebeia.dyndns.org/4d/vis/vis

simias 8y ago

The problem is that while everything you see on your (2D) monitor is effectively 2D our brain is good at taking 3D cues (shadows, perspective, parallax etc...) to reconstitute 3D "data" from it. For instance look at 3D rotating cube on your monitor: https://www.youtube.com/watch?v=5E10zYKln3g https://www.youtube.com/watch?v=5E10zYKln3g Even though there's no shading, a bad resolution and zero context anchoring it in a familiar setting you immediately see it as a 3D object when effectively that's just 3 2D quadrilaterals changing colors and shape. We really need very little to be able to extrapolate depth information from a very limited 2D-only display. If you did that with 4D cues in 3D space (say in VR or something) I think it wouldn't work. It'd just look like a 3D object changing shape, not a projection of a higher dimension (the same way an hypothetical 2D being would see 3 quadrilaterals changing shape in the above video, not a solid 3D object that would be an abstract concept to them, not a familiar reality). Could we teach our brain to "think in 4D" and interpret these cues differently? To a certain extent probably, maybe if you spent hours and hours and hours in a VR simulation with 4D objects you'd start getting a feel for it. I doubt you'd ever get as good as with 3D objects since we've been exposed to these literally every waking moment since birth, but maybe I underestimate the plasticity of our brain. That actually leads me to an other question: is 3D somewhat hardcoded in our brains or is it purely learned? If we were mad scientists and took a baby brain into a 4D "plato's cave" style simulation, could it grow into being able to perceive 4D as intuitively and effectively as we do with 3D space? Also unrelated question: does anybody have a baby I could borrow?

scrollaway 8y ago
> Could we teach our brain to "think in 4D" and interpret these cues differently? To a certain extent probably, maybe if you spent hours and hours and hours in a VR simulation with 4D objects you'd starting getting a feel for it. I doubt you'd ever get as good as with 3D objects since we've been exposed to these literally every waking moment since we're born.* Good lord! Has anyone tried? This reminds me of the backwards bicycle video on youtube (https://www.youtube.com/watch?v=MFzDaBzBlL0 https://www.youtube.com/watch?v=MFzDaBzBlL0). I... really want to try what you're suggesting.
- ianai 8y ago

dan_mctree 8y ago

I think you can't. The problem is that we don't actually see 3D, we basically see 2D twice. This gives our senses enough info to reconstruct a 3D mental image, but the information we gain isn't truly 3D. Actual 3D-sight might include us seeing the inside of objects, what's behind them, etc. We're aided in converting our 2x 2D sight into 3D mental models through the use of color and parallax. In projections on images, it's easy to just use color and get rid of parallax, and still get sort of close. Suppose we tried to make a 3D projection of a 4D world, we don't actually get any significant new tools. We're still going to be limited by having to use parallax and color. This doesn't allow us to add enough information to add that 4D projection into 3D as neatly as the 3D into 2D works. At best you could try to distribute the information somehow. For example by giving information about the third dimension through brightness and the fourth dimension through hue. It certainly wouldn't work nearly as well though.

bunderbunder 8y ago
> The problem is that we don't actually see 3D, we basically see 2D twice. Most of us do. But people who only have sight in one eye have no difficulty perceiving three dimensions. And some fairly large percentage of people with binocular vision are stereoblind, typically without even realizing it. If anything, I'd say it's not that > We're aided in converting our 2x 2D sight into 3D mental models through the use of color and parallax. so much as that we're aided in converting our 2D sight into 3D mental models through the use of stuff like color and parallax through the use of stereopsis.
- lostlogin 8y ago
  As you mention, binocular vision is lacking in a surprising number of people and until recently this was thought to be uncorrectable if not treated while the child was still young. That turned out to be not entirely accurate. Those without binocular vision can have a fairly large range of non-specific symptoms and perception problems. There is an interesting discussion of the change in understanding in this link. http://www.strabismus.org/all_about_strabismus.html http://www.strabismus.org/all_about_strabismus.html
8y ago

machiavelli1024 8y ago

You can’t reconstruct a 3D scene from a 2D projection unless you take educated guesses. E.g. When you look at a picture with something that looks like a chair, you assume that it’s indeed a chair, and then you can estimate its size/pose/etc. But there are infinitely many non-chair shapes that would produce the exact same projection. It’s just that you won’t encounter them in real life, except maybe in trickshots like this: https://youtu.be/SKpOKXAVjGo https://youtu.be/SKpOKXAVjGo

gumby 8y ago

If you start from the understandings (such as we have)of physiology and evolutionary psychology I think the answer is "no" (perhaps there's a trick someone can figure out via a different path -- I hope so). It seems pretty clear that looking at 2D images (and in particular, still ones) is a learned task, like reading, rather than an innate one. Both appear to tie into deep structures in the brain, but both are very recent inventions. From the perspective of developmental stages described by Piaget, children learn to view in three space primarily with objects within reach (parallax pretty much peters out around the ends of your arms. Once the child becomes mobile, she is able to use semantic understanding to estimate the size of distant objects and get a rough idea of distance. The whole human process of seeing is very different from the way, say, a NN is trained on an image: the whole thing isn't gulped in at once, but we foveate on various parts of the image and assemble / confabulate a whole. You can see this in the structure of Chinese classical painting or pre-persepctive European paintings: distant objects aren't sized in any way proportional to their apparent size. This really maps more to how much attention you pay to various objects in the scene. You then learn to map that into a 3D model which I believe (but am not digging up refs this instance, sorry) has hardware support. Thus the 2D->3D process exploits a lot of learned and innate knowledge and technique that you have already developed. With one exception you haven't any 4D experience. That one exception is temporal data -- we can easily extrapolate from, say, a sphere shrinking and growing. Apart from that, there isn't much to work with.

csomar 8y ago

I don't think the problem is rendering but understanding. Your brain reads a cube, as eh, a cube. If I give you this [2,4,4,6]+ and tell you rzwitserloot that is a cube in the space. You can still imagine that. So a 4D rendering is just more information in a 2D flat screen. Your brain can't process all of this information. So it "can't see it". But it is there. +dunno if these coordinates translates to a cube or just some 2-3D object but I hope you get the point.

essayist 8y ago

Study from 45 years ago: Heinz von Foerster’s 1970-1971 experiment at the Biological Computing Laboratory for apprehending the fourth dimension is unique ... combining four dimensional geometry, stereoscopic vision, and joystick manipulation of objects on the screen. ... The fourth dimension was chosen as the knowledge to be acquired because there was no chance that any subjects would have attempted such knowledge before the experiment. By allowing the physical “grasping” of the visual object, where one hand coordinated movement on three axes in the 3rd dimension, while the other similarly controlled three axes of movement in the 4th dimension, subjects were able to intuitively figure out that the strange succession of transforming 3D objects they were seeing (with 3D glasses) were cross-sections of a single 4D object. http://psychedelicsandlanguage.com/dimensionality/ http://psychedelicsandlanguage.com/dimensionality/ Disclaimer: I studied with von Foerster years ago, and he'd mentioned this. I only today found this online reference. I want those 4D Toys. Steam, here I come.

romwell 8y ago
The next thing to come from the same person, Miegakure[1], is going to be even more epic and mind-blowing. But it will take a while for it to be done. [1]http://miegakure.com/ http://miegakure.com/

amakuri 8y ago

Wow, I had the same idea a while back and thought about writing some kind of engine which does this - but I didn't even begin to research whether someone already did something similar. Now I see this on HN :) So I searched and found this: http://www.urticator.net/maze/ http://www.urticator.net/maze/ It seems to be exactly what you propose, a 4D world rendered into 3D space. It even has a kind of "stereo" mode, enabling a 3D experience on a 2D moniter. And to expand on the idea: Once we become familiar with 4D, we can continue and use it to explore 5D, can't we?

modeless 8y ago

> Can you render a 4D scene onto a 3D viewport [...] I wonder what that would look like. It would look like VR. 4D toys supports VR already. I tried it but it did not advance my understanding further than the 2D version, although using 3D tracked controllers is definitely a big improvement over a 2D mouse for manipulating 3D objects or 3D projections of 4D objects.

nikanj 8y ago

If you use time for 4th dimension, you can easily do a 4d-scene on a 2d-screen

mukuro 8y ago

There are YouTube videos where people attempt to render 4D scenes, usually by slicing a mesh along the fourth component of its position, then drawing it as 3D. But since we lack the intuition for 4D space it just looks like a bunch of objects popping in and out of existence.

grawprog 8y ago

Has anybody tried any of the 4d maze games? https://en.m.wikipedia.org/wiki/List_of_four-dimensional_games https://en.m.wikipedia.org/wiki/List_of_four-dimensional_gam... I've tried a maze game and a 4D space shooter before and I could never wrap my head around them. I don't know if it was just poor representations or if it was just because my brain is incapable of understanding.