12 ms·

Spatial computing could be the next big thing

93 points by bra-ket 6y ago

anupamchugh 6y ago
Is this by any chance related to Spatial Audio? The technology and API Apple exposed with iOS 14 in AirPods.
- moonbug 6y ago
  Would it hurt you to RTFA?
  - spicybright 6y ago
    Would it hurt you to be a bit nicer?
- visarga 6y ago
  No, someone just renamed computer vision into 'spatial computing' and called it the next big thing.
joezydeco 6y ago
So, a rebranding of IoT. Makes sense to me.
- tmilard 6y ago
  Yes I agree. Iot is one of the technical sub element of spatial computing
  - Animats 6y ago
    There's robotics in there, moving around tables and beds and such. That's hard, and a maintenance headache. So, like regular IoT, it won't get done, and you just get the surveillance part. The Internet of Things crowd hasn't even been able to do affordable power windows for houses yet. Even basic heating duct dampers don't integrate well, much to the annoyance of Nest owners. Let alone an outside air intake.
    - joezydeco 6y ago
      But renaming it to "spatial computing" will magically enable all these devices with new and affordable capabilities?
      - Animats 6y ago
        Right. I once went to an "Internet of Things" meeting in SF. It was in a converted factory space near the shipyard, with big windows looking out on the bay, and skylights in the roof. The big windows had a manual endless chain which could be used to crank them open, as did the skylights. There were also ceiling fans and a conventional HVAC system. None of this was integrated into an automatic climate control system, which was amusing to see while being pitched on IoT. It would be nice to see the windows, skylights, and fans coordinated to provide a good environment without consuming fuel unnecessarily, but no. It got too stuffy when there were too many people inside, despite all that access to the outside. When I saw "Spatial computing", I thought this was going to be about Spatial OS, from Improbable. They built a system intended to be a back end for big-world MMOs. They burned through several hundred million dollars (from Softbank, who else?) and produced a back end nobody can afford to use. You have to host on Google and pay for every in-game event. Several games shipped using it and they all went bust within months.
- yummypaint 6y ago
  Agreed, it mostly sounds like more contrivances for surveillance capitalists to make money.
tmilard 6y ago
When technonoly settles down, I imagine spatial computing as a physical small device (mini computer) in which you plug one two.. or 20 cameras. Then, after an initialisation process, that spatial automat is ready. It can now detect things in an environment it has automatically detected (indoor flat, hospital, ect). Then the programming development will only be to : - link between 'spatial events' like a dog barka, a lying grandma seems inconscient,ect.. - and fonctional processes like 'call ambulance', 'put alarm on', ect...
- ampdepolymerase 6y ago
  And uploaded to a cloud analytics platform so your insurance provider can better do risk modeling.
andreygrehov 6y ago
> Major companies, including Microsoft and Amazon, are heavily invested in the technology. Source, examples?
- moron4hire 6y ago
  Windows Mixed Reality, HoloLens, Azure Spatial Anchors, Azure Kinect, all fairly large projects (well, except Azure Kinect, but they have kept it around a lot longer than I would have expected, so they seem pretty dedicated).
jandrewrogers 6y ago
The big challenge for spatial computing is that there is no spatial computer science, and casual observers overlook this fact. As in, there is a dearth of computer science -- basic data structures and algorithms -- for efficiently reasoning about spatiotemporal relationships of the type common to multimodal sensor data models at scale. Neither Amazon nor Microsoft, companies mentioned in the article, are addressing this technology gap in a meaningful way (I have some contact with their programs). They are solving very narrow cases that don't generalize or scale usefully. A question I ask when doing due diligence on "spatial computing" technology is what computer science problems did you solve to create the technology? The common answer is "none" and most people dabbling in the space aren't even aware hard computer science problems exist as prerequisites to building such technology unless they've been banging their head against it for a while. That's the state of the spatial computing market today. We also know that it is (currently) impossible to build such a thing by sprinkling some magic spatial computing dust on open source technology stacks -- they can't express the necessary mechanics. You'll be writing a lot of code from first principles, it isn't a business execution problem. We've had petabytes of this type of sensor data for decades, and the vision and desire to build digital twins with that data, that part isn't new. There are hardcore technical reasons this doesn't exist as a real thing, it isn't that no one has ever thought of spatial computing before. To make matters worse, most of the companies with the expertise and desire to use such a platform lack the expertise and execution to create it. Creating such a platform has to come from somewhere else. It is possible to build a spatial computing platform today that actually works, the computer science challenges have solutions, but the combination of domain expertise and computer science skillset to execute it is extremely rare at the moment. That said, spatial computing will be a trillion dollar platform business once someone pulls it off.
- waihtis 6y ago
  Would you be willing to expand a bit more on the hard problems and their relation to spatial computing?
  - jandrewrogers 6y ago
    The elementary operations for spatial computing are a mixture of complex constraint/event processing and ad hoc graph search, frequently on data models with types that are not representable as an integer and therefore lack the characteristic properties of integers like order and trivial partitioning. Three of the above are notoriously difficult to scale individually. For spatial computing we need to scale all three simultaneously in a single computational construct. And if that wasn't enough, these are real-time systems that need to support extremely high write throughput while all of those operations are happening -- offline batch reorganization isn't an option. These are petabyte scale systems in the trivial case (tens of TB in the most minimalist data models), so you aren't going to be building a bunch of indexes or similar. It is a vexing computer science and architecture problem. And even if you solved that elementary computing infrastructure problem, there are several more hard/interesting computer science problems related to topological types and conflict resolution that you'll likely need to solve before you have a useful system. It is headaches all the way down.
disown 6y ago
I thought it was quantum computing? It's crazy how quickly the hype switches. What ever happened to graphene? I remember reading a bunch of articles in 2015 that it would revolutionize society by 2020.
- mlyle 6y ago
  > I thought it was quantum computing? It's crazy how quickly the hype switches. It's almost like the future needs multiple kinds of progress and breakthroughs. Quantum computing looks like it's going to be very useful for some unusual, top-end problems before too, too long (let's say, 7 years). Spatial computing might enable lots of small benefits in many sectors. > What ever happened to graphene? I remember reading a bunch of articles in 2015 that it would revolutionize society by 2020. Quietly growing at a 30-60% CAGR in dollars (and by a ridiculous factor in usage, since prices are coming down rapidly), finding dozens of high-end, simple (early) applications (with prospects for many new uses on the horizon).
enriquto 6y ago
I understand most words in this essay, but not the high-level meaning. What is spatial computing? How is it different to 3D reconstruction, slam, and the like? (EDIT: changed "the words" to "most words".)
- mlyle 6y ago
  3d reconstruction is one small piece of a suite that would solve problems like this-- there's also path solving, semantic scene representation, kinematics, etc. What's in the space, how would people in the space understand/describe it, how can actuators interact with it, how can people and devices find paths, what does a person moving along this path mean, how can a device assist in this abstract goal... It's a lot more than a mesh of geometry and a set of probable robot locations to do something useful. It's a whole lot of data and a whole lot of things that are poorly understood.
homonculus1 6y ago
Technical hurdles aside, this vision really sucks! Old lady shells out a small fortune to surround herself with robotic spyware to facilitate the utopia of... living completely alone in one's elder years? The real problem is that she lives in an alienated society which has left her without the presence of kids, boarders, neighbors, church friends, etc. in her home. What's the name of that story where everyone lives in their own automated room and the central computer won't let them above ground? Solving for that world is a crime against society, not technological progress.
- generalizations 6y ago
  > name of that story where everyone lives in their own automated room and the central computer won't let them above ground The Machine Stops. https://en.wikipedia.org/wiki/The_Machine_Stops https://en.wikipedia.org/wiki/The_Machine_Stops
bitwize 6y ago
So if us-east-1 goes down, does Martha become a prisoner in her own home? Worse, will she be bludgeoned by a piece of malfunctioning smart home furniture? We didn't think through the implications of benign failure cases the first time we tried IoT, let alone determined malicious attackers, and things went to shit. I'm not convinced that we'll do any better once "spatial computing" goes commercial.
- spicybright 6y ago
  What do you mean, us-east-1 is too big to ever go down, amazon says it's super reliable! /s
o-__-o 6y ago
Once upon a time about four years ago I made a system that would monitor my location based on WiFi signal strength. If I was watching a movie from my local DLNA server, I could track the progress of the video (remotable software on a fire tv). If I was watching tv, I could query what channel my hdhomerun was on. I then put this together. If my phone moved from downstairs to upstairs or vice versa, the movie would rewind 10 seconds and would be played on the tv upstairs or downstairs. This gave me the ability to move between my two rooms with TVs and the show would follow me. For antenna tv, the hdhomerun app would be loaded and set to the correct channel. I wanted to expand this to my zwave lights using motion sensors but never got around to it because I was fixated on my garage. My garage has a camera that can stream rtsp. I had recently played with opencv and got it to detect our cars with pretty good accuracy but our cars are generic cars that half of america drives. So I spent almost 2 years trying to figure a way to get the system to identify our cars. I went so far down the path of using a raspberry pi in the car to authenticate with the house which would trigger the garage door to open as we pull up the driveway. Then my buddy recommended two colored stickers on our windshield. So simple and it worked! (Edit: a lot of this can be resolved today with a high resolution camera that can make out our license plate from a far enough distance... but I just lowered my car and ripped off the front license plate so back to square one) I’ve been fighting small bugs for the last year or so that makes it hard to use (dog may set off the motion sensor, or you forget your phone and lights and tv turns off thinking no one is around). Plus we got a new tv which replaced the buggy fire tv last Christmas which has a completely different api and less interaction. I’ve considered using Bluetooth so smart wristbands could replace our phones but other projects have creeped into scope. I would love to solve it but have a very hard time selling the system to friends and family who think nothing of flipping a light switch or picking up a remote. Maybe this is a tech too early for its time?
mcoliver 6y ago
The days of store and batch analyze are over. There is just way too much data being generated with the myriad of cheap sensors available. Being able to analyze data and adjust your models in realtime is the only way forward. Check out Software in Motion https://www.swimos.org/ https://www.swimos.org/ One of the more interesting platforms in this space.
cliverani 6y ago
Spatial computing is really that the physical world and digital world are fully interacting. Where you can overlay digital information into the real world and what you do in the real world affects the digital world. The most common use cases today would be AR/VR.Spatial computing could be embedded into something as small as a room, a city, or even a country. Spatial computing within an operating room is a clear value prop. Spatial computing companies like Magic Leap have shown that the infrastructure is bulky and expensive. Most importantly, consumers aren't willing to pay for it just yet. Expect to see enterprise and government applications before smart furniture.
jcun4128 6y ago
Not entirely sure if directly related to point clouds/slam The newer devices coming with Lidar... imagine integration with that and "upload your living room" to a furniture store then shows your living room with random combinations of products -- like that scene in Fight Club. I'm coming from web interface/api work so trying to get into this space(3d slam) is kind of hard... the stuff I'm working on is pathetic computation wise eg. single core pi. But the idea of getting a running state in real time with real measurements paired with imu data is pretty cool to me.

93 points by bra-ket 6y ago

anupamchugh 6y ago

Is this by any chance related to Spatial Audio? The technology and API Apple exposed with iOS 14 in AirPods.

moonbug 6y ago
Would it hurt you to RTFA?
- spicybright 6y ago
  Would it hurt you to be a bit nicer?
visarga 6y ago
No, someone just renamed computer vision into 'spatial computing' and called it the next big thing.

joezydeco 6y ago

So, a rebranding of IoT. Makes sense to me.

tmilard 6y ago
Yes I agree. Iot is one of the technical sub element of spatial computing
- Animats 6y ago
  There's robotics in there, moving around tables and beds and such. That's hard, and a maintenance headache. So, like regular IoT, it won't get done, and you just get the surveillance part. The Internet of Things crowd hasn't even been able to do affordable power windows for houses yet. Even basic heating duct dampers don't integrate well, much to the annoyance of Nest owners. Let alone an outside air intake.
  - joezydeco 6y ago
    But renaming it to "spatial computing" will magically enable all these devices with new and affordable capabilities?
    - Animats 6y ago
      Right. I once went to an "Internet of Things" meeting in SF. It was in a converted factory space near the shipyard, with big windows looking out on the bay, and skylights in the roof. The big windows had a manual endless chain which could be used to crank them open, as did the skylights. There were also ceiling fans and a conventional HVAC system. None of this was integrated into an automatic climate control system, which was amusing to see while being pitched on IoT. It would be nice to see the windows, skylights, and fans coordinated to provide a good environment without consuming fuel unnecessarily, but no. It got too stuffy when there were too many people inside, despite all that access to the outside. When I saw "Spatial computing", I thought this was going to be about Spatial OS, from Improbable. They built a system intended to be a back end for big-world MMOs. They burned through several hundred million dollars (from Softbank, who else?) and produced a back end nobody can afford to use. You have to host on Google and pay for every in-game event. Several games shipped using it and they all went bust within months.
yummypaint 6y ago
Agreed, it mostly sounds like more contrivances for surveillance capitalists to make money.

tmilard 6y ago

When technonoly settles down, I imagine spatial computing as a physical small device (mini computer) in which you plug one two.. or 20 cameras. Then, after an initialisation process, that spatial automat is ready. It can now detect things in an environment it has automatically detected (indoor flat, hospital, ect). Then the programming development will only be to : - link between 'spatial events' like a dog barka, a lying grandma seems inconscient,ect.. - and fonctional processes like 'call ambulance', 'put alarm on', ect...

ampdepolymerase 6y ago
And uploaded to a cloud analytics platform so your insurance provider can better do risk modeling.

andreygrehov 6y ago

> Major companies, including Microsoft and Amazon, are heavily invested in the technology. Source, examples?

moron4hire 6y ago
Windows Mixed Reality, HoloLens, Azure Spatial Anchors, Azure Kinect, all fairly large projects (well, except Azure Kinect, but they have kept it around a lot longer than I would have expected, so they seem pretty dedicated).

jandrewrogers 6y ago

The big challenge for spatial computing is that there is no spatial computer science, and casual observers overlook this fact. As in, there is a dearth of computer science -- basic data structures and algorithms -- for efficiently reasoning about spatiotemporal relationships of the type common to multimodal sensor data models at scale. Neither Amazon nor Microsoft, companies mentioned in the article, are addressing this technology gap in a meaningful way (I have some contact with their programs). They are solving very narrow cases that don't generalize or scale usefully. A question I ask when doing due diligence on "spatial computing" technology is what computer science problems did you solve to create the technology? The common answer is "none" and most people dabbling in the space aren't even aware hard computer science problems exist as prerequisites to building such technology unless they've been banging their head against it for a while. That's the state of the spatial computing market today. We also know that it is (currently) impossible to build such a thing by sprinkling some magic spatial computing dust on open source technology stacks -- they can't express the necessary mechanics. You'll be writing a lot of code from first principles, it isn't a business execution problem. We've had petabytes of this type of sensor data for decades, and the vision and desire to build digital twins with that data, that part isn't new. There are hardcore technical reasons this doesn't exist as a real thing, it isn't that no one has ever thought of spatial computing before. To make matters worse, most of the companies with the expertise and desire to use such a platform lack the expertise and execution to create it. Creating such a platform has to come from somewhere else. It is possible to build a spatial computing platform today that actually works, the computer science challenges have solutions, but the combination of domain expertise and computer science skillset to execute it is extremely rare at the moment. That said, spatial computing will be a trillion dollar platform business once someone pulls it off.

waihtis 6y ago
Would you be willing to expand a bit more on the hard problems and their relation to spatial computing?
- jandrewrogers 6y ago
  The elementary operations for spatial computing are a mixture of complex constraint/event processing and ad hoc graph search, frequently on data models with types that are not representable as an integer and therefore lack the characteristic properties of integers like order and trivial partitioning. Three of the above are notoriously difficult to scale individually. For spatial computing we need to scale all three simultaneously in a single computational construct. And if that wasn't enough, these are real-time systems that need to support extremely high write throughput while all of those operations are happening -- offline batch reorganization isn't an option. These are petabyte scale systems in the trivial case (tens of TB in the most minimalist data models), so you aren't going to be building a bunch of indexes or similar. It is a vexing computer science and architecture problem. And even if you solved that elementary computing infrastructure problem, there are several more hard/interesting computer science problems related to topological types and conflict resolution that you'll likely need to solve before you have a useful system. It is headaches all the way down.

disown 6y ago

I thought it was quantum computing? It's crazy how quickly the hype switches. What ever happened to graphene? I remember reading a bunch of articles in 2015 that it would revolutionize society by 2020.

mlyle 6y ago
> I thought it was quantum computing? It's crazy how quickly the hype switches. It's almost like the future needs multiple kinds of progress and breakthroughs. Quantum computing looks like it's going to be very useful for some unusual, top-end problems before too, too long (let's say, 7 years). Spatial computing might enable lots of small benefits in many sectors. > What ever happened to graphene? I remember reading a bunch of articles in 2015 that it would revolutionize society by 2020. Quietly growing at a 30-60% CAGR in dollars (and by a ridiculous factor in usage, since prices are coming down rapidly), finding dozens of high-end, simple (early) applications (with prospects for many new uses on the horizon).

enriquto 6y ago

I understand most words in this essay, but not the high-level meaning. What is spatial computing? How is it different to 3D reconstruction, slam, and the like? (EDIT: changed "the words" to "most words".)

mlyle 6y ago
3d reconstruction is one small piece of a suite that would solve problems like this-- there's also path solving, semantic scene representation, kinematics, etc. What's in the space, how would people in the space understand/describe it, how can actuators interact with it, how can people and devices find paths, what does a person moving along this path mean, how can a device assist in this abstract goal... It's a lot more than a mesh of geometry and a set of probable robot locations to do something useful. It's a whole lot of data and a whole lot of things that are poorly understood.

homonculus1 6y ago

Technical hurdles aside, this vision really sucks! Old lady shells out a small fortune to surround herself with robotic spyware to facilitate the utopia of... living completely alone in one's elder years? The real problem is that she lives in an alienated society which has left her without the presence of kids, boarders, neighbors, church friends, etc. in her home. What's the name of that story where everyone lives in their own automated room and the central computer won't let them above ground? Solving for that world is a crime against society, not technological progress.

generalizations 6y ago
> name of that story where everyone lives in their own automated room and the central computer won't let them above ground The Machine Stops. https://en.wikipedia.org/wiki/The_Machine_Stops https://en.wikipedia.org/wiki/The_Machine_Stops

bitwize 6y ago

So if us-east-1 goes down, does Martha become a prisoner in her own home? Worse, will she be bludgeoned by a piece of malfunctioning smart home furniture? We didn't think through the implications of benign failure cases the first time we tried IoT, let alone determined malicious attackers, and things went to shit. I'm not convinced that we'll do any better once "spatial computing" goes commercial.

spicybright 6y ago
What do you mean, us-east-1 is too big to ever go down, amazon says it's super reliable! /s

o-__-o 6y ago

Once upon a time about four years ago I made a system that would monitor my location based on WiFi signal strength. If I was watching a movie from my local DLNA server, I could track the progress of the video (remotable software on a fire tv). If I was watching tv, I could query what channel my hdhomerun was on. I then put this together. If my phone moved from downstairs to upstairs or vice versa, the movie would rewind 10 seconds and would be played on the tv upstairs or downstairs. This gave me the ability to move between my two rooms with TVs and the show would follow me. For antenna tv, the hdhomerun app would be loaded and set to the correct channel. I wanted to expand this to my zwave lights using motion sensors but never got around to it because I was fixated on my garage. My garage has a camera that can stream rtsp. I had recently played with opencv and got it to detect our cars with pretty good accuracy but our cars are generic cars that half of america drives. So I spent almost 2 years trying to figure a way to get the system to identify our cars. I went so far down the path of using a raspberry pi in the car to authenticate with the house which would trigger the garage door to open as we pull up the driveway. Then my buddy recommended two colored stickers on our windshield. So simple and it worked! (Edit: a lot of this can be resolved today with a high resolution camera that can make out our license plate from a far enough distance... but I just lowered my car and ripped off the front license plate so back to square one) I’ve been fighting small bugs for the last year or so that makes it hard to use (dog may set off the motion sensor, or you forget your phone and lights and tv turns off thinking no one is around). Plus we got a new tv which replaced the buggy fire tv last Christmas which has a completely different api and less interaction. I’ve considered using Bluetooth so smart wristbands could replace our phones but other projects have creeped into scope. I would love to solve it but have a very hard time selling the system to friends and family who think nothing of flipping a light switch or picking up a remote. Maybe this is a tech too early for its time?

mcoliver 6y ago

The days of store and batch analyze are over. There is just way too much data being generated with the myriad of cheap sensors available. Being able to analyze data and adjust your models in realtime is the only way forward. Check out Software in Motion https://www.swimos.org/ https://www.swimos.org/ One of the more interesting platforms in this space.

cliverani 6y ago

Spatial computing is really that the physical world and digital world are fully interacting. Where you can overlay digital information into the real world and what you do in the real world affects the digital world. The most common use cases today would be AR/VR.Spatial computing could be embedded into something as small as a room, a city, or even a country. Spatial computing within an operating room is a clear value prop. Spatial computing companies like Magic Leap have shown that the infrastructure is bulky and expensive. Most importantly, consumers aren't willing to pay for it just yet. Expect to see enterprise and government applications before smart furniture.

jcun4128 6y ago

Not entirely sure if directly related to point clouds/slam The newer devices coming with Lidar... imagine integration with that and "upload your living room" to a furniture store then shows your living room with random combinations of products -- like that scene in Fight Club. I'm coming from web interface/api work so trying to get into this space(3d slam) is kind of hard... the stuff I'm working on is pathetic computation wise eg. single core pi. But the idea of getting a running state in real time with real measurements paired with imu data is pretty cool to me.