16 ms·
Show HN: Immersive Gaussian Splat experience of Sutro Tower, San Francisco
- smithclay 2y agoFun fact: NBC does not broadcast from Sutro tower, but San Bruno… Which means effectively zero over-the-air reception in parts of the Mission. Lesson learned during a Super Bowl party the 2010s.
- amgee 2y agoThis is fantastic. I've been trying to build something myself for biological and heritage captures. Could you elaborate at all on the JS decoder method used or resources that ended up enabling you to make it? I've been playing with Playcanvas React for the past week, would your method integrate with their platform?
- ovenchips 2y agoThis application was built on PlayCanvas Web Components: https://github.com/playcanvas/web-components/ https://github.com/playcanvas/web-components/ It's very similar to PlayCanvas React, but with a fully Web Components-based architecture. You can use the same scripts interchangeably between both frameworks.
- amgee 2y agoI'm assuming you started working on this before their React release, or was Web Components chosen for a specific reason? I'm trying to make something that can take in real-time data and also handle some XR/AR if possible as well, but I am pretty ignorant beyond html/css/js.
- akanet 2y agoWe're trying to get this upstreamed so I actually expect PlayCanvas to talk about it in the future. Here's the reference (cpu) decoder: https://github.com/vincentwoo/blog/blob/master/3d/sutro_tower/sogs-decoder.js https://github.com/vincentwoo/blog/blob/master/3d/sutro_towe..., as well as a gpu one: https://github.com/vincentwoo/blog/blob/master/3d/sutro_tower/sogs.js https://github.com/vincentwoo/blog/blob/master/3d/sutro_towe.... For more information on the format itself you can look at https://github.com/nerfstudio-project/gsplat/blob/main/gsplat/compression/png_compression.py https://github.com/nerfstudio-project/gsplat/blob/main/gspla....
- slater 2y agoUncaught SyntaxError: import assertions are not currently supported
- ovenchips 2y agoI can't find the easter egg. Clues appreciated. :)
- akanet 2y agoLook around closely at the top level of the tower :)
- qingcharles 2y agoI wonder if it's the dude waving from inside the gantry at the top?
- daft_pink 2y agoThis is amazing. Hoping you will share more of these soon.
- falcor84 2y agoIt's really cool, but I can't shake off an "uncanny valley" feeling, with all of the small quirks in the geometry. What I think I'd be interested in is a post-processing step where this splat is automatically converted to a 3d model that approximates each component, only falling back to the point cloud if there's no simple shape that fits the observation at a particular location.
- c-fe 2y agoThis is close to the idea of convex splatting (recent paper) in which convex shapes are used to approximate these real 3d objects as they are better suited than gaussians
- lbeckman314 2y agohttps://convexsplatting.github.io/ https://convexsplatting.github.io/ TIL — very cool work!
- thot_experiment 2y agoIf you haven't seen Tunnel Vision (same author) please do yourself a favor and watch it. Dude does some fantastic projects.
- pinoy420 2y agoGreat video if you like watching videos from the front of BART which is pretty boring tbf. Much better videos of locos travelling through the alps
- akanet 2y agoDude
- thot_experiment 2y agoContext defines content.
- whatever1 2y agoThe fidelity is amazing !
- pinoy420 2y agoCan we see the images it was made with
- CamperBob2 2y agoWhat blows me away is that R/C drones can operate that close to those antennas. Frequency differences don't tend to matter much once you go past 100,000 watts.
- xrd 2y agoSo gorgeous.
- FlamingMoe 2y agoBeautiful, the visuals combined with the music gave me quite a nostalgic feeling for the city where I once worked daily but haven't visited in years.
- null0pointer 2y agoI had the same feeling. Excellent work!
- yapyap 2y agoI know this from the Watchdog game
- JKCalhoun 2y agoThe wife got me a Sutro Tower shirt one year for Christmas. This was when OTA digital television rolled out. When the rest of San Jose seemed to be taking down their TV aerials or letting them fall apart I was nerding out: purchasing a new one and mast sections from Rat Shack to set up one on our house. Scanning the spectrum to pull in KQED, etc. The first Austin City Limits I saw in HD blew my mind. I think all but one TV station came to the South Bay by way of Sutro. Quite a reach.
- atarian 2y agovery cool, reminds me of watchdogs. where's the music from? it's very calming
- akanet 2y agoIt's actually a mix my girlfriend made, but the main track is https://soundcloud.com/martinlandh/moonlit-serenity https://soundcloud.com/martinlandh/moonlit-serenity
- diogocp 2y agoWhat about the track in your video "I finally made it to the top of Sutro Tower"? It's beautiful! The video: https://www.youtube.com/watch?v=A7-3CKulsCc https://www.youtube.com/watch?v=A7-3CKulsCc
- Samin100 2y agoWow! I'd love to read a more in-depth blog post describing how to create one of these myself, and maybe even contribute my own splats to a collaborative library for iconic landmarks. I could see interactive splats being added to Wikipedia for popular locations.
- akanet 2y agoI give a bit more color in the twitter thread https://x.com/fulligin/status/1892685973731061937 https://x.com/fulligin/status/1892685973731061937
- wonger_ 2y agohttps://xcancel.com/fulligin/status/1892685973731061937 https://xcancel.com/fulligin/status/1892685973731061937
- pbronez 2y agoOnly seems to have the first post; rest of thread didn’t load
- marcellus23 2y agoWorks for me.
- corysama 2y agohttps://reddit.com/r/GaussianSplatting/ https://reddit.com/r/GaussianSplatting/ has been slowly talking about the subject for a while now. There are probably several articles and vids in the search bar. If you want GS news, https://radiancefields.com/ https://radiancefields.com/ reports a lot of advances all the time.
- BoredPositron 2y agoPretty much the same workflow as photogrammetry take a lot of images/videos and put them in one of the SOTA gaussian splatting tools.
- LeoPanthera 2y agoBroadcast television is amazing, and I'm so sad it's dying out. OK, maybe we don't need to tune in at 6 to watch the latest episode of "Friends" anymore, but for any kind of live events - news, sport, politics, having high-definition video you can pull right out of the air without having to worry about paying for data, latency, or bandwidth limitations, is amazing. For certain applications the internet can never compete with "broadcast".
- perching_aix 2y ago? Is over-the-air TV broadcast not encrypted and compressed to oblivion in the US? Cause it definitely is here, and you're expected to pay for a decoder card, except for a small handful of channels.
- vlovich123 2y agoIt’s not encrypted but compression and reliability of the signal is a mixed bag. It’s not encrypted because congress mandated it as a condition for privatizing the analog bandwidth (ie there’s a carve out for public digital and broadcasters have to broadcast publicly accessible signal just like they had to on analog)
- perching_aix 2y agoOh wow okay, definitely didn't expect this. That's quite the power move. I guess it's also why it's disappearing, at least partly.
- vlovich123 2y agoNot really a power move but public broadcast will probably disappear at some point in a boiling the frog kind of way, but it’s definitely being starved and killed by corporate interests.
- LeoPanthera 2y agoIt's not encrypted, no. And the picture quality is very good. Not 4K, but good quality 720 or 1080, depending on the channel.
- spps11 2y agovery cool! always wondered what that tower was haha, now i know!
- IshKebab 2y agoThis does not work at all on Android Chrome. The about dialog flies in super slowly, continually re-layouting the text, and there's no "small cube" and no way to dismiss the about dialog either. The background looks tantalising but maybe a little more testing is needed... (e.g. for the most common browser/OS in the world).
- akanet 2y agoWow surprising, do you have any console errors? I tested extensively in Android chrome. To dismiss the modal you can tap in the margin.
- IshKebab 2y agoI'm not sure how to see console errors on mobile (without the remote dev tools anyway). I tried again today and it's still really slow but I do get the controls this time so I was able to use it. Very neat! This is on a Pixel 8, Chrome 113.
- 8n4vidtmkvmk 2y agoWorked ok for me on Android Chrome. Pixel 7
- mhandley 2y agoWorked for me on a 2.5 year old OnePlus Nord 2T, both in Chrome and Firefox. Not a high frame rate, but perfectly usable, even on this pretty old mid-range phone.
- moondev 2y agoIt runs excellently on my nothing phone 2. What phone are you on?
- rd 2y agoDoes anything like this exist for just flying around cities (not SF, anywhere) in general? Would love to experience what a drone sees even if it's in a limited area.
- dag11 2y agoMore conventional mesh-based photogrammetry options include: - https://maps.google.com https://maps.google.com (satellite view) - https://earth.google.com https://earth.google.com (also in browser, possibly better camera controls for what you want) - Bing Maps (3D flyover mode, more stale data in my experience) - Apple Maps satellite view (only on macOS/iOS) - Google Earth VR[1] (requires a Windows PC and a VR headset that can connect to it) - Microsoft Flight Sim 2020/2024 (requires a beefy Windows PC, uses Bing Maps plus a lot of other enhancements and rendering goodness. Most lifelike "feels like I'm there" but not true to earth) I'm not aware of splat-based city photogrammetry aside from one-offs like this but I'd love to learn if there's any such projects! [1] https://store.steampowered.com/app/348250/Google_Earth_VR/ https://store.steampowered.com/app/348250/Google_Earth_VR/
- whycome 2y agoThere are a couple okay apps on the Meta Quest that use the Google Earth API and you can fly around. It's neat. Wooorld. Fly. EarthQuest.
- wst_ 2y agoI think it would help to be able to move up and down (ex. space and shift keys) without changing the camera angle. This seems a better solution, to me, than existing pan with 3rd mouse button.
- akanet 2y agoYou can use Q and E
- michaeloder 2y agoFantastic work. This is one of the best gaussian splats I've experienced. Especially in regards to the distant objects and sky. I was surprised at how many more details I could perceive in the VR mode. I couldn't spot the "easter egg" until I switched over.
- legitster 2y agoCompletely unrelated, but aerial views of San Francisco blow my mind with how under-zoned the city is. One of the most desirable places on earth to live and it's on a small peninsula. Yet it's a sea of single-family homes as far as the eye can see. The distance between Sutro Tower and the "Downtown" SF is less than the distance between the Brooklyn Bridge and Central Park. But could you imagine if that space was filled with 2-3 story townhomes?
- cbeach 2y ago> One of the most desirable places on earth to live and it's on a small peninsula. Yet it's a sea of single-family homes as far as the eye can see. Ever wondered if it’s desirable BECAUSE it’s not a dense urban jungle?
- dangoodmanUT 2y ago[flagged]
- kaonwarb 2y agoYes... but the climate and geography alone would make it highly desirable at, say, 10x density.
- gcapu 2y agoMaybe you're right, but we'll never know. It would be great if they allowed some sections to develop so we can test it out. To me it is a desirable location because of the companies, not the lifestyle.
- mhh__ 2y agoThere's a difference between urban jungle and (say) nice 5 story European city buildings. Londons problem for example is that they tried to be clever and cheap at the same time in the 60s and now we're stuck with it.
- scoofy 2y agoThis is why incrementalism is always the best method of development. Spread out the pain so everyone only suffers a little. Spread out the development across different architectural eras. Spread out density to the point where you have diminishing returns. The city shouldn't be changed overnight, but the city should be allowed to change an a consistent rate that slowly accelerates. A good example is to allow each building to only double the square footage of the median building within, say, a quarter-mile radius of the property being redeveloped. This means that SFH's can only become duplexes until duplexes are the norm. After that, quad-plexes can be built, and then when that's normal, you start building large, eight-unit, european-style flats. This allows different areas to grow at different rates, while allowing density to remain generally uniform across neighborhoods. This incentivizes people who very much want low density to have a reasonably, predictably low-density neighborhood to invest in, while giving up the ghost when a piece of land is just to valuable to reasonably keep low density. It would work, and would work quickly in areas where lots of development is needed.
- fuddle 2y agoThat's super cool, I like how they explain what each antenna is used for.
- mcstempel 2y agowow, this is wonderfully made
- aqueueaqueue 2y agoBeautiful. Slow on my phone, but it is doing alot!
- toephu2 2y agoTen television stations, three FM radio stations, and 20 wireless and mobile communications users (i.e. law enforcement agencies, taxi cabs, school buses, wireless internet, etc.) rely on Sutro Tower antennas to transmit signals over the air to the entire Bay Area.
- roughly 2y agoFor some reason it never occurred to me that Sutro was still a live radio tower - it’s such an SF landmark that I think I just assumed it was decommissioned or something.
- bastawhiz 2y agoThis is great! Back in 2009 or so, I took dozens of high-resolution photos with my digital camera from the observation spot at Sutro Tower (towards the city, not the tower), and combined the images together in Microsoft Photosynth [0] to create an astoundingly high resolution point cloud of the city. I started with lots of zoomed-out photos, then took an overlapping grid of photos at various zoom levels. I wish Photosynth was still around; I'd love to look at the result again. [0] https://en.wikipedia.org/wiki/Photosynth https://en.wikipedia.org/wiki/Photosynth
- akanet 2y agothe tower released some gigapixel imagery from the top at https://explore.sutrotower.com/ https://explore.sutrotower.com/
- DonHopkins 2y agoDecades ago a radio technician friend of mine took me up the elevator in the west leg to the top platform, to enjoy the fantastic view of the city. As the caption at the base of the west leg elevator entrance says, it was quite small and cramped indeed, and it definitely was disconcerting when it changed orientation passing through the waist of the tower!
- bentt 2y agoThat's wonderful, nice work! Love the ambient audio track.
- armchairhacker 2y agoHow feasible would it be to generate high-quality guassian splats of everywhere, starting with big cities, like Google Maps street view and 3D view? Then you can make video games and interactive experiences in real-world locations. I doubt the collision handling is there, but you can at least start with something like Microsoft Flight Simulator with low-flying drones. It would also be great for training AI to generate realistic-looking scenarios. Besides playing and working (ex: VR) in real places, you can in very-realistic liminal spaces. (And it's training on public areas, so less ethical issues.)
- akanet 2y agothe collision handling part is actually not that hard to back into from the splat data. i think HQ splats of everywhere is something we'll start seeing in the next decade! we haven't quite solved scaling issues, LoD, and streaming for splats but the early work is promising
- jg0r3 2y agoCould you direct me to resources about collisions and splat data? I have an extreme interest in deriving measurements from splat data of vegetation, as it tends to reconstruct thin planes like leaves far better than other traditional SfM techniques.
- ipsum2 2y agoThey already are using NeRFs for Street View. (NeRF is similar to a Gaussian splat with different rendering trade-offs) https://blog.google/products/maps/sustainable-immersive-maps-announcements/ https://blog.google/products/maps/sustainable-immersive-maps...
- keepamovin 2y agoThat is so cool! I really wanna do a project like this just for fun to understand it
- rsch 2y agoThat looks rad. One bit of feedback: don't move the camera if someone clicks one of the circles, that is super disorienting. There is also a bug that if a drag to move the camera happens to end on a circle, the popup opens.
- ghfhghg 2y agoIt doesn't seem to load on Android Chrome or Firefox. Maybe a hug of death?
- mortenjorck 2y agoAs a child of the 90s, I see this as one of those rare, genuine examples of the “museum in cyberspace” imagined by the futurists of the day. Thank you for keeping the dream alive!
- biofox 2y agoGlad I'm not the only one who thought of that. On opening it, my mind went immediately to the 3D virtual tours in Encarta.
- aa-jv 2y agoIndeed, you might be interested in the Artificial Museum project, which has managed to realize that ol' cyberpsace dream all over the place: https://artificialmuseum.com/ https://artificialmuseum.com/ For example, check the Vienna map .. so many interesting locations!
- gertrunde 2y agoIn the same vein, it reminds me of the 1980's Domesday project, which had some sections that were similar to this, although given that it was published in 1986 , it was pretty much point and click to move between static photographs. [1] https://en.wikipedia.org/wiki/BBC_Domesday_Project https://en.wikipedia.org/wiki/BBC_Domesday_Project
- czbond 2y agoThank you for posting this - really cool project. Also, thank you for sharing your splat experience and methodology.
- merillecuz56 2y ago[dead]
- gameofby 2y agoExcellent!
- chriscjcj 2y agoI had the privilege of spending about 30 minutes at level 6 in May of 2023. It was as spectacular as you might imagine. I would say that if you have even the slightest fear of heights the whole experience would be a nightmare.
- arjvik 2y agoHow’d you get this opportunity?
- chriscjcj 2y agoI work in broadcast TV in San Francisco and am very good friends with one of the engineers who is responsible for the care and maintenance of some of the facilities up there. We talked about him taking me up there for ten years before we finally got around to it. :-)
- keepamovin 2y agoThis is great but please implement mouse look! It makes it way more immersive and is the lingua franca of FPS interactions.
- akanet 2y agoYou can free look with the right mouse button
- keepamovin 2y agoRight, or with multitouch gestures on trackpad but I want true mouse look. Better. Pleeease! You can gate it with pointer capture API but please do it!!!!
- archagon 2y agoMy favorite structure in SF. I hope I get the chance to visit the top deck someday.
- mihirsahu 2y agoThis is beautiful. I didn't know about gaussian splatting, seems like really cool tech
- jmux 2y agoThis is really cool, nice work! Something I’ve been looking for for a while is an interactive view of SF from above - I think it’d be cool to experience the verticality of SF and see how all the different hills relate to each other (and gmaps/earth just isn't cutting it). this is actually pretty good for around the panhandle, but if anyone is aware of something like that for the whole city please let me know!
- nickvec 2y agoThis is awesome. Have been enamored with Sutro Tower since I moved here a few years back. Love that you can see what the different antennas are for as well.
- scottrogers86 2y agolove it. love this city, too <3
- 01HNNWZ0MV43FF 2y ago30 MB! Very cool! It runs about 0.5 FPS on my iGPU but alas... good work anyway
- jasonjmcghee 2y agoVery interesting. It runs seemingly flawlessly on my 3.5 year old iphone.
- jcarrano 2y agoRuns perfect on my iGPU, no idea how many FPS but surely enough.
- deanmoriarty 2y agoVery beautiful and nostalgic, so well done. I used to live nearby and my favorite “urban” hike was going up Glen Canyon, up and down the two Twin Peaks, loop through Sutro Forest and then go back to downtown walking down 17th. Loved those weekend walks.
- teeeg 2y agoHa yes love that loop, so strange to see someone else tying it all together! Not quite as good but when I lived in the sunset — grand view to golden heights along the ridgeline then down to Laguna Honda trails and up to twin peaks and out sutro forest to Parnassus.
- boguscoder 2y agoLooking under the surface has its own charm
- dmazin 2y agoSan Francisco is just so beautiful. There's a serenity to it that I can't quite put my finger on. The way the fog waterfalls over the hills when you take the train in from the south... the view from Sutro. The gum tree trove near the tower. This captured that serenity.
- archagon 2y agoVincent Woo (the author) also has some lovely time lapse videos of the fog! (And other bits of the city.) https://www.youtube.com/playlist?list=PLXhk-g2kj99k6FMp2tZ2XrJTAulZTu2cs https://www.youtube.com/playlist?list=PLXhk-g2kj99k6FMp2tZ2X...
- Lorin 2y agoSadly the controls are not mobile friendly and I ended up spinning wildly.
- kiririn 2y agoWorks fine on iOS safari, two finger drag to move
- kevthecoder 2y agoThe Metaverse Standards Forum has had some activity around gaussian splats recently, for example debating whether it's too early to standardise. There's a town hall on 5th March with speakers from Niantic and Cesium: https://metaverse-standards.org/event/gaussian-splats-town-hall-part-2/ https://metaverse-standards.org/event/gaussian-splats-town-h.... The previous splats town hall, and other related talks, are on the videos page (there was another gaussian splat talk a couple of days ago from Adobe). https://metaverse-standards.org/presentations-videos/ https://metaverse-standards.org/presentations-videos/
- Luc 2y agoI wasn't expecting to be able to see through the lattice structure. Amazing amount of detail. Naively, it seems to me that the many needle-like artifacts further away from the tower could be filtered out?
- desdenova 2y agoThe help text mentions a "little cube" that would enable AR mode, but I can see no cube. Touch controls are very weird and don't rotate the view as expected.
- andybak 2y agoit so nearly runs on a Quest 3. The next gen of mobile XR chipsets (or maybe even the current gen with less thermal throttling) are going to be able to handle these kinds of scenes with a bit more optimization.
- akanet 2y agothere is still some perf headroom in software to get, too!
- zokier 2y agoWhat is the limiting factor for level of detail here? Is it the source data (too low res camera/too far away), the processing, or getting it rendered in real-time in browser? Is this the full detail, or is this somehow downsampled/compressed version for the web?
- akanet 2y agoThe processing and alignment of source imagery, imo
- FinnKuhn 2y agoI can also recommend "Tunnel Vision: An Unauthorized BART Ride", which was made by the same author and is a really great documentary film. It's free on YouTube: https://www.youtube.com/watch?v=3-Jrp6it9Ss https://www.youtube.com/watch?v=3-Jrp6it9Ss
- franky47 2y ago"Good morning, and welcome to the Black Mesa transit system" is the first thing that popped in my head when the train started moving.
- chadd 2y agoI'm biased bc I worked with the team while there, but I believe Snap's acquisition of PlayCanvas was one of their most underrated. Incredible technology.
- cloudfudge 2y agoThis brings a meta quest 3s to its knees. It's almost so bad you can't quit it, and the video lags 15 seconds, which is very disorienting to be immersed in (it can make you fall down). Shame, since it looks gorgeous.
- poutrathor 2y agowhat part is the culprit : the meta quest or the 3D implementation from the website ? On a classis laptop, it behaves well
- PaulHoule 2y agoThe quest is underpowered (it's basically a midrange Android phone you wear.) More efficient coding or a simpler model would help. Inside the Scaniverse does something similar with a high framerate but the models are simple and don't look very good. I had to power cycle mine to get out, but boy was the view great despite the motion sickness.
- ladon86 2y agoThe Quest's Snapdragon GPU, like most mobile GPUs, uses a tiled rendering [1] architecture. The basic technique for rendering gaussian splats is kryptonite for this architecture, essentially implementing every worse practice for rendering on a mobile GPU: * Tons of overdraw (overlapping splats) * Tons of alpha blending * Millions of splats in the distance generate a lot of tiny triangles resolving to a single pixel * Long thin splats in the foreground generate triangles that cover multiple tiles These are all the ingredients you need to bring a mobile GPU to its knees! Any desktop GPU (including most laptops) will be far less sensitive to these issues, even if it's not very powerful. It's a fundamental issue of architecture rather than one of raw FLOPs. [1] https://en.wikipedia.org/wiki/Tiled_rendering https://en.wikipedia.org/wiki/Tiled_rendering
- OrangeMusic 2y agoIt _is_ possible to render incredible and gorgeous scenes using gaussian splats on the Quest platform though, as shown by this tech demo from Meta: https://www.youtube.com/clip/UgkxGlXM3v93kLg1D9qjJIKmvIYW-vHvdbd0 https://www.youtube.com/clip/UgkxGlXM3v93kLg1D9qjJIKmvIYW-vH...
- PaulHoule 2y agoAmazing. I tried it on my MQ3 last night and it was the first thing like that which was photorealistic, but it badly overloaded the MQ3, so it was the closest experience to Sword Art Online I've had yet in VR. (The sky was transparent and my room showed through!) I should have been sitting when I started it but since the frame rate was low and the horizon improperly oriented I could have fallen transitioning to the couch if I hadn't steeled myself to rely 100% on proprioception. Contrasted to the way too lo-fi Inside the Scaniverse and the bland but cringe Horizon Worlds it's a hit. I gotta try it again in tethered mode.
- cytocync 2y ago[dead]
- accrual 2y agoThis is very cool. I feel like the technique used (gaussian splatting) gives it a more realistic appearance from certain viewpoints. It almost felt like my monitor turned into a window in the sky looking out over the city for a moment. The illusion falls apart once once drifts too far from the nominal viewing area, but until then, it looks even better than something like Google Earth which renders actual polygons for everything.
- jcarrano 2y agoNot only does it look amazing, it renders super fast even on an iGPU. Feels like magic. A couple of questions: Why do objects flicker when moving the camera? And why do surfaces get translucent when close up?
- kevinwang 2y agovery nice, vincent!
- pppoe 2y agoCan anyone give some numbers for a more intuitive understanding of the advantages from GS? How large would the file/content be if it is in mesh? Can we get similar rendering FPS?
- tuckerpo 2y agoExponential back-off while zooming in is nice, but maybe reset if scrolling back out.
- skeeter_sky 2y agoThe background music seamlessly blended with the chorus of Nina Nesbitt's song 'On The Run,' which was playing when I opened the site. I genuinely didn't realize there was any background music at all.