7 ms·
ViziCities Dev Diary #1: Visualising Cities in 3D using WebGL
- matthojo 14y agoBeautiful.
- robhawkes 14y agoThanks Matt!
- chinchang 14y agoIts really inspiring to go through the dev stages you guyz went through while building it. Thanks for sharing :)
- robhawkes 14y agoAppreciated! There's still a whole bunch left to do but we're really happy with how it's turned out after just 1 month.
- rohern 14y agoThis is a terrific blog post. Excellent project and excellent sharing. I feel like we have a lot of games set in real world cities to look forward to. Also: autonomous vehicle pathing?
- robhawkes 14y agoThanks! The vehicle path side of things is something I'd love to explore in more detail at a later date. Right now it's all rather basic but it could certainly be a whole lot more interesting.
- rohern 14y agoConsidering the incredible amount you have already achieved, I am optimistic.
- aw3c2 14y ago> If I'm to be honest, after compiling and writing this entry I'm actually in a state of disbelief about how much we've managed to get done in such a short space of time. That is the magic of having and utilising the power of free and open-source software. Imagine not having three.js, QGIS, PostGIS etc. Gorgeous work and a highly interesting "how we are doing it" insight. Thanks! Please more!
- robhawkes 14y agoYou're entirely right. What I've learnt most about this project is that by utilising and combining free data and software you can quite quickly knock up something amazing. However, free and open stuff doesn't fix performance and all the little detailed issues that took up most of our time this past month. I'm glad you like it, we'll make sure to write up more as we progress.
- camus 14y agoBy the way , i'm reading your canvas book and making a framework out of it ;) , though i'm not using javascript direclty but Haxe. Keep on the awesome work.
- robhawkes 14y agoThat's awesome! Always chuffed to hear about people benefiting from the book. And likewise, sounds like you're working on cool things too ;)
- kmfrk 14y agoYou guys are friggin' wizards.
- robhawkes 14y agoHaha, thanks!
- ameen 14y agoExcellent! Love the blog post. And ViziCities already looks beautiful. I can only imagine how wonderful it's going to be once a lot more vectors (population, vehicles, weather, energy, etc) are added in. Would a simulation component be added in? Cause if it is, It'll be nothing short of awesome. All the best!
- robhawkes 14y agoThanks! We've been thinking a little about simulation and, although not likely for the first release, it's something we'd love to explore in the future. It's not a huge step to go from our basic AI to a full-on simulated game-type world. We're not ruling anything out.
- chime 14y ago> Unfortunately, in our efforts for global domination we quickly discovered the performance limitations of the project as it stood, as well as my personal knowledge with Three.js and WebGL. In short, the expanded version of London ran at a horrible 20–40fps depending on the system you viewed it on. You got a lot farther than me when I tried Three.js before hitting performance problems. I was trying to make a WebGL 3d warehouse visualization app: http://chir.ag/projects/wh/ http://chir.ag/projects/wh/ and stopped because it was too slow once I loaded 1000 bins. I also never quite figured how to handle rotation/panning nicely (src: http://chir.ag/projects/wh/main.js http://chir.ag/projects/wh/main.js). I came up with a shorthand http://chir.ag/projects/wh/bins/wh.txt http://chir.ag/projects/wh/bins/wh.txt that expands to http://chir.ag/projects/wh/bins/bins.php http://chir.ag/projects/wh/bins/bins.php - I was going to make it an easy to use app where you could describe how the warehouse was organized with 20-30 lines of bin#s and positions and let the system generate the rest. Then it could parse APIs containing various properties per bin (full/empty, inventory amount, frequency of usage, days-since-last-used etc.) Clicking on any bin would open up a panel with more information and you'd have filtering/highlighting capability for the whole warehouse. I made good progress initially but got busy with life and never got around to solving the too-many-bins problem. I still don't know how to improve the performance. Even when entire objects are outside the view, they still take up resources and time. I see you mentioned LOD support in Three.js - no idea if it would help me or not because each object is already as simple as it can be - just a box. If someone wants to hack on this and make it functional, I'll help however I can.
- robhawkes 14y agoAny chance you could put that up / a version of it up on jsFiddle or something so I can edit it online and see how I can help? There's a few techniques that I could play with. 1000 objects should be silky smooth…
- chime 14y agoAbsolutely! Give me a few hours and I'll have it up. Thanks for the offer. Also it ended up being 6000 objects because a box is 6 planes. I wanted to reduce that to only visible planes but because boxes could be partially translucent depending on some property and be selectable when clicked, it would look weird if inside sides were missing.
- gmac 14y agoLooks awesome. The Kinect could be a fun way to explore your visualizations -- seen here hooked up to Google Earth: https://github.com/jawj/pigeonsim https://github.com/jawj/pigeonsim Also, I'm sure http://www.bartlett.ucl.ac.uk/casa http://www.bartlett.ucl.ac.uk/casa would love to hear from you.
- robhawkes 14y agoThanks George. I replied on Twitter but worth saying here also… I saw Pigeon Sim via Andy @ CASA the other day, it looks like awesome fun! We'd also like to play with Leap Motion and create something similar to the Google Liquid Galaxy stuff. Quite keen to work with CASA, I'm sure there is a tonne that we can help each other with. I told Andy about the project a while back but that was before we had anything visual. I'll try pinging Andy again, unless you have another suggestion?
- chime 14y agoLeap Motion has JS plugin so it would be super easy to integrate with it.
- robhawkes 14y agoLet's hope so! I've applied for a dev kit so hoping I get accepted.
- gmac 14y agoYep, @digitalurban is still your man at CASA -- and @frogo (who now maintains Pigeon Sim) too. I'm now only honorary there, but keep up an interest.
- skore 14y agoCrossing my fingers real hard for somebody to pick this up eventually to make a no-nonsense FOSS SimCity clone out of this. Please, please, please.
- robhawkes 14y agoThat would certainly be kinda cool. My recent background is in HTML5 games (I used to work at Mozilla) and I'd love take what we've done so far with ViziCities and see how a game would look.
- skore 14y agoDo I hear a kickstarter coming? ;-) Holler if you need somebody to make some music. Heavily into making ambient stuff right now anyways...
- robhawkes 14y agoHadn't considered it. If we need the money we may give it a go! Re: Music. Do you have any examples of your stuff anywhere? We'll be having music and effects of some sort but haven't yet looked into it much.
- skore 14y agoSure, you can check out some of my stuff on soundcloud: https://soundcloud.com/sk0re https://soundcloud.com/sk0re It's somewhat mixed, but there is a good amount of ambient stuff in there as well. (Also note that almost all of it - except for the one remix - is CC licensed.) Would sure love to help out any way I can!
- mcrider 14y agoI've been thinking about this a lot lately -- What's most interesting to me is the implications for urban data modeling and prediction when the end user has full control of how the data is used. Remember that article a few weeks back about the fellow who used the new SimCity to model his town's traffic problems? Imagine if you had full control over the model and can tweak the city at any level you want, not just what the game UI allows you to adjust (e.g. feeding in data from a government API)? Urban planners could potentially get a lot of useful information from gamers playing around with variables in their city. Another interesting idea could be turning a city into a sort of MMORPG where each user has control over a smaller section of the city (instead of the whole thing). If the user only pays attention to optimizing a small part of the city, interesting patterns could emerge in the overall model.
- bvdbijl 14y agoWow, very interesting, this tickles my fancy... Imagine something like this but with an airfield, you could model the airfield and every airplane landing and getting up in real-time.
- madlag 14y agoReally amazing to get this after only one month, when you know how difficult is building 3D stuff !
- robhawkes 14y agoThanks! It's definitely easier when you're standing on the backs of free software (like PostGIS and Three.js) but the 3D aspect has certainly been tricky.
- LAMike 14y agoThis is amazing! It was cool to see the process of turning data points into a 3d city.. Combine this with the Occulus Rift and the Singularity won't be that far away :)
- robhawkes 14y agoIf only I could get my hands on an Occulus Rift. We'd love to do some cool things with it!
- rurounijones 14y agoMy god, that is amazing and wonderful. You should be hired by the uk.gov team to work on this full-time. This, combined with various statistical information (crime, traffic, other stuff from the ONS) would be a huge boon to anyone using it and the interface makes it beginner friendly.
- robhawkes 14y agoThanks! We'd love to talk to the data.gov.uk guys about working together, though I have no contacts there atm. We do plan to visualise huge quantities of data, including the datasets you mentioned. Most of what we've been using so far is from the 2011 Census, as well as the Indices of Multiple Deprivation. Traffic too, we're looking into using the live data from TFL to do some cool stuff.
- rurounijones 14y agoThis is exactly the sort of thing the whole "Show us a better way" site was supposed to encourage. I took a stab at some stuff using various sources but didn't have enough time (Edit: or skill, or access to data sources). While I appreciate some of the game related stuff this looks most promising for allowing people to easy see statistical information at-a-glance, data journalism etc. The NapTAN datbase would be an awesome addition that should be relatively easy to add. Regarding the data.gov.uk team. They do have a twitter account and website, hit them, hit them hard. Even if they can only smooth the way for data access (Like the building heights) that would be awesome.
- robhawkes 14y agoThanks for the advice, we'll definitely follow up on it! And I agree, the data side of things is very interesting and is something we're keen to explore as a separate entity to the playful/art aspect.
- dirkk0 14y agoI am working on something similar with the prospect of getting CannonJS physics to work so that one can walk the city (in my case procedurally generated): https://twitter.com/dirkk/status/309024330115842049 https://twitter.com/dirkk/status/309024330115842049 I also tried a multi-device approach for visualisation: http://www.youtube.com/watch?v=-TiZzovx5YE http://www.youtube.com/watch?v=-TiZzovx5YE Any chance of collaboration? From this thread I reckon there are quite a few people working on something similar. I guess it would be extremely cool if we could join forces.
- robhawkes 14y agoVery cool, thanks for sharing it! The multi-device thing is something we're actually keen to explore; we basically want to replicate the Liquid Galaxy thing that Google did. We haven't yet worked out what we're doing re: opening the project and collaboration but we're open to talk more. Mind reaching out to us via hello@vizicities.com?
- dirkk0 14y agoSure. Mail sent.
- arpineh 14y agoI would love to see more details about this. How did you implement your data path from getting GIS data (shapefiles I presume) to PostGIS then GeoJSON and finally into ThreeJS? What about combining the statistical data with the location? And the heatmap/choropleth, is that something Three.js can do? I have no knowledge of developing 3D stuff, but I have been looking into using D3 with canvas rendering just for this kind of visualizations. I'm playing with multidimensional data as stacked 3D cubes. So, getting something like this would really bootstrap my project to another level. Any thoughts about open sourcing? In all, this looks very useful and expendable. The sky is the limit. Literally, when is sky coming? ;)
- robhawkes 14y ago> The sky is the limit. Literally, when is sky coming? ;) Definitely one of my favourite comments about the project so far! =p The rough and ready approach we took to transforming data was to import the shapefiles into PostGIS using pgShapeLoader, doing any manipulation in PostGIS and exporting straight out to GeoJSON using the ST_ToGeoJSON method in PostGIS, or using the GeoJSON options in ogr2ogr. From there it's a simple case of importing the JSON into Three.js and constructing the objects from the vertex coordinates. Re: Combining statistical data with the geographic data, we can do that with GeoJSON and you can use ogr2ogr to include related data about each location when exporting to GeoJSON. The choropleth map is something we constructed from scratch. In reality, it's nothing more than a bunch of shapes in Three.js that each have a different material colour – it's very simple. D3.js is a fantastic place to start. In fact, that's exactly where we started with all this. We actually use the Geo module within D3 for a few parts of ViziCities. We haven't decided yet about opening the full project but we'll certainly be documenting and releasing aspects of it (most of the juicy bits). We'll announce that via the @ViziCities Twitter account. As for the sky… it's coming ;)
- arpineh 14y agoAh, thank you for your reply. There was a WebGL 3D flying simulator on HN few weeks back which used the sky and light changes to great effect. Perhaps something like morning rush hour traffic against evening or rainy days againts sunny. I'll definetely have to delve more into PostGIS. Three.js should follow when I get a better graps of D3. I think there was an example on how to generate a satellite 3D like map from GeoJSON at Mike Bostocks example site. Very exciting stuff!
- a_n_onymous 14y agoFWIW, Nokia L&C does not have "massive development teams" working on HTML5 visualization of geo-data, quite the opposite...
- robhawkes 14y agoWhile I know you're right (I was being over zealous), I'm quite sure it wasn't 2 guys in their spare time with zero budget.
- Tycho 14y agoThis should be used as some sort of case study in CS/IT courses. "How to be incredibly friggin' awesome."
- robhawkes 14y agoHaha, let's make it happen!
- n3rdy 14y agoAre those clouds I see in that blurred image? Are you teasing us with the prospect of 3d weather (perhaps by using radar data)? If not that's cool, but if so, great idea.
- robhawkes 14y agoMayhaps ;) But seriously, weather and night/day cycles is something we're looking into.
- olalonde 14y agoIf you haven't already, you guys should absolutely get some inspiration from Baidu's amazing 3D city maps. http://j.map.baidu.com/m8vSh http://j.map.baidu.com/m8vSh
- robhawkes 14y agoSome of the maps for China are stunning, we've definitely been using few of them for inspiration.
- AndyBMapMan 14y agoThe Ordnance Survey doesn't even hold height data on all buildings in the UK. They've started a few isolated projects but nothing as a deliverable dataset yet. I don't think there's a US equivalent despite your link to NYC Open Data. You might want to look to Germany and CityGML. You might discover some town datasets that are complete in height and geometry detail. If you happen on a LIDAR data source you've then got to tidy up against the buildings but you will run into errors caused by the time spent between the two data capture points. There's SRTM but i'm sure it'll be too coarse for your needs. I'm sure there's a replacement to SRTM but the name escapes me. There's LandXplorer - now owned by Autodesk which will generate models from oblique aerial photogrammetry. There's also CityEngine that'll create fake cities based upon road networks. @AndyBMapMan.
- robhawkes 14y agoThanks for the reply Andy, appreciated. We're actually talking to Ordnance Survey atm so hopefully we can work something out. Building height data for the UK (that's available outside higher education) is a pain to get hold of so I'm hoping we can work something out. I'll definitely look into the CityGML stuff, it sounds great. LIDAR is certainly something we're keen to explore further. If anything, it'll get us a much more accurate estimation of height, at least better than an educated guess or via SRTM (which is way to low detail and can't be relied on for buildings). The CityEngine stuff is amazing. I've not actually played with it myself yet but I've been having a good look at it during my research. Would be cool to have a go on it at some point.
- superqd 14y agoGreat project. Looks really nice.
- robhawkes 14y agoThanks!