12 ms·

Dynamicland

297 points by andyjohnson0 6y ago

sparsely 6y ago
It feels like this has been reposted regularly for years, but the demo videos and slightly threadbare use cases haven't changed at all. Re-imagining how we interact with computers sounds great, but it doesn't feel like this project is actually going to achieve it's goals beyond letting a small number of lucky users play around with stuff. I'm not sure if darklang is going to succeed, but that seems like a far more fruitful direction to approach the problem from. It's a very hard problem, and they are attempting to make programming a little bit more visual while not removing any power from the user, instead of making it entirely visual and almost completely dis-empowering the user. Importantly, it's actually accessible to people all over the world, and they can use it to achieve real world goals.
- scroot 6y ago
  This issue here is research funding
- crazypython 6y ago
  "2020: Realtalk-2020. We're bringing together everything we've learned to create the next iteration of our Realtalk computing system. Realtalk-2020 will form the foundation for the next decade of research and applications." "2022: Dynamicland meets the world, in the form of new kinds of libraries, museums, classrooms, science labs, arts venues, and businesses. We will empower these communities to build what they need for themselves, to design their own futures."
- bschne 6y ago
  > ...but it doesn't feel like this project is actually going to achieve it's goals beyond letting a small number of lucky users play around with stuff. Will they end up building some product that will be adopted by droves of customers and offers a completely new paradigm for interaction? Probably not. But I'd argue that's hardly where "esoteric" research like this ends up going, and in my book that's OK. Bret Victor, who is behind Dynamicland, never shipped the full drawing app from his interactive visualizations talk as far as I know. Neither did anyone ever get to buy or download the editor he shows in "Inventing on Principle" as a new IDE that offers incredibly great feedback while programming. Nevertheless, his talks are among the most inspiring things I and I think many others have ever seen in the area of HCI, and at least in my case are responsible for a large part of my renewed interest in the field. Will anything out of Dynamicland capture people's imagination and enthusiasm like that as well? Maybe. Maybe not. But the point is to explore, and I applaud those who do.
  - endergen 6y ago
    The influence of his talks show up everywhere and are explicitly called out as inspirations. Elm Lang's time travel debugger, therefore redux, hot reloading work, Observable calls him out as influential, and on and on.
    - bschne 6y ago
      Exactly my point — there's space for those who put together amazing but infeasible-as-product demos to communicate lofty ideas, those who take those and repackage them for "mass consumption", and of course the rare genius who does both, and anything in between.
      - IshKebab 6y ago
        I feel like putting together a janky tech demo is not especially impressive though. Anybody can think of screens projected onto paper. It's not particularly interesting that the tech demo is possible. Actually making it usable is the really hard part and the reason it remains a tech demo. That's why Apple is rich. They do the hard part.
- endergen 6y ago
  I personally hope that the output ends up being systems that are good for individuals with their own dynamic lands at home, not just strong group collaboration. I know there are already people who have setups done in coordination with the Dynamic Land team. I've visited dynamic land also, and the vision of this more tangible visible computation is bigger than the incarnation/progress last shown. I'm a fan of Darklang's goals too. But I do think Dynamic Land is targeting more approachable, teachable, and communal computing. Darklang is going to be more for professionals making backends and doesn't target any user facing audio/visuals whatsoever. It could in the end of course, so I hope it does well as an effort too.
  - pbiggar 6y ago
    > Darklang is going to be more for professionals making backends and doesn't target any user facing audio/visuals whatsoever. It could in the end of course, so I hope it does well as an effort too. That is the plan, might be a while away though.
- beaconstudios 6y ago
  I hope darklang succeeds too, but I personally believe that visual programming is at least partly the way to go. This is because I firmly believe that the "most-correct" view of regular programming is that of a directed graph of computations, and directed graphs don't compress very well into text. I'm building something visual to prove (or disprove) this concept, but I'm also thinking that power users would want a textual language for faster input, as GUIs are better for output where TUIs are better for input.
  - sparsely 6y ago
    I agree that visual programming can be great - my argument would be that starting from a model that we know works and making incremental steps towards something more visual is more likely to succeed, than attempting to jump all the way to something fully visual based
    - beaconstudios 6y ago
      in general I agree that incrementalism is the best way to improve a model. I'm just not certain how to do that with such a drastic change as going from textual to visual - but darklang seem to be doing a great job at that, and I do hope they succeed at their mission.
      - pbiggar 6y ago
        Thanks! We're very much of the opinion that we got into this mess we're in (gestures broadly) because everything has been incremental to date. That's how you get layers upon layers.
        beaconstudios 6y ago
        Yes 100% - my founding hypothesis was that because language progress so far has all been improvements on opcodes, that the idioms of programming are still based on the human catering to the computer rather than the other way around. It gets subtler over time, but we still have to, for example, emulate execution in our heads while coding, which is really really hard to learn.
- crazypython 6y ago
  How is darklang related to dynamicland? I only found https://darklang.com/ https://darklang.com/
  - theon144 6y ago
    Right, it honestly feels like the parent commenter either misunderstands Dynamicland's aims, or is willfully drawing comparisons to make it seem like the projects are tackling the same issues, out of which Dark comes out as the more successful one (at least measured by "adoption"). I don't see what the "almost completely dis-empowering the user" is about either; if anything, Dynamicland seems the more empowering of the two, whereas Dark's endgame is more efficient development of a fundamentally conventional type of software. It feels rather dishonest; the comparisons are sweeping and vague, with very little substance as to an actual criticism.
- bmitc 6y ago
  > instead of making it entirely visual and almost completely dis-empowering the user Why do you think that making something visual completely disempowers the user?
_Microft 6y ago
Is the technology of detecting and using marked everyday items (sheets of paper/whatever) as input devices and parts of programs patented? I couldn't find anything on that yet. It feels like something that would benefit from being patents-free.
- jackcviers3 6y ago
  It was used in Killer Game Programming in Java by Andrew Davidson published in 1995. That used a Webcam and a color coded bracelet you made yourself as an input device for one of the code examples. Microsoft had Surface touchscreen 2.0 pre 2011 that tracked real world objects [1]. 1. https://youtu.be/57k2iJbotV4 https://youtu.be/57k2iJbotV4
rodiger 6y ago
The shadows and latency are rough currently but totally agree that single, flat, rectangular input devices shouldn't be the future. Tactility is amazing and a large reason people prefer their old "dumb" interfaces.
kumarvvr 6y ago
I think this is a better paradigm than Augmented Reality glasses. The medium is limited but interesting. Perhaps this would be more portable if there is a projector underneath a table with a transparent glass top and a camera on top to view inputs. Potential Applications 1. Mapping (Interactive maps, interactive routing, real-time weather, etc) 2. Collaborative art 3. Engineering CAD and CAM modeling and design.
- porcc 6y ago
  Tilt5 might interest you as IMO the best paradigm of AR glasses--headmounted projectors with retroreflective surfaces. https://www.tiltfive.com https://www.tiltfive.com Portable, multiplayer--and comes with massive improvements in FOV and contrast due to its design.
- gmueckl 6y ago
  The mythical AR glasses of the future should be able to do everything your proposed projection table setup can. I see both setups as just two different manifestations of AR as a concept.
  - kumarvvr 6y ago
    Haha. Mythical indeed. But more importantly, they are impersonal. In a collaborative setting, they are simply not conducive enough.
    - indigochill 6y ago
      Are you thinking of VR? AR merges the virtual environment with the physical one. One fun example was that I played a Finnish board game sort of like Pictionary with a Finnish friend. All the cards were in Finnish which I don't know, but I could use my phone's AR translation app to "read" the cards in effectively real time. AR glasses could hypothetically share a networked virtual environment, so I don't see why they couldn't be collaborative.
      - kumarvvr 6y ago
crazypython 6y ago
This is the most "object-oriented" programming language I've seen.
anarticle 6y ago
To contextualize this, check out Seymour Papert's work on LOGO, especially teams of kids working together to solve problems. Some of this is also related to Alan Kay's work. It's all inspiring, and shows that there may be many ways to compute.
jkscm 6y ago
Games like Kerbal Space Programm and factorio exhibit a lot of the concepts from the original research agenda[^1] because they are "Dynamic Environments-To-Think-In" [^1]: http://worrydream.com/cdg/ResearchAgenda-v0.19-poster.pdf http://worrydream.com/cdg/ResearchAgenda-v0.19-poster.pdf
olah_1 6y ago
Has anyone used Scratch style blocks for real physical blocks? They fit together like puzzle pieces. Kind of a neat visual that could translate to physical medium.
- gremlinsinc 6y ago
  I'd love something like this for my kids... Maybe have a board with lego slots for blocks that represent flow and maybe a little bit of interfacing with a computer to assign graphics to an object, say a sprite then the blocks tell it how to move, you could maybe have blocks that represent data and you can name it but the data stored is just json so no real schema for simplicity.... You could create interactive stories from a board of legos, or game like have a squirrel escape a yard without running into the dog, etc...
gklitt 6y ago
If you're looking for more concrete details on what it's actually like to make things at Dynamicland, I recommend this writeup by Omar Rizwan. (from 2018, so doesn't reflect recent progress, but it's still helpful for getting a feel for the place): https://omar.website/posts/notes-from-dynamicland-geokit/ https://omar.website/posts/notes-from-dynamicland-geokit/ I also wrote a short explanation of some little experiments I did there when I visited a couple years ago: https://www.geoffreylitt.com/projects/dynamicland.html https://www.geoffreylitt.com/projects/dynamicland.html
- rtkwe 6y ago
  I wish there was more out there about this. Seems kind of neat despite the limitations but I haven't seen much new come from it since it was introduced years ago.
jarmitage 6y ago
For those wondering about an update, they posted a "Narrative Description of Activities" document (something to do with 501c status) recently (https://twitter.com/worrydream/status/1367894681342799875 https://twitter.com/worrydream/status/1367894681342799875), choice quote re comments in this thread: > The Dynamicland researchers are not developing a product. The computers that the researchers build are models: not for distribution, but for studying and evaluating. The goal of the research is not to provide hardware or software to users, but to discover a new set of ideas which will enable all people to create computational environments for themselves, with no dependence on Dynamicland or anyone else, and to understand how these ideas can be taught and learned. The end result of the research is fundamentally intended to be a form of education, empowering communities to be self-sufficient and teach these ideas to each other.
- tiglionabbit 6y ago
  Yeah but can we go there?
  - VonGuard 6y ago
    Last I checked, they were in downtown Oakland and you could just knock on the door.
thrower123 6y ago
This looks like the most vaporware thing I have seen since Duke Nukem Forever.
throwaway789256 6y ago
People who love Bret Victor's work and thought should know that he is also part of this effort.
- crispyambulance 6y ago
  Although I much admire Bret Victor, this Dynamicland project seems like a really, REALLY deep rabbit-hole. I hope he finds it enjoyable and rewarding, but I do hope re-joins the world and starts giving illuminating talks again!
adenozine 6y ago
What in the fresh hell! This is so awesome. I've always dreamt of something like this with some form of handwritten programming language, a group of people sketching a program on a single, giant canvas. I guess this isn't a new project, but I'm glad it's been reposted. I do worry that this format has some pretty sharp limits, just due to the spatiality and trying to cram functionality into a limited area of a room, amongst other people. Some form of code storage might need to be designed, so one could stack those little code papers on top of one another. Who knows. Very though-provoking project.
SamBam 6y ago
I went to a meeting at Dynamicland a few years ago, and got to spend a few days playing with it and chatting with people. The positives are many: I think it's awesome to conceive of computers outside the actual computer boxes, especially in an educational setting. I think the notion of "collaboration" is way more engaging when it's kids standing shoulder-to-shoulder and actually pushing pieces of paper around and arguing about things out loud, rather than "collaborating" on a Google Doc, sitting at different computers. I also think that this can be enlarged: all sorts of working meetings could be improved by people standing shoulder-to-shoulder at a table and pushing around tangible objects, with simple programming about how they'll interact. My negatives are what I took away from this, and may be incorrect or out-of-date, so feel free to jump in if I'm wrong, but I felt that Bret Victor was very much a purist regarding his vision, and had no desire to help spawn clones or variants of Dynamicland anywhere else. In many ways this may be laudable, but it felt like he was protecting his baby from going out into the wild, which has meant that the spread of ideas and possibilities has been greatly curtailed. It seems like it will only ever be destined to be a tiny playground for Bret and the few friends working with him.
- theon144 6y ago
  I see how that might be constructed as a negative, but IMO it's too early to tell whether that's a genuine setback to the project. "Protecting his baby" might be a very wise decision at this point, if only because of how the reception generally goes; pigeonholing the project into something like "an AR coding environment" or "visual programming with projectors" is a very real risk that could damage the project's aims - even "clones" such as https://paperprograms.org/ https://paperprograms.org/ make it abundantly clear that they are not attempting to be an "opensource Dynamicland". It's only been 3 years since it was founded. While that might be generally considered an eternity in tech-time, I feel it's barely enough to get one's feet wet given the scale of the project, which seems to aim to be decades long. Besides, the roadmap they've got on their website mentions 2022 as the year they go public - so, I personally am stoked for what that will bring.
  - sleepybrett 6y ago
    Might be wise but might not be, look at Xanadu, with the web we got a massive 'halfstep' without any xanadu hands on the wheel.
coloneltcb 6y ago
Great to see this on the front page. I've been lucky enough to visit Dynamicland/volunteer there and it absolutely lives up to the hype.
mwcampbell 6y ago
I sent the following message to Dynamicland a few years ago and never got a satisfactory response. Does anyone know if they're doing anything about accessibility for blind people or people with other disabilities? Hi, First of all, Dynamicland's goal of "agency, not apps" resonates with me. I'm in favor of things that make programming more accessible to normal people. However, I worry that Dynamicland will be a step backward for people with disabilities, particularly blind people and people with impaired mobility. For us (I'm legally blind myself), I believe the virtual worlds of today's computers aren't imprisoning but liberating. Consider that a blind person can't see that "fully functional" scrap of paper, and a person who can't use their hands can't write on that paper or manipulate the other physical objects in Dynamicland. Do you have a plan to solve this problem while holding to your goal of "No screens, no devices"? It seems to me that there's no way to reconcile these conflicting requirements without making an exception for people who can't work directly with the paper and other objects. Or have you decided that it's better to undo the equalizing effect of computers for people with disabilities, for the good of everyone else? I would obviously be disappointed if that's the case. But I understand that everything's a trade-off, and perhaps it's not reasonable to confine the majority to an inhumane way of working for the benefit of a few. So I don't mean this as an accusation; I really want to know. Thanks, Matt
- gugagore 6y ago
  I think your question is important, and I wish you had received a reply. I think it's common for people's needs to be in conflict; everything is a trade-off, as you acknowledged. But I think you baked into your question a false dichotomy. "Dynamicland" does not necessarily have to _undo_ the "equalizing effect of computers", nor does its absence mean that the majority is confined to an _inhumane_ (wow...) way of working.
  - mwcampbell 6y ago
    > an _inhumane_ (wow...) way of working. That was Bret Victor's characterization of being confined to a screen, not mine.
- chromanoid 6y ago
jude- 6y ago
Is it open-source yet? I can't seem to find any source code anywhere.
sesuximo 6y ago
Man that looks cool! I’d love to use paper and a projector instead of looking at a screen all day
jackcviers3 6y ago
Dynamicland feels to me like The Mother of all Demos. Ahead of its time, groundbreaking practical applications of interactivity research. Dozens of companies and projects will spring from this. Just give it time. We'll be back to using real objects instead of keyboards at some point in the near-future. Brett's right that a lot of the difficulty in actually using computers to produce things is in the translation of human interactions into a medium the computer can understand, and translating the data a computer outputs into a medium humans understand. We aren't two-dimensional creatures. The state of the art cannot remain in pictures and text forever. I'm not sold that everything will be done this way - some things will always need the precision of automated, textual input (see tool assisted speed runs in video-gaming if you think human beings can match pre-programmed precision inputs).
- twobitshifter 6y ago
  They actually cite Englebert at the top of the page as guiding the project spirit.
haolez 6y ago
I saw something like this in a presentation at the MIT Media Lab a few years ago (2016?). It's impressive, but it was a little glitchy and it felt overkill for the tasks that they were being used on. However, I'd guess that new products and ideas will proliferate if we somehow develop incredibly small and powerful projectors. Something that could be attached to a table, or even to your smartphone, and result in the same experience. I'd certainly want that :)
Animats 6y ago
There's the usual scaling problem with graphical representations. The now-discontinued Blender game engine was an example of building a complex system by connecting up boxed with arrows. You could easily get to a few square meters of program, and then finding anything was tough. Dynamicland PR: "Programs are flexible, and compose readily." So what do you compose? A functional block with ins and outs? What's the equivalent of a subroutine? That's where graphical systems usually fall down.
- myhf 6y ago
  There are some interesting examples of large visual programs at https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ I think the goal of Dynamicland is to build computational paradigms other than procedure-oriented programming. Individual cards can call lua subroutines, but the emergent behavior between cards is not the same type of composition as connecting subroutines. It reminds me of the gameplay in "Baba is You": the "program" is the set of rules currently in play, and the emergent behavior is the set of moves allowed by those rules.
  - Animats 6y ago
    There are some interesting examples of large visual programs at https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ Yes, same problem, and same solution, as the Blender game engine. ROS, the Robot Operating System, which is really a message passing library that runs on Linux, is something like that, too, with blocks connected by one-way connections. Emergent behavior between cards is not the same type of composition as connecting subroutines. Good point. You can start to see where this model works. Things where there's a lot going on in parallel, and loosely coupled modules need some interconnection. People keep re-inventing this idea, but so far, it tends to get really messy as it scales up. There's a second level of concepts needed here, something comparable to the invention of modules or classes or traits in programming.
- robenkleene 6y ago
  > So what do you compose? A functional block with ins and outs? What's the equivalent of a subroutine? That's where graphical systems usually fall down. Most visual programming languages have nodes with ins and outs that contain other nodes, just like you’re suggesting. And yes, I’d consider them the equivalent of subroutines. Programmers seem to talk about visual programming languages as if they’ve failed, but outside of software engineering they appear to me to be more popular than text-based languages. Some examples are VFX, compositing, audio/MIDI, prototyping, and automation. My own working hypothesis is that plain-text based version control is why most software engineering uses text files. The big differentiator between software engineering and other fields is there’s very little collaboration with visual programming language projects, they’re almost always made by one person.
1-6 6y ago
If I can only find people to sit beside like that video.
jancsika 6y ago
I'm trying to think of the simplest way to solve this problem: 1. Start with an analog synth from the 70s that's just a bunch of big, bulky metal modules connected by patch cords. 2. Remove the guts of each module and shrink them down to lego size 3. Print an English word name on each lego-sized module 4. Connect them up with patch cords. Now I have a physical artifact that represents a DSP graph in the visual diagram interface of a program like Pure Data. Question-- what is the easiest/cheapest way to continually pipe the topology data of the physical artifact into my laptop? It wouldn't be too hard to just have each module shouting a superaudio signal at my laptop and instantiating the corresponding object in the software when it's detected. But that doesn't cover the interconnections. I could take video input of the artifact but that would be crappy UX with the user constantly having to "show" the artifact to the camera from a non-ambiguous angle. I feel like there's some simple solution lurking out there with something like tinfoil and a 9v battery...
- sigstoat 6y ago
  if we ignore the issue of powering them, a CV cable is enough for a half-duplex serial connection. you don't need to transfer a lot of data to establish the topology. i think (hand waving!) every module could re-broadcast all the broadcasts it hears (that don't include itself) with its ID appended. those will make their way to the edges where they can be picked up, and would describe all the possible paths through the system. some microcontroller in each brick with a bunch of uarts would be ideal. then somewhere you need a usb-to-whatever link. (maybe somebody really clever could put the right set of passive parts in each brick so that every topology of devices would be distinguishable by some sort of analog probing from the periphery. i'm not that clever.)
  - Animats 6y ago
    If we ignore the issue of powering them, a CV cable is enough for a half-duplex serial connection. you don't need to transfer a lot of data to establish the topology. i think (hand waving!) every module could re-broadcast all the broadcasts it hears (that don't include itself) with its ID appended. those will make their way to the edges where they can be picked up, and would describe all the possible paths through the system. Not too hard. 1-Wire, a very low end LAN from Dallas Semiconductor, would be good for this. The parts are cheap, low-power, and powered over the connection cable. (1-wire requires 3 wires. You could use stereo phone jacks.) There are probably musician applications for this sort of thing. Some people like cables. A similar form of fakery is seen in DJ systems where you have vinyl records that contain not music, but time code.[1] The DJ can do DJ turntable stuff as if playing analogue records. They're just sending time code info to the the control unit which has the audio in memory, and the output is the appropriate audio for the time code. [1] https://en.wikipedia.org/wiki/Vinyl_emulation https://en.wikipedia.org/wiki/Vinyl_emulation
jakubp 6y ago
I don't get why there is no way to edit apps live through the operating system. Before I knew how programs work I didn't understand why I can't recode anything as user - change menus in Windows 3.1, change what they do, change logic of forms, etc. Today I know how this works and I'm even more convinced this would be good for everyone, exposing the actual logic behind all we see in apps (start with text, forms, links, buttons, etc.) would only be to everyone's benefit - would expose bugs, help people learn UIs in depth, suggest better functionalities and have swarms of users contribute to computers. Same with gathering user feedback -- the fact that we have such ridiculously unusable basic UI elements on mobile especially (people tend to NOT find basic UI elelemtsn of apps for months, sometimes years - how the f* is that even possible) is just one consequence of the fact that even if say 1000 users intend to do something and fail, the authors of the app never learn about that. WE get clubhouse to listen to one more type of radio, but we never get "userhouse" to get instant stream of people's complaints about an app (and a special physical button ON THE smartphone itself to launch that "instant feedback to the app author"mode so it's part of the base aspect of being a user of a smartphone)... sigh.... one can dream.
- aidenn0 6y ago
  > I don't get why there is no way to edit apps live through the operating system. Before I knew how programs work I didn't understand why I can't recode anything as user - change menus in Windows 3.1, change what they do, change logic of forms, etc. Today I know how this works and I'm even more convinced this would be good for everyone, exposing the actual logic behind all we see in apps (start with text, forms, links, buttons, etc.) would only be to everyone's benefit - would expose bugs, help people learn UIs in depth, suggest better functionalities and have swarms of users contribute to computers. This has been done before more than once, including at PARC (which is listed as an inspiration in TFA)
- Sophistifunk 6y ago
  I just had to help my wife move her taskbar back from the left hand side back to the bottom where she likes it. She has no idea how it got moved, and I've got no idea how long she just put up with it in the wrong place before it annoyed her enough to get help. Regular users might want to automate things, but they absolutely should not be able to accidentally go inside systems and break them.
platz 6y ago
The actual interactions seem rather limited. Rotations, translations, associations. not sure how much fidelity that actually gives you. How many problems can you actually solve with that?
rglover 6y ago
As soon as I saw it I was like "this looks like a Bret Victor project." Bingo.
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
bwy 6y ago
This project on a quick glance reminds me of the teamlab Borderless Museum in Japan. Interactive shapes, colors, and lines on real materials.
tianshuo 6y ago
Bret Victor has inspired me and my team for many years. His blog is a must read for PM's and curriculum designers. It is a pity that Dynamicland is non-opensource, or at least is not scalable to millions/billions of people. IMHO, building a scalable product and business around an innovative idea is crucial to helping a huge amount of real people.
warpspin 6y ago
Reminds me a lot of some of the things you can see at Ars Electronica. If you're in Austria, near Linz, very worth a visit: https://ars.electronica.art/center/en/ https://ars.electronica.art/center/en/
canada_dry 6y ago
Building on the way Dynamicland makes assembling programs a physical thing, I wonder why this couldn't fairly easily be extended to something like scratch[i]/snap![ii]/blocks[iii]? [i] https://scratch.mit.edu/developers https://scratch.mit.edu/developers [ii] https://appinventor.mit.edu/explore/understanding-blocks.html https://appinventor.mit.edu/explore/understanding-blocks.htm... [iii] https://snap.berkeley.edu/ https://snap.berkeley.edu/
granshaw 6y ago
I imagine playing collaborative tower defense on this surface would be pretty cool
crubier 6y ago
It seems that this whole thing is mostly about colorful stickers.
astrange 6y ago
Reading this reminded me that somewhere out there I've seen another computer system you operate by moving things around in real life. But after a minute I realized I was just thinking of the Bistromathic Drive in Hitchhiker's Guide, which is a spaceship you pilot by ordering things at an Italian restaurant.

297 points by andyjohnson0 6y ago

It feels like this has been reposted regularly for years, but the demo videos and slightly threadbare use cases haven't changed at all. Re-imagining how we interact with computers sounds great, but it doesn't feel like this project is actually going to achieve it's goals beyond letting a small number of lucky users play around with stuff. I'm not sure if darklang is going to succeed, but that seems like a far more fruitful direction to approach the problem from. It's a very hard problem, and they are attempting to make programming a little bit more visual while not removing any power from the user, instead of making it entirely visual and almost completely dis-empowering the user. Importantly, it's actually accessible to people all over the world, and they can use it to achieve real world goals.

scroot 6y ago
This issue here is research funding
crazypython 6y ago
"2020: Realtalk-2020. We're bringing together everything we've learned to create the next iteration of our Realtalk computing system. Realtalk-2020 will form the foundation for the next decade of research and applications." "2022: Dynamicland meets the world, in the form of new kinds of libraries, museums, classrooms, science labs, arts venues, and businesses. We will empower these communities to build what they need for themselves, to design their own futures."
bschne 6y ago
> ...but it doesn't feel like this project is actually going to achieve it's goals beyond letting a small number of lucky users play around with stuff. Will they end up building some product that will be adopted by droves of customers and offers a completely new paradigm for interaction? Probably not. But I'd argue that's hardly where "esoteric" research like this ends up going, and in my book that's OK. Bret Victor, who is behind Dynamicland, never shipped the full drawing app from his interactive visualizations talk as far as I know. Neither did anyone ever get to buy or download the editor he shows in "Inventing on Principle" as a new IDE that offers incredibly great feedback while programming. Nevertheless, his talks are among the most inspiring things I and I think many others have ever seen in the area of HCI, and at least in my case are responsible for a large part of my renewed interest in the field. Will anything out of Dynamicland capture people's imagination and enthusiasm like that as well? Maybe. Maybe not. But the point is to explore, and I applaud those who do.
- endergen 6y ago
  The influence of his talks show up everywhere and are explicitly called out as inspirations. Elm Lang's time travel debugger, therefore redux, hot reloading work, Observable calls him out as influential, and on and on.
  - bschne 6y ago
    Exactly my point — there's space for those who put together amazing but infeasible-as-product demos to communicate lofty ideas, those who take those and repackage them for "mass consumption", and of course the rare genius who does both, and anything in between.
    - IshKebab 6y ago
      I feel like putting together a janky tech demo is not especially impressive though. Anybody can think of screens projected onto paper. It's not particularly interesting that the tech demo is possible. Actually making it usable is the really hard part and the reason it remains a tech demo. That's why Apple is rich. They do the hard part.
endergen 6y ago
I personally hope that the output ends up being systems that are good for individuals with their own dynamic lands at home, not just strong group collaboration. I know there are already people who have setups done in coordination with the Dynamic Land team. I've visited dynamic land also, and the vision of this more tangible visible computation is bigger than the incarnation/progress last shown. I'm a fan of Darklang's goals too. But I do think Dynamic Land is targeting more approachable, teachable, and communal computing. Darklang is going to be more for professionals making backends and doesn't target any user facing audio/visuals whatsoever. It could in the end of course, so I hope it does well as an effort too.
- pbiggar 6y ago
  > Darklang is going to be more for professionals making backends and doesn't target any user facing audio/visuals whatsoever. It could in the end of course, so I hope it does well as an effort too. That is the plan, might be a while away though.
beaconstudios 6y ago
I hope darklang succeeds too, but I personally believe that visual programming is at least partly the way to go. This is because I firmly believe that the "most-correct" view of regular programming is that of a directed graph of computations, and directed graphs don't compress very well into text. I'm building something visual to prove (or disprove) this concept, but I'm also thinking that power users would want a textual language for faster input, as GUIs are better for output where TUIs are better for input.
- sparsely 6y ago
  I agree that visual programming can be great - my argument would be that starting from a model that we know works and making incremental steps towards something more visual is more likely to succeed, than attempting to jump all the way to something fully visual based
  - beaconstudios 6y ago
    in general I agree that incrementalism is the best way to improve a model. I'm just not certain how to do that with such a drastic change as going from textual to visual - but darklang seem to be doing a great job at that, and I do hope they succeed at their mission.
    - pbiggar 6y ago
      Thanks! We're very much of the opinion that we got into this mess we're in (gestures broadly) because everything has been incremental to date. That's how you get layers upon layers.
      - beaconstudios 6y ago
        Yes 100% - my founding hypothesis was that because language progress so far has all been improvements on opcodes, that the idioms of programming are still based on the human catering to the computer rather than the other way around. It gets subtler over time, but we still have to, for example, emulate execution in our heads while coding, which is really really hard to learn.
crazypython 6y ago
How is darklang related to dynamicland? I only found https://darklang.com/ https://darklang.com/
- theon144 6y ago
  Right, it honestly feels like the parent commenter either misunderstands Dynamicland's aims, or is willfully drawing comparisons to make it seem like the projects are tackling the same issues, out of which Dark comes out as the more successful one (at least measured by "adoption"). I don't see what the "almost completely dis-empowering the user" is about either; if anything, Dynamicland seems the more empowering of the two, whereas Dark's endgame is more efficient development of a fundamentally conventional type of software. It feels rather dishonest; the comparisons are sweeping and vague, with very little substance as to an actual criticism.
bmitc 6y ago
> instead of making it entirely visual and almost completely dis-empowering the user Why do you think that making something visual completely disempowers the user?

_Microft 6y ago

Is the technology of detecting and using marked everyday items (sheets of paper/whatever) as input devices and parts of programs patented? I couldn't find anything on that yet. It feels like something that would benefit from being patents-free.

jackcviers3 6y ago
It was used in Killer Game Programming in Java by Andrew Davidson published in 1995. That used a Webcam and a color coded bracelet you made yourself as an input device for one of the code examples. Microsoft had Surface touchscreen 2.0 pre 2011 that tracked real world objects [1]. 1. https://youtu.be/57k2iJbotV4 https://youtu.be/57k2iJbotV4

rodiger 6y ago

The shadows and latency are rough currently but totally agree that single, flat, rectangular input devices shouldn't be the future. Tactility is amazing and a large reason people prefer their old "dumb" interfaces.

kumarvvr 6y ago

I think this is a better paradigm than Augmented Reality glasses. The medium is limited but interesting. Perhaps this would be more portable if there is a projector underneath a table with a transparent glass top and a camera on top to view inputs. Potential Applications 1. Mapping (Interactive maps, interactive routing, real-time weather, etc) 2. Collaborative art 3. Engineering CAD and CAM modeling and design.

porcc 6y ago
Tilt5 might interest you as IMO the best paradigm of AR glasses--headmounted projectors with retroreflective surfaces. https://www.tiltfive.com https://www.tiltfive.com Portable, multiplayer--and comes with massive improvements in FOV and contrast due to its design.
gmueckl 6y ago
The mythical AR glasses of the future should be able to do everything your proposed projection table setup can. I see both setups as just two different manifestations of AR as a concept.
- kumarvvr 6y ago
  Haha. Mythical indeed. But more importantly, they are impersonal. In a collaborative setting, they are simply not conducive enough.
  - indigochill 6y ago
    Are you thinking of VR? AR merges the virtual environment with the physical one. One fun example was that I played a Finnish board game sort of like Pictionary with a Finnish friend. All the cards were in Finnish which I don't know, but I could use my phone's AR translation app to "read" the cards in effectively real time. AR glasses could hypothetically share a networked virtual environment, so I don't see why they couldn't be collaborative.
    - kumarvvr 6y ago

crazypython 6y ago

This is the most "object-oriented" programming language I've seen.

anarticle 6y ago

To contextualize this, check out Seymour Papert's work on LOGO, especially teams of kids working together to solve problems. Some of this is also related to Alan Kay's work. It's all inspiring, and shows that there may be many ways to compute.

jkscm 6y ago

Games like Kerbal Space Programm and factorio exhibit a lot of the concepts from the original research agenda[^1] because they are "Dynamic Environments-To-Think-In" [^1]: http://worrydream.com/cdg/ResearchAgenda-v0.19-poster.pdf http://worrydream.com/cdg/ResearchAgenda-v0.19-poster.pdf

olah_1 6y ago

Has anyone used Scratch style blocks for real physical blocks? They fit together like puzzle pieces. Kind of a neat visual that could translate to physical medium.

gremlinsinc 6y ago
I'd love something like this for my kids... Maybe have a board with lego slots for blocks that represent flow and maybe a little bit of interfacing with a computer to assign graphics to an object, say a sprite then the blocks tell it how to move, you could maybe have blocks that represent data and you can name it but the data stored is just json so no real schema for simplicity.... You could create interactive stories from a board of legos, or game like have a squirrel escape a yard without running into the dog, etc...

gklitt 6y ago

If you're looking for more concrete details on what it's actually like to make things at Dynamicland, I recommend this writeup by Omar Rizwan. (from 2018, so doesn't reflect recent progress, but it's still helpful for getting a feel for the place): https://omar.website/posts/notes-from-dynamicland-geokit/ https://omar.website/posts/notes-from-dynamicland-geokit/ I also wrote a short explanation of some little experiments I did there when I visited a couple years ago: https://www.geoffreylitt.com/projects/dynamicland.html https://www.geoffreylitt.com/projects/dynamicland.html

rtkwe 6y ago
I wish there was more out there about this. Seems kind of neat despite the limitations but I haven't seen much new come from it since it was introduced years ago.

jarmitage 6y ago

For those wondering about an update, they posted a "Narrative Description of Activities" document (something to do with 501c status) recently (https://twitter.com/worrydream/status/1367894681342799875 https://twitter.com/worrydream/status/1367894681342799875), choice quote re comments in this thread: > The Dynamicland researchers are not developing a product. The computers that the researchers build are models: not for distribution, but for studying and evaluating. The goal of the research is not to provide hardware or software to users, but to discover a new set of ideas which will enable all people to create computational environments for themselves, with no dependence on Dynamicland or anyone else, and to understand how these ideas can be taught and learned. The end result of the research is fundamentally intended to be a form of education, empowering communities to be self-sufficient and teach these ideas to each other.

tiglionabbit 6y ago
Yeah but can we go there?
- VonGuard 6y ago
  Last I checked, they were in downtown Oakland and you could just knock on the door.

thrower123 6y ago

This looks like the most vaporware thing I have seen since Duke Nukem Forever.

throwaway789256 6y ago

People who love Bret Victor's work and thought should know that he is also part of this effort.

crispyambulance 6y ago
Although I much admire Bret Victor, this Dynamicland project seems like a really, REALLY deep rabbit-hole. I hope he finds it enjoyable and rewarding, but I do hope re-joins the world and starts giving illuminating talks again!

adenozine 6y ago

What in the fresh hell! This is so awesome. I've always dreamt of something like this with some form of handwritten programming language, a group of people sketching a program on a single, giant canvas. I guess this isn't a new project, but I'm glad it's been reposted. I do worry that this format has some pretty sharp limits, just due to the spatiality and trying to cram functionality into a limited area of a room, amongst other people. Some form of code storage might need to be designed, so one could stack those little code papers on top of one another. Who knows. Very though-provoking project.

SamBam 6y ago

I went to a meeting at Dynamicland a few years ago, and got to spend a few days playing with it and chatting with people. The positives are many: I think it's awesome to conceive of computers outside the actual computer boxes, especially in an educational setting. I think the notion of "collaboration" is way more engaging when it's kids standing shoulder-to-shoulder and actually pushing pieces of paper around and arguing about things out loud, rather than "collaborating" on a Google Doc, sitting at different computers. I also think that this can be enlarged: all sorts of working meetings could be improved by people standing shoulder-to-shoulder at a table and pushing around tangible objects, with simple programming about how they'll interact. My negatives are what I took away from this, and may be incorrect or out-of-date, so feel free to jump in if I'm wrong, but I felt that Bret Victor was very much a purist regarding his vision, and had no desire to help spawn clones or variants of Dynamicland anywhere else. In many ways this may be laudable, but it felt like he was protecting his baby from going out into the wild, which has meant that the spread of ideas and possibilities has been greatly curtailed. It seems like it will only ever be destined to be a tiny playground for Bret and the few friends working with him.

theon144 6y ago
I see how that might be constructed as a negative, but IMO it's too early to tell whether that's a genuine setback to the project. "Protecting his baby" might be a very wise decision at this point, if only because of how the reception generally goes; pigeonholing the project into something like "an AR coding environment" or "visual programming with projectors" is a very real risk that could damage the project's aims - even "clones" such as https://paperprograms.org/ https://paperprograms.org/ make it abundantly clear that they are not attempting to be an "opensource Dynamicland". It's only been 3 years since it was founded. While that might be generally considered an eternity in tech-time, I feel it's barely enough to get one's feet wet given the scale of the project, which seems to aim to be decades long. Besides, the roadmap they've got on their website mentions 2022 as the year they go public - so, I personally am stoked for what that will bring.
- sleepybrett 6y ago
  Might be wise but might not be, look at Xanadu, with the web we got a massive 'halfstep' without any xanadu hands on the wheel.

coloneltcb 6y ago

Great to see this on the front page. I've been lucky enough to visit Dynamicland/volunteer there and it absolutely lives up to the hype.

mwcampbell 6y ago

I sent the following message to Dynamicland a few years ago and never got a satisfactory response. Does anyone know if they're doing anything about accessibility for blind people or people with other disabilities? Hi, First of all, Dynamicland's goal of "agency, not apps" resonates with me. I'm in favor of things that make programming more accessible to normal people. However, I worry that Dynamicland will be a step backward for people with disabilities, particularly blind people and people with impaired mobility. For us (I'm legally blind myself), I believe the virtual worlds of today's computers aren't imprisoning but liberating. Consider that a blind person can't see that "fully functional" scrap of paper, and a person who can't use their hands can't write on that paper or manipulate the other physical objects in Dynamicland. Do you have a plan to solve this problem while holding to your goal of "No screens, no devices"? It seems to me that there's no way to reconcile these conflicting requirements without making an exception for people who can't work directly with the paper and other objects. Or have you decided that it's better to undo the equalizing effect of computers for people with disabilities, for the good of everyone else? I would obviously be disappointed if that's the case. But I understand that everything's a trade-off, and perhaps it's not reasonable to confine the majority to an inhumane way of working for the benefit of a few. So I don't mean this as an accusation; I really want to know. Thanks, Matt

gugagore 6y ago
I think your question is important, and I wish you had received a reply. I think it's common for people's needs to be in conflict; everything is a trade-off, as you acknowledged. But I think you baked into your question a false dichotomy. "Dynamicland" does not necessarily have to _undo_ the "equalizing effect of computers", nor does its absence mean that the majority is confined to an _inhumane_ (wow...) way of working.
- mwcampbell 6y ago
  > an _inhumane_ (wow...) way of working. That was Bret Victor's characterization of being confined to a screen, not mine.
chromanoid 6y ago

jude- 6y ago

Is it open-source yet? I can't seem to find any source code anywhere.

sesuximo 6y ago

Man that looks cool! I’d love to use paper and a projector instead of looking at a screen all day

jackcviers3 6y ago

Dynamicland feels to me like The Mother of all Demos. Ahead of its time, groundbreaking practical applications of interactivity research. Dozens of companies and projects will spring from this. Just give it time. We'll be back to using real objects instead of keyboards at some point in the near-future. Brett's right that a lot of the difficulty in actually using computers to produce things is in the translation of human interactions into a medium the computer can understand, and translating the data a computer outputs into a medium humans understand. We aren't two-dimensional creatures. The state of the art cannot remain in pictures and text forever. I'm not sold that everything will be done this way - some things will always need the precision of automated, textual input (see tool assisted speed runs in video-gaming if you think human beings can match pre-programmed precision inputs).

twobitshifter 6y ago
They actually cite Englebert at the top of the page as guiding the project spirit.

haolez 6y ago

I saw something like this in a presentation at the MIT Media Lab a few years ago (2016?). It's impressive, but it was a little glitchy and it felt overkill for the tasks that they were being used on. However, I'd guess that new products and ideas will proliferate if we somehow develop incredibly small and powerful projectors. Something that could be attached to a table, or even to your smartphone, and result in the same experience. I'd certainly want that :)

Animats 6y ago

There's the usual scaling problem with graphical representations. The now-discontinued Blender game engine was an example of building a complex system by connecting up boxed with arrows. You could easily get to a few square meters of program, and then finding anything was tough. Dynamicland PR: "Programs are flexible, and compose readily." So what do you compose? A functional block with ins and outs? What's the equivalent of a subroutine? That's where graphical systems usually fall down.

myhf 6y ago
There are some interesting examples of large visual programs at https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ I think the goal of Dynamicland is to build computational paradigms other than procedure-oriented programming. Individual cards can call lua subroutines, but the emergent behavior between cards is not the same type of composition as connecting subroutines. It reminds me of the gameplay in "Baba is You": the "program" is the set of rules currently in play, and the emergent behavior is the set of moves allowed by those rules.
- Animats 6y ago
  There are some interesting examples of large visual programs at https://blueprintsfromhell.tumblr.com/ https://blueprintsfromhell.tumblr.com/ Yes, same problem, and same solution, as the Blender game engine. ROS, the Robot Operating System, which is really a message passing library that runs on Linux, is something like that, too, with blocks connected by one-way connections. Emergent behavior between cards is not the same type of composition as connecting subroutines. Good point. You can start to see where this model works. Things where there's a lot going on in parallel, and loosely coupled modules need some interconnection. People keep re-inventing this idea, but so far, it tends to get really messy as it scales up. There's a second level of concepts needed here, something comparable to the invention of modules or classes or traits in programming.
robenkleene 6y ago
> So what do you compose? A functional block with ins and outs? What's the equivalent of a subroutine? That's where graphical systems usually fall down. Most visual programming languages have nodes with ins and outs that contain other nodes, just like you’re suggesting. And yes, I’d consider them the equivalent of subroutines. Programmers seem to talk about visual programming languages as if they’ve failed, but outside of software engineering they appear to me to be more popular than text-based languages. Some examples are VFX, compositing, audio/MIDI, prototyping, and automation. My own working hypothesis is that plain-text based version control is why most software engineering uses text files. The big differentiator between software engineering and other fields is there’s very little collaboration with visual programming language projects, they’re almost always made by one person.

1-6 6y ago

If I can only find people to sit beside like that video.

jancsika 6y ago

I'm trying to think of the simplest way to solve this problem: 1. Start with an analog synth from the 70s that's just a bunch of big, bulky metal modules connected by patch cords. 2. Remove the guts of each module and shrink them down to lego size 3. Print an English word name on each lego-sized module 4. Connect them up with patch cords. Now I have a physical artifact that represents a DSP graph in the visual diagram interface of a program like Pure Data. Question-- what is the easiest/cheapest way to continually pipe the topology data of the physical artifact into my laptop? It wouldn't be too hard to just have each module shouting a superaudio signal at my laptop and instantiating the corresponding object in the software when it's detected. But that doesn't cover the interconnections. I could take video input of the artifact but that would be crappy UX with the user constantly having to "show" the artifact to the camera from a non-ambiguous angle. I feel like there's some simple solution lurking out there with something like tinfoil and a 9v battery...

sigstoat 6y ago
if we ignore the issue of powering them, a CV cable is enough for a half-duplex serial connection. you don't need to transfer a lot of data to establish the topology. i think (hand waving!) every module could re-broadcast all the broadcasts it hears (that don't include itself) with its ID appended. those will make their way to the edges where they can be picked up, and would describe all the possible paths through the system. some microcontroller in each brick with a bunch of uarts would be ideal. then somewhere you need a usb-to-whatever link. (maybe somebody really clever could put the right set of passive parts in each brick so that every topology of devices would be distinguishable by some sort of analog probing from the periphery. i'm not that clever.)
- Animats 6y ago
  If we ignore the issue of powering them, a CV cable is enough for a half-duplex serial connection. you don't need to transfer a lot of data to establish the topology. i think (hand waving!) every module could re-broadcast all the broadcasts it hears (that don't include itself) with its ID appended. those will make their way to the edges where they can be picked up, and would describe all the possible paths through the system. Not too hard. 1-Wire, a very low end LAN from Dallas Semiconductor, would be good for this. The parts are cheap, low-power, and powered over the connection cable. (1-wire requires 3 wires. You could use stereo phone jacks.) There are probably musician applications for this sort of thing. Some people like cables. A similar form of fakery is seen in DJ systems where you have vinyl records that contain not music, but time code.[1] The DJ can do DJ turntable stuff as if playing analogue records. They're just sending time code info to the the control unit which has the audio in memory, and the output is the appropriate audio for the time code. [1] https://en.wikipedia.org/wiki/Vinyl_emulation https://en.wikipedia.org/wiki/Vinyl_emulation

jakubp 6y ago

I don't get why there is no way to edit apps live through the operating system. Before I knew how programs work I didn't understand why I can't recode anything as user - change menus in Windows 3.1, change what they do, change logic of forms, etc. Today I know how this works and I'm even more convinced this would be good for everyone, exposing the actual logic behind all we see in apps (start with text, forms, links, buttons, etc.) would only be to everyone's benefit - would expose bugs, help people learn UIs in depth, suggest better functionalities and have swarms of users contribute to computers. Same with gathering user feedback -- the fact that we have such ridiculously unusable basic UI elements on mobile especially (people tend to NOT find basic UI elelemtsn of apps for months, sometimes years - how the f* is that even possible) is just one consequence of the fact that even if say 1000 users intend to do something and fail, the authors of the app never learn about that. WE get clubhouse to listen to one more type of radio, but we never get "userhouse" to get instant stream of people's complaints about an app (and a special physical button ON THE smartphone itself to launch that "instant feedback to the app author"mode so it's part of the base aspect of being a user of a smartphone)... sigh.... one can dream.

aidenn0 6y ago
> I don't get why there is no way to edit apps live through the operating system. Before I knew how programs work I didn't understand why I can't recode anything as user - change menus in Windows 3.1, change what they do, change logic of forms, etc. Today I know how this works and I'm even more convinced this would be good for everyone, exposing the actual logic behind all we see in apps (start with text, forms, links, buttons, etc.) would only be to everyone's benefit - would expose bugs, help people learn UIs in depth, suggest better functionalities and have swarms of users contribute to computers. This has been done before more than once, including at PARC (which is listed as an inspiration in TFA)
Sophistifunk 6y ago
I just had to help my wife move her taskbar back from the left hand side back to the bottom where she likes it. She has no idea how it got moved, and I've got no idea how long she just put up with it in the wrong place before it annoyed her enough to get help. Regular users might want to automate things, but they absolutely should not be able to accidentally go inside systems and break them.

platz 6y ago

The actual interactions seem rather limited. Rotations, translations, associations. not sure how much fidelity that actually gives you. How many problems can you actually solve with that?

rglover 6y ago

As soon as I saw it I was like "this looks like a Bret Victor project." Bingo.

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

bwy 6y ago

This project on a quick glance reminds me of the teamlab Borderless Museum in Japan. Interactive shapes, colors, and lines on real materials.

tianshuo 6y ago

Bret Victor has inspired me and my team for many years. His blog is a must read for PM's and curriculum designers. It is a pity that Dynamicland is non-opensource, or at least is not scalable to millions/billions of people. IMHO, building a scalable product and business around an innovative idea is crucial to helping a huge amount of real people.

warpspin 6y ago

Reminds me a lot of some of the things you can see at Ars Electronica. If you're in Austria, near Linz, very worth a visit: https://ars.electronica.art/center/en/ https://ars.electronica.art/center/en/

canada_dry 6y ago

Building on the way Dynamicland makes assembling programs a physical thing, I wonder why this couldn't fairly easily be extended to something like scratch[i]/snap![ii]/blocks[iii]? [i] https://scratch.mit.edu/developers https://scratch.mit.edu/developers [ii] https://appinventor.mit.edu/explore/understanding-blocks.html https://appinventor.mit.edu/explore/understanding-blocks.htm... [iii] https://snap.berkeley.edu/ https://snap.berkeley.edu/

granshaw 6y ago

I imagine playing collaborative tower defense on this surface would be pretty cool

crubier 6y ago

It seems that this whole thing is mostly about colorful stickers.

astrange 6y ago

Reading this reminded me that somewhere out there I've seen another computer system you operate by moving things around in real life. But after a minute I realized I was just thinking of the Bistromathic Drive in Hitchhiker's Guide, which is a spaceship you pilot by ordering things at an Italian restaurant.