14 ms·

Rethinking Design Tools in the Age of Machine Learning

79 points by mayava 9y ago

huula 9y ago
Great article. Covers a lot of aspects on how ML can help designers be more creative and productive! At Huula, I'm a firm believer that ML can automate various parts of web designs. We just released a new experiment CSSToucan[1] to auto color texts on web pages with Recurrrent Neural Networks. It learns to color texts on web pages without a single line of color theories in the code. All learned from the data. Hope to see more and more ML powered design tools emerging! [1]: https://huu.la/ai/csstoucan https://huu.la/ai/csstoucan
oron 9y ago
Great well written article
mtdewcmu 9y ago
It sounds like one of the core issues is dimensionality reduction. I would suggest reducing everything to text. Text is one-dimensional. In this case: >> Like any other map, we can add textual labels — street signs if you will. This allows us to navigate the design space verbally with commands like: “take me to a maple leaf.” Once there, we could say something like: “take me a bit closer to an oak leaf.” The change here would be subtle. Rather than a two-way conversation between tool and user, instead it's more like the user is telling a story and the tool is making suggestions along the way: start: "Create a leaf" tool inserts a suggestion: "Create a [maple,oak,cherry,...] leaf" user updates the story with the clarification: "Create a maple leaf" tool suggests: "Create a [red,green,...] maple leaf" user: "Create a green maple leaf" user starts typing more: "Create a green maple leaf on a" tool inserts suggestions: "Create a maple leaf on a [branch,wreath,...]" Hope this is helpful.
pavlov 9y ago
This is an area where the vision is very desirable, but user expectations also run sky-high. A company called The Grid collected about $5 million from users (and another sum in the same ballpark from VCs) with the promise of an AI that designs websites for you. Years later, they still have very little to show. It seems like most of those tens of thousands of aspiring web creators who paid $96 to be "founding members" of this AI revolution have quietly licked their wounds and given up, because there doesn't even seem to be too much active discussion about the product anymore. (The Reddit forum is dead, with a few messages of the "Can we sue?" kind.)
- mtdewcmu 9y ago
  Maybe I was reading into it, but I got the impression that the end product here was something more of an artistic nature rather than a website. If the end product is something that is basically a computer program, like a website, then I expect it to fail. My conjecture is that code and a text editor is already the optimal tool interface for creating computer programs. You can't create a superior tool interface, because code and a text editor is already optimal. When it comes to creating art, there seems to be a great potential for better tool UIs. A guitar and hands is not the optimal way to create music. We don't seem to have found the optimal tool for creating music yet (if there is one).
  - andyjsong 9y ago
    >When it comes to creating art, there seems to be a great potential for better tool UIs. A guitar and hands is not the optimal way to create music. We don't seem to have found the optimal tool for creating music yet (if there is one). My company is working on just that, creating the next generation of music instruments to allow anyone to create music. Here is our first prototype where we've changed the UI of the guitar, but also left in traditional elements to allow beginners or experts to play music by switching from what we call "Magic Mode" to traditional guitar: https://www.youtube.com/watch?v=vkM4sr2socQ https://www.youtube.com/watch?v=vkM4sr2socQ
    - mtdewcmu 9y ago
      Music is not my area of expertise, but 1. It looks like there are limitless possibilities in that area of technology. 2. Magic Mode reminds me of Omnichord. It looked like with the Omnichord, you could do certain things more easily than with an analog instrument, but other things became harder or impossible. So it was a trade off. 3. In addition to allowing anyone to create music, it would be especially interesting if you could create new possibilities for expert players. For ex: some kind of augmented drum could allow you to create and play more intricate rhythms than would normally be possible, perhaps by having ML extrapolate and combine patterns in ways that are too complex to even write (I used to play drums). It seems like the ear can hear patterns that are beyond what can be played or even created by a person.

79 points by mayava 9y ago

Great article. Covers a lot of aspects on how ML can help designers be more creative and productive! At Huula, I'm a firm believer that ML can automate various parts of web designs. We just released a new experiment CSSToucan[1] to auto color texts on web pages with Recurrrent Neural Networks. It learns to color texts on web pages without a single line of color theories in the code. All learned from the data. Hope to see more and more ML powered design tools emerging! [1]: https://huu.la/ai/csstoucan https://huu.la/ai/csstoucan

oron 9y ago

Great well written article

mtdewcmu 9y ago

It sounds like one of the core issues is dimensionality reduction. I would suggest reducing everything to text. Text is one-dimensional. In this case: >> Like any other map, we can add textual labels — street signs if you will. This allows us to navigate the design space verbally with commands like: “take me to a maple leaf.” Once there, we could say something like: “take me a bit closer to an oak leaf.” The change here would be subtle. Rather than a two-way conversation between tool and user, instead it's more like the user is telling a story and the tool is making suggestions along the way: start: "Create a leaf" tool inserts a suggestion: "Create a [maple,oak,cherry,...] leaf" user updates the story with the clarification: "Create a maple leaf" tool suggests: "Create a [red,green,...] maple leaf" user: "Create a green maple leaf" user starts typing more: "Create a green maple leaf on a" tool inserts suggestions: "Create a maple leaf on a [branch,wreath,...]" Hope this is helpful.

pavlov 9y ago

This is an area where the vision is very desirable, but user expectations also run sky-high. A company called The Grid collected about $5 million from users (and another sum in the same ballpark from VCs) with the promise of an AI that designs websites for you. Years later, they still have very little to show. It seems like most of those tens of thousands of aspiring web creators who paid $96 to be "founding members" of this AI revolution have quietly licked their wounds and given up, because there doesn't even seem to be too much active discussion about the product anymore. (The Reddit forum is dead, with a few messages of the "Can we sue?" kind.)

mtdewcmu 9y ago
Maybe I was reading into it, but I got the impression that the end product here was something more of an artistic nature rather than a website. If the end product is something that is basically a computer program, like a website, then I expect it to fail. My conjecture is that code and a text editor is already the optimal tool interface for creating computer programs. You can't create a superior tool interface, because code and a text editor is already optimal. When it comes to creating art, there seems to be a great potential for better tool UIs. A guitar and hands is not the optimal way to create music. We don't seem to have found the optimal tool for creating music yet (if there is one).
- andyjsong 9y ago
  >When it comes to creating art, there seems to be a great potential for better tool UIs. A guitar and hands is not the optimal way to create music. We don't seem to have found the optimal tool for creating music yet (if there is one). My company is working on just that, creating the next generation of music instruments to allow anyone to create music. Here is our first prototype where we've changed the UI of the guitar, but also left in traditional elements to allow beginners or experts to play music by switching from what we call "Magic Mode" to traditional guitar: https://www.youtube.com/watch?v=vkM4sr2socQ https://www.youtube.com/watch?v=vkM4sr2socQ
  - mtdewcmu 9y ago
    Music is not my area of expertise, but 1. It looks like there are limitless possibilities in that area of technology. 2. Magic Mode reminds me of Omnichord. It looked like with the Omnichord, you could do certain things more easily than with an analog instrument, but other things became harder or impossible. So it was a trade off. 3. In addition to allowing anyone to create music, it would be especially interesting if you could create new possibilities for expert players. For ex: some kind of augmented drum could allow you to create and play more intricate rhythms than would normally be possible, perhaps by having ML extrapolate and combine patterns in ways that are too complex to even write (I used to play drums). It seems like the ear can hear patterns that are beyond what can be played or even created by a person.