22 ms·
Show HN: I trained a deep learning model to build sites for me
- johndamaia 7y agoFounder here. As a front end developer, I turn design files into front-end code regularly. But it's a repetitive and demotivating task. It takes time, lots of meetings and boring back and forth questions. I want to change that. So I tried to automate this process. I trained a deep learning (DL) model to visually understand what elements are in a webpage (e.g. input, button, nav, etc) and build the front-end boiler plate automatically for me. I learned a lot in the process, the actual site builder is a mathematical interpretation of a Sketch file and the DL model outputs what elements to include in what section. To be completely transparent I'm not quite there yet. I'm only able to have ~90% of the site right. The other ~10% are misinterpretations or incorrect positioning of the elements. In order for this to work properly, it requires manual validation - which for my own projects I’m more than happy to do. Even at this stage I think this tool can help others as well, so please let me know if you have any questions. We can be more productive without repetitive tasks. [1] Demo: https://zecoda.com/#demo https://zecoda.com/#demo [2] Request early access: https://zecoda.com/#subscribe https://zecoda.com/#subscribe
- nkozyra 7y agoInterested to see this. I've done something similar in the past (building models that translated paper drawings into HTML+css) and ultimately found it usually didn't save me much time. Translating a look into code is much less labor intensive than it once was and code reuse and stylesheet preprocessers and the ilk (webpack, etc) save tons of time. I'd get something reasonable out of the model but then the actual look and feel work came into play. My next step was to be able to reference other sites for styling hints but ultimately I wasn't getting enough utility out of it to continue. Do you find the same with this approach? If so, what plans do you have to take it the next few yards so it's a more comprehensive front end tool.
- johndamaia 7y agoActually my workflow improved drastically after getting the boiler plate to work on the fun stuff (interactivity, animations, etc). I use this tool to get started much faster.
- mkagenius 7y agoHey, why not give something for people to try. I know[1] it takes (GPU) resources and may give it a hug of death but its much better to let at least some people try it out and give this a feedback while its hot. People may lose interest later on, who knows. 1. Something I made and let people use but nobody uses: https://comic2gif.com https://comic2gif.com
- johndamaia 7y agoGood point. I'll definitely give it for people to try in the next couple weeks.
- ioeatcode 7y agoEh, if anything maybe invite one or two people for usability testing. Otherwise, having that many data points this early on might hurt your business.
- ALittleLight 7y agoYou may want to have an example comic gif on the homepage. I didn't really understand what it did until I tried it. Plus, it would be neat if panels with more text were visible longer.
- mkagenius 7y ago> You may want to have an example comic gif on the homepage There is one. If you click on "show today's" button. But I guess default should be changed to show instead of hidden. > Plus, it would be neat if panels with more text were visible longer. There is some logic in code to do that, amount of white pixels (in mask) is proportional to the time it appears. But maybe it needs tweaking.
- gilbertmpanga12 7y agoWhat stack did you use?
- onion2k 7y agoIt takes time, lots of meetings and boring back and forth questions. I want to change that. So I tried to automate this process. The reason for all the meetings and questions is that the designs you're working from are ambiguous, and you don't really have enough information to turn the design files in to working code. I can't quite see how a deep learning model solves that problem. By the sounds of it you've automated turning the original, ambiguous, wrong designs the you have at the start in to code and tried to remove the discussion process that takes those files and turns them in to the right solution to the customer's problem. This is not going to lead to better front end code.
- wlesieutre 7y agoDepending on the customer, that could still be helpful. I've worked with people where you can ask them very specific questions over and over and get nowhere - the only way they're capable of moving forward is if you give them something you know is wrong so they can look at it and say "No, it needs to X." It's annoying to do, but if you can automate producing the initially wrong product for review it'd be easier.
- throwaway743 7y agoCouldn't agree with you more
- kthejoker2 7y agoCunningham's Law is one of the truest design patterns in all of mankind.
- moehm 7y agoFor everyone else: "Cunningham's Law states 'the best way to get the right answer on the internet is not to ask a question; it's to post the wrong answer.'" [0] [0] https://meta.wikimedia.org/wiki/Cunningham%27s_Law https://meta.wikimedia.org/wiki/Cunningham%27s_Law
- sizzle 7y ago
- friendlybus 7y agoHow well do you think this concept will scale to bigger code bases and bigger production tools like photoshop? The next step up has been pushing ML/DL/AI to do more sophisticated tasks and that means more sophisticated tools.
- johndamaia 7y agoAt this stage we are working hard to solve this simple problem of going from a design file to a front-end boiler plate. No animations or logic. We are looking forward to get as much feedback as possible from our users to understand where to go next.
- SteveMoody73 7y agoInteresting idea but how maintainable is the code? I've used a few tools over the years that automatically generate code, mostly in C, but I've usually had to rewrite it as it never fits in with any decent coding standard
- johndamaia 7y agoThat's our main goal. To always give you good code. Otherwise this tool doesn't make sense. Hopefully you'll find this code maintainable with close to zero rewrites.
- digikata 7y agoIs the access to provide data tagging inputs and outputs or to the model too? Will either the data or the model be open sourced?
- thomasfl 7y agoSeems like you’ e created the ultimate sketch plugin.
- doubleorseven 7y agoWow, nice work! I used to work for Webydo on the part that translate the DB to static files, tons of work and the result was basically an "uneditable HTML blob". I would really love to see the output zip file, any change you have something less complicated that goes responsive to show?
- johndamaia 7y agoIf you think it can help, I can send you the output code for the demo. Drop me a line at john [@] zecoda.com
- nicklovescode 7y agoHi! I'm super curious about how you structured the input and output to the neural network, and which architecture you chose, if you're willing to share (I work at OpenAI) If you do share I'm happy to give you feedback here if I have any!
- noonespecial 7y agoI'm honestly kind of surprised nothing like this has found its way into Sketch itself yet. Looks cool so far.
- stunt 7y agoI also always thought this is a natural thing for Sketch to have it. Webflow gives similar experience of Sketch but to build web pages.
- diggan 7y agoFrom https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html > Show HN is for something you've made that other people can play with. HN users can try it out, give you feedback, and ask questions in the thread. This should not be a Show HN as there is nothing to try out, just a video + early access list that currently doesn't give you access.
- johndamaia 7y agoUpdated. Thanks for the heads up.
- otsdr 7y agoThere's a useless vertical scrollbar on desktop.
- johndamaia 7y agoWhich page?
- FroshKiller 7y agoAll of them.
- johndamaia 7y agoCan't replicate the issue... Can you show me a screen shot?
- tiniuclx 7y agoI have the same issue on Firefox. It's to the right of the webpage, next to Firefox's scrollbar.
- madhato 7y ago#app { ... overflow-y: scroll !important; } Is causing it and not needed.
- johndamaia 7y agoThanks!!!
- mattkevan 7y agoSurprised it’s Vue and not React. Vue is great, but there’s a lot of activity in the Sketch <-> React space, for example AirBnb’s Lona [0]. This could fit in nicely. [0] https://github.com/airbnb/Lona https://github.com/airbnb/Lona
- johndamaia 7y agoWe are part of its community for years. It's always good to start with tools you master. React is definitely on our roadmap. Are you more of a Vue or React dev?
- jongold 7y agoLona is After Effects -> web / native. You might be thinking about react-sketchapp[1] or SharpieClassifier [2] (I built both :P) [1] https://airbnb.design/painting-with-code/ https://airbnb.design/painting-with-code/ [2] https://airbnb.design/sketching-interfaces/ https://airbnb.design/sketching-interfaces/
- mattkevan 7y agoI thought Lottie was the After Effects to code plugin. Happy to be wrong though. Anyway, it’s a pleasure to meet you - I’ve been impressed with both your projects and following them for a while now. Love to talk more about them.
- jppope 7y agoIt would make sense since react does the jsx thing and Vue uses straight html inside of the template tags.
- gotrythis 7y agoHi. Wish you had one-off pricing. I'm sure there are agencies that do that kind of volume. I would love to use your service a few times a year.
- johndamaia 7y agoWill an one-off monthly fee work? Or were you thinking on a per artboard approach?
- acallaghan 7y agoI'm the same - I'd pay $30/40 for a single one, but yearly cost of $99 is too much
- johndamaia 7y agoCan you please clarify what you mean by "single one"? You'd pay $30/40 for how many pages of one site?
- acallaghan 7y agoAs in using it a single time - I guess maybe at most 2/3 pages from one design?
- johndamaia 7y agoOk, cool. In fact you are right in our price point of $9 per artboard. The only think we need to evaluate is the one-off possibility which I'm sure we can try.
- anteatersa 7y agoIt's actually $99/month if paid yearly.
- mxcrossb 7y agoI think with a one off it’s better just to outsource it. But the benefit of the automatic approach is that you could iterate a design over the course of a few weeks, getting quick turn around on each tweak.
- nl 7y agoI built a thing like thi, and went quite a long way down the commercialization path. The problem is that round-tripping is what almost everyone wants, and it's really hard to get to work.
- johndamaia 7y agoCan you elaborate on what you mean by round-tripping please?
- dminor 7y agoI assume they mean from web back to sketch.
- johndamaia 7y agoMeaning going from front-end code to Sketch, right? Is this something you need?
- bsaul 7y agoI suppose most website html templates are later filled with code ( php, js, whatever). Then design change, you don’t want to restart inserting php code all over again. At least that was one issue i thought about reading OP.
- itsmefaz 7y agoThis is very interesting. I love the approach to the problem that you are trying to solve. However, there are few questions that I will try to answer with respective to the product. Q. Will I pay to use your problem? A. Probably not, with the current version Q. What problems I see with the platform? A. I seriously don't know how you'd proceed forward with the platform. If it would only be used for creating static webpages then you'd end up creating a platform similar to Wix or other such website generation platforms. Please let me know what you think about the suggestion.
- johndamaia 7y agoLet me turn back some questions to you: 1. What's not enough in this current version for you to pay? What would make you be able to pay? 2. Do you have this problem recurrently? i.e. going from design to code.
- monkeynotes 7y agoIt would be interesting to see the quality of the code. Way back in the 90s Macromedia had a product called Fireworks, which Adobe subsequently bought (along with Flash, Dreamweaver etc.). It could turn designs into HTML+CSS but the code was horrible and no-one used it professionally. I know we are decades past that, and the machine learning algos may do a better job but without seeing some code examples it has limited appeal to me. I don't really know the kind of person that has to turn around so many websites that the boilerplating exercise is a big enough problem to pay to solve. I can see the appeal to people who have no experience building websites and a company like Wix might be interested in the tech. But professionals who need a monthly subscription has to be a tiny niche. Impressive all the same though, what you have learned in doing this will lead to other great things I am sure.
- johndamaia 7y agoThanks so much for your words. Happy to share our demo code, would love your feedback. If you think it makes sense, drop me a line at john [@] zecoda.com
- dudado 7y agoThis looks promising, however, I'm not sure about the pricing model, $99 per month (IF billed annually) for 10 designs (10 artboards)?
- johndamaia 7y agoYes, one artboard usually correspond to one webpage. How much were you willing to pay? Is this a problem you have every week?
- stormtv 7y agoThis is really cool, it is really hard to evaluate the value of Zencoda without the code output of the demo. I get that the full product is not ready but being able to see the input and one good output would tell us a lot.
- johndamaia 7y agoThanks! Happy to share our demo code, would love your feedback. If you think it makes sense, drop me a line at john [@] zecoda.com
- 247yak 7y agoLooks like Yotako...https://www.yotako.io/ https://www.yotako.io/ Whats the difference? Yotako seems to be in the deadpool though...havn't seen anything new in ages. Hard to scale this stuff.
- johndamaia 7y agoDidn't know them. The concept seems the same :)
- preommr 7y agoIs this just a quick and short project or have you been working on this long term?
- johndamaia 7y agoOne month full time; around one year part-time.
- potench 7y agoI’m sorry but this feels really hand-wavy to me. What did you feed your model for it to learn how to build vuejs sites? Vuejs template language is total chaos if you haven’t put in quite a bit of time with vue before, there are 2 or 3 ways to do everything. I am having trouble believing this was auto-generated from a deeply learned model. How do you write css, are you writing sass, vue inline styles, or something vanilla? Are you bundling with with webpack, what custom modules does your webpack use for handling those inline svgs? So so many questions about how and if this was auto generated...
- johndamaia 7y agoGreat questions. And I also understand your skepticism. You're not the first and don't expect you to be the last :) Trying to answer all your questions: - Our models learnt website's components visually (e.g. what is a navbar, an input, a button, etc) - The output is a .zip file with the .html files and directories for css, js and img - All Vue components can be found in main.js inside the js directory - Vue components are built when 'duplication' is recognized (e.g. a sequence of cards or several links for a navbar). Nothing fancy - CSS is vanilla, trying to ditch as many inline style as possible (working on it as we speak :) ) - All SVGs are imported with the img element. Let me know if you have more great questions.
- potench 7y agoCool, thanks for the reply. For me making components is the hard work, anything that helps automate organizing / naming / and synchronizing a component library with sketch sounds intriguing to me. Is the output intended to be code-edited by a developer to add hrefs, interactivity? Is it intended to generate pseudo components for later finessing or is it intended to produce the final outcome of a site directly from sketch (making Sketch the new dreamweaver?)
- johndamaia 7y agoIt's intended to be code-edited by a developer. The goal is to have an advance starting point. You'll start with "everything" ready to work on the fun stuff. If you curious enough to see the output I'm happy to share the code from our demo. If it makes sense drop me a line at john [@] zecoda.com
- llarsson 7y agoIs the home page itself built with this? Can you share the input file and an estimate on how long it took to work with the output to make it production ready?
- johndamaia 7y agoMost of the website was started with the model. However I was learning along the way, so most pages were fairly changed. Regarding the time for the optimization of the output, we are estimating ~20/30 minutes per page. It could be very faraway from the reality or pretty close, not sure yet.
- SandyAndyPerth 7y agoA dogfooding video, where you build your own site with it, would be much more convincing.
- markbnj 7y agoThis is a really interesting project, thanks for sharing. I think it's going to be difficult to get the model to the point where it generalizes well for lots of different designs, but that's no reason not to try. Really hard problems sometimes lead to really big wins.
- bitL 7y agoThe next step is to allow voice/touch-driven repositioning of elements that the initial model didn't get right, e.g. telling it to "move Submit above Cancel" or just grabbing it and placing it there. That would clear most issues customers have and they could do that themselves, including changing styles/themes and seeing how it behaves right away.
- ReDeiPirati 7y agoIt's really great to see more and more companies emerging in the market from these initial ideas/PoC: [1] Pix2Code: https://arxiv.org/abs/1705.07962 https://arxiv.org/abs/1705.07962 [2] Sketch2Code: https://airbnb.design/sketching-interfaces/ https://airbnb.design/sketching-interfaces/ [3] ScreenShot2Code: https://blog.floydhub.com/turning-design-mockups-into-code-with-deep-learning/ https://blog.floydhub.com/turning-design-mockups-into-code-w...
- jongold 7y agoTotally. The SharpieClassifier we built at Airbnb was super simple because we had a relatively fixed set of components that could be declaratively rendered with React or react-sketchapp as simply as <Marquee /> etc. I can't imagine how much more complex UIzard & Zecoda are - mad respect to the developers and for being able to commercialize these ideas. Aside: I actually built react-sketchapp as a roundabout way of being able to build more interesting AI plugins for Sketch.
- forthispurpose 7y agoSince it is completely automated and the code is doing all the processing, why is the price so high and why is there even a charge is per artboard? Sketch is like 1/10th of that
- johndamaia 7y agoNot completely automated at this stage. Check my last paragraph on my first comment here. As soon we can make it work 100% automatically we will have much more freedom to change our pricing model.
- tasubotadas 7y agoThe fact that operational costs are low doesn't mean that development costs can now be ignored.
- mrhappyunhappy 7y agoI predict that one day you'll be able to input your content, specify some site layouts you like and press a button to generate a completely ujique website based on your content, fully designed and coded. The first person to make that is going to be very rich.
- mLuby 7y agoFiredrop was doing that, but it seems they've pivoted to something else. As I recall, it was literally upload your text and photo/video content, and their AI will design a static site for you, optimizing for SEO and engagement.
- aasasd 7y agoIt would help if there were examples of input and output—images of Sketch files and HTML pages that were generated (without manual fixes, if that's an option in your offers).
- baalimago 7y agoExcuse my pessimism in this sea of optimism, but I see a .sketch-> vue.js converter which has proprietary elements of DL which later on gets checked with manual work anyway. Where's the building of websites? What's the difference between this solution and something which simply converts .sketch into html with far simpler methods?
- iamleppert 7y agoHow many FTE engineers can this replace? Do you have a case study or whitepaper?
- johndamaia 7y agoOur goal is not to replace engineers. Is to increase their productivity and give them more time to create awesome products.
- Kerrick 7y agoSure, but every bit of automation that "increases productivity" and "gives more time for other things" naturally ends up replacing those that it was helping -- if not directly, at least replacing potential future hires in the same role.
- blabla321 7y agoA solution for a problem that's not really a problem... Garbage in garbage out. As for the automation piece, surely a bunch of if/else will do. Not exactly rocket science
- thrownaway954 7y agowouldn't be surprised if this dude is bought out by Invision in the coming year or two.
- modo_ 7y agoSorry to be a bit meta here-- but a question about the design of the site. This style of pastel colors, wavy svg graphics, and happy avatars seems super popular right now [1] [2] [3]. I'm curious if it has a name and where it originated from (if anywhere?) Overall I like the aesthetic quite a bit, but it's such a distinctive look. I'm surprised I keep seeing it crop up nearly copy+pasted between all of these companies [1] https://www.gusto.com https://www.gusto.com [2] https://www.tandem.chat https://www.tandem.chat [3] https://www.notion.so https://www.notion.so
- jayventura 7y agoYou'll see a similar thing in branding and advertising. For all the new yorkers on here, have you all seen the consumer brand ads on the subway? Almost every brand looks the same. I'm imagining it has to do with the way designers work. They generally have stakeholders moodboard (basically creating a pinterest board with inspiration from different brands), and then they create a similar "feeling". As humans, we're kind of evolved to recognize patterns, so when we start seeing the same branding art direction it stands out to us and we may try to conflate some deeper meaning to it. But in reality, it's just a few brands that share the same branding art direction probably because some execs liked the way it looked and felt and asked their designers to do something similar.
- admn2 7y agoThis style of web design is super in at the moment. Not sure it has a specific name, but I believe the previous Intercom site started it. Possibly Wealthfront as well. A lot of companies seem to be using it to take more risks with their aesthetic, particularly Dropbox and Mailchimp.
- reneherse 7y agoAnother question about the design of the page: Does this kind of conversational / persona based pitch actually work to convert users? E.G. "Hi I'm Clara, my life's goal is... I struggle with..." "If you relate with Clara, try X now..." At least for me the net emotional effect of the "For Desigers & Developers" section of the homepage was a strong impulse to bounce. (I'm in the target market.) Ultimately for the sake of research I clicked through to the ~/product page, where I found the diagrammatic graphics much more compelling.
- xcubic 7y agoHey john. Congrats on this, this is very interesting. From someone interested in ML/AI, can you explain just a bit more how the deep learning works?
- johndamaia 7y agoWithout going into detail, our model outputs the coordinates of the html elements it recognize. We are taking a visual approach to the problem.
- katsura 7y agoSorry to be that guy, but fullstory, google tagmanager, and hotjar all on the same page, without any indication to the user that their every move is being tracked. This site is not too GDPR compatible. Also, I'm working on something similar, but from a different perspective. My problem with this is that you have to design your website in Sketch, then turn it into code with a different service potentially waiting hours until you'll see the end result. So my idea is that I can build an app where you can basically draw the page elements, similarly to how you'd do it in Sketch, and it automagically turns the whole thing into a responsive website right away (no position:absolute hacks involved). I still have a lot to do though... :D
- egman_ekki 7y agoSeems a bit similar to what Avocode was/is trying to solve. Haven't followed them closely recently, but seems like they pivoted from pure design->code tool to design & collaboration tool that enables easy export to code...
- gitgud 7y agoHi John, great idea but the demo didn't really convince me. How much of the system relies on engineers cleaning up the code? And I can't get past the fact that it becomes a responsive site, is the logic in the responsive design in the Sketch file too?
- johndamaia 7y agoHey! No worries, I'm starting to like the skepticism, it makes the job funnier. Right now we are doing it right ~90% of the whole page. The responsiveness is done by the system automatically (it makes decisions on its own, therefore they could be wrong). No need to have it on the Sketch file.
- gitgud 7y agoI guess when you present magic, people get skeptical ;) But seriously, I hope you guys do well, thanks for responding to all these skeptical comments
- sombremesa 7y agoYour rates are higher than paying someone (offshore) to convert the artboards - why? Should be much cheaper if it's automated.
- johndamaia 7y agoWow, do you know developers that convert artboards to full webpage for $9? But I understand your point, we are intentionally pricing it at a high point. Hopefully we'll be able to decrease it in the future.
- sombremesa 7y agoThe definition of "full webpage" can be scaled out of that price point, but yes. I wish you best of luck.
- bkyan 7y agoI'm curious how you are handling ambiguity in the original design. Does the algorithm create multiple variants when something could be interpreted in multiple ways, and then the manual validation part of the process involves picking from among the different variants that the algorithm created?
- johndamaia 7y agoIt's simpler than that. The model makes only one choice based on what it learnt. However you gave an excellent idea :)
- Nohak 7y agoAlways a pleasure to see others end up with a project that I've working on myself for 2 years on my free time... Don't expect to have the market... I'll. (even if I have to set prices twice cheaper than yours). Sorry, I was working on this project for... 2 years on my free time. I can't fail. I don't matter if I "steal" your customers or what. Because during these 2 years, I was working for my firm, then I was going to workout for 1 hour, then I was working on this project for 3 hours, then bedtime. My weekends and holidays were dedicated-only to the project dev. I didn't have any social life for 2 years in order to work on a Websites generator. And YOU come with your own sites generator????? And you created it with a team, not me I was alone. I will have the market for me only. Too many sacrifices.
- deleted 7y ago[deleted]
- miguelrochefort 7y agoI once built a tool that generates XAML from Sketch files and Zeplin projects. I can now see how AI would have made my job easier.