4 ms·

Generate designs from prompts based on a design system

15 points by ishan28mkip 3y ago

ishan28mkip 3y ago
Hey everyone for the past couple of months I have been working on a design tool that generates designs from prompts based on a design system. As a UI engineer of 5 years I have been obsessing over the fact that if AI is generating designs then they should be based on a design system, much like co-pilot suggests code based on the pre existing codebase. If you're interested in trying it out, you can sign up for early access on http://magify.design http://magify.design. Would be really glad to hear your feedback and thoughts on this tool, and I hope it can be used to take out the tedious parts of design.
skrish13 3y ago
Interesting! Can it work with any design system file? or is there some specific structure, naming and details that are needed for the model to understand and place it on designs?
- ishan28mkip 3y ago
  Right now design system has to be manually defined as in a particular file and components in the design system have to named a certain way but I am trying to automate the process so that this manual effort doesn't need to be done. But maybe I can let it be like this also as it is only a one time effort and I am planning on providing several pre-named starter design systems.
  - skrish13 3y ago
    Automating that would be helpful but I'm not sure how. Similar Text2SQL models require full description of each fields in the database. That is somewhat automated but in real world systems, it still requires manual explanation since they have the context on it.
    - ishan28mkip 3y ago
      Yeah I am also not exactly sure how I would go automate that process. Maybe I can use the figma code of the design system component and then run it through some classifier to classify it. I will work on it as soon as I release the beta.
pallavbh23 3y ago
This looks really promising! Looking forward to a future where apps can be deployed 100x faster! :D
- ishan28mkip 3y ago
  I don't know about 100x faster app deployments but I am sure it can help create faster designs for freelance projects. Trying to understand the needs of complex designs in enterprises and how it can be incorporated to so solve more complex design problems.
Lemmechecknow 3y ago
How are you generating those screens from prompts or like Steve using ai to make (builder.io)
- ishan28mkip 3y ago
  It is generative. I have used a combination of LLM, fined tuned text to image models and a model that tags the components from design systems. Post this I am writing code to convert this to figma components using figma API. Also using some heuristic algorithms to correct for layout artefacts.
anshudweeep 3y ago
As a designer, I have seen lot of AI tools with a promise of delivering and none do. They just solve a UI generation problem that is not really a problem.
deleted 3y ago
[deleted]
meinanshu 3y ago
Are you using GPT4 behind the scene?
- ishan28mkip 3y ago
  As mentioned in another comment I am using LLM, fine tuned text to image and a model tagging components. I haven't been able to get GPT4 API access as of yet but have applied and plan on shifting the LLM to GPT4 once I receive the access.
gerardpaul 3y ago
What will be the expected wait time? Looking to try it as fast as possible.
- ishan28mkip 3y ago
  I am trying my best to get the beta out as early as possible. I just need to get hold of some GPU to train the larger model to make it more generalised and usable for a public release. I am either going to apply for some grants or do some consultancy project or maybe even raise funding.
valliappan 3y ago
how are you generating coherent ux copies?
- ishan28mkip 3y ago
  For UX writing I am using GPT giving it context from the design and surrounding UX copy.

15 points by ishan28mkip 3y ago

Hey everyone for the past couple of months I have been working on a design tool that generates designs from prompts based on a design system. As a UI engineer of 5 years I have been obsessing over the fact that if AI is generating designs then they should be based on a design system, much like co-pilot suggests code based on the pre existing codebase. If you're interested in trying it out, you can sign up for early access on http://magify.design http://magify.design. Would be really glad to hear your feedback and thoughts on this tool, and I hope it can be used to take out the tedious parts of design.

skrish13 3y ago

Interesting! Can it work with any design system file? or is there some specific structure, naming and details that are needed for the model to understand and place it on designs?

ishan28mkip 3y ago
Right now design system has to be manually defined as in a particular file and components in the design system have to named a certain way but I am trying to automate the process so that this manual effort doesn't need to be done. But maybe I can let it be like this also as it is only a one time effort and I am planning on providing several pre-named starter design systems.
- skrish13 3y ago
  Automating that would be helpful but I'm not sure how. Similar Text2SQL models require full description of each fields in the database. That is somewhat automated but in real world systems, it still requires manual explanation since they have the context on it.
  - ishan28mkip 3y ago
    Yeah I am also not exactly sure how I would go automate that process. Maybe I can use the figma code of the design system component and then run it through some classifier to classify it. I will work on it as soon as I release the beta.

pallavbh23 3y ago

This looks really promising! Looking forward to a future where apps can be deployed 100x faster! :D

ishan28mkip 3y ago
I don't know about 100x faster app deployments but I am sure it can help create faster designs for freelance projects. Trying to understand the needs of complex designs in enterprises and how it can be incorporated to so solve more complex design problems.

Lemmechecknow 3y ago

How are you generating those screens from prompts or like Steve using ai to make (builder.io)

ishan28mkip 3y ago
It is generative. I have used a combination of LLM, fined tuned text to image models and a model that tags the components from design systems. Post this I am writing code to convert this to figma components using figma API. Also using some heuristic algorithms to correct for layout artefacts.

anshudweeep 3y ago

As a designer, I have seen lot of AI tools with a promise of delivering and none do. They just solve a UI generation problem that is not really a problem.

deleted 3y ago

[deleted]

meinanshu 3y ago

Are you using GPT4 behind the scene?

ishan28mkip 3y ago
As mentioned in another comment I am using LLM, fine tuned text to image and a model tagging components. I haven't been able to get GPT4 API access as of yet but have applied and plan on shifting the LLM to GPT4 once I receive the access.

gerardpaul 3y ago

What will be the expected wait time? Looking to try it as fast as possible.

ishan28mkip 3y ago
I am trying my best to get the beta out as early as possible. I just need to get hold of some GPU to train the larger model to make it more generalised and usable for a public release. I am either going to apply for some grants or do some consultancy project or maybe even raise funding.

valliappan 3y ago

how are you generating coherent ux copies?

ishan28mkip 3y ago
For UX writing I am using GPT giving it context from the design and surrounding UX copy.