6 ms·

Show HN: Convert sketches into images using ControlNet

76 points by herval 4y ago

Hi HN! We're working on an AI studio product, and have been experimenting with the recently-released ControlNet (https://github.com/lllyasviel/ControlNet https://github.com/lllyasviel/ControlNet) After wasting hours with it, we decided to put together a quick fun project. "Sketch" allows you to draw anything (works w/ touch inputs on mobile too), enter a prompt (it can be as simple as "drawing" and it'll try to guess) and you get something out fast. Keeping it all on a public feed as well, to serve as inspiration. Hope anyone here finds it fun :-)

chr-s 4y ago
"A very racist caricature of the prophet Mohammed"
- 22c 4y ago
  Yes a suggestion to OP would be to remove the public feed ASAP, it's too juicy a target for abuse. Even if you try to ban certain terms, people will find a way around it and you'll be in constant game of whac-a-troll. Remove the public feed and they are likely to lose interest.
  - _scat 4y ago
    [flagged]
  - jackblemming 4y ago
    You know, people used to be ok with having a bit of fun on the internet. Not everything needed to be censored or sanitized, even if it meant you saw a naughty or mean word occasionally.
    - techdragon 4y ago
      More people… more assholes. It’s an unfortunate situation that we can’t all just be better people on all fronts, better at not abusing systems, better at taking a joke, better at accepting the failures of others, better at holding ourselves to account… But at the end of the day, prevention beats cure and it’s not too hard to do, even if it does take a little bit of the spontaneity out of things.
    - rpastuszak 4y ago
      > You know, people used to be ok with having a bit of fun on the internet They still do, but their definition of fun is different than an equivalent of doodling a swastika on a bar urinal. I’ll tell the manager to stop cleaning those. Perhaps spend a moment asking yourself next time what’s so fun here, and at whose expense. Vulgarity is fine, xenophobia isn’t.
    - 22c 4y ago
      I think you misremember the internet. There were and are still very few places on the internet (outside of hosting your own platform) where you can post whatever you want without recourse. No woodworking forum would allow a thread of "black man assaulting asians on the subway". You wouldn't see "jews stealing money" in comp.lang.perl. Removing a submission from your own website that goes against the spirit of the site is not censorship any more than cleaning swastika graffiti off your front door is censorship.
    - dylan604 4y ago
      I think it's less about the intent of people having fun, but other people's reception of that fun.
gedy 4y ago
Thanks for this. Super cool but the public feed shows the true hearts of mankind... lol may want to hide that.
fastball 4y ago
Are you also using Stable Diffusion under-the-hood? Because the results in the public feed look much less... coherent than the examples in the ControlNet Github repo (including examples using sketches as the control system).
- herval 4y ago
  yes! there's some new params added to the controlnet demo which I didnt have the time to incorporate yet, will do that later today. Makes the images much more coherent.
numpad0 4y ago
It seems public feed isn't updating for an hour, maybe backend is down or rate-limited? 0: https://imgur.com/a/nwqUZHT https://imgur.com/a/nwqUZHT
- pfoof 4y ago
  By looking at public prompts, I believe the reason is different...
pfoof 4y ago
Very cool tool but the public feed looks like every day on 4chan.
SillyUsername 4y ago
By fast do you mean >3 mins and not showing progress, updating or returning after 4+ minutes?
- herval 4y ago
  It got overloaded with juvies posting dick pics and went OOM. It takes ~1-5 seconds to generate, normally
thrdbndndn 4y ago
As an automation engineer, I'd say ControlNet is really an unfortunate name.
- dylan604 4y ago
  would Skynet have more to your liking?
herval 4y ago
Thanks for the load-test, HN! Time to improve the public feed (and the sketch engine) :-)
Larrikin 4y ago
Would be nice to have downloadable images of the original sketch as well, or a side by side. There doesn't seem to be a way to share what you drew and the results with someone
- herval 4y ago
  working on it :-) thanks for the feedback!
KomoD 4y ago
Every time I visit the site I get: "Application error: a client-side exception has occurred"
- herval 4y ago
  can you check if there's any error on the web console? tks
  - KomoD 4y ago
    Yes: "ReferenceError: user is not defined" Rest on pastebin: https://pastebin.com/CHxFRXd2 https://pastebin.com/CHxFRXd2
    - herval 4y ago
      should be fixed now! thanks for spotting that <3
deleted 4y ago
[deleted]
gus_massa 4y ago
Try to change the title to exactly "Show HN: Convert sketches into images using ControlNet", so it's shown also in the special section.
- herval 4y ago
  thanks! Just did that :-)
  - gus_massa 4y ago
    It needs an space between "Show HN:"
- deleted 4y ago
  [deleted]
gus_massa 4y ago
I tried to made a few tables. Regular tables are fine, but triangular tables are very confusing for the AI :) I tried to grab a permalink, but there it's not supported https://imgur.com/a/lMcl77S https://imgur.com/a/lMcl77S Also, it would be nice to be able to retrieve my original drawing to see how horrible it is.
- dylan604 4y ago
  "I know there's meant to be a fourth leg on this thing damnit!!"
  - gus_massa 4y ago
    It's strange because one of the common problems of this type of AI is that it can't count, so hands may have an unusual number of fingers. I think my original drawing has only three legs. Two "long" legs in the extremes and a "short" one in the middle.
nnextai 4y ago
For those looking to learn the inner workings of ControlNet, here's a brief introduction. https://nnext.ai/wiki/ControlNet-Explained-3842b7f090a046eda565a55d7a01ec03 https://nnext.ai/wiki/ControlNet-Explained-3842b7f090a046eda...

76 points by herval 4y ago

chr-s 4y ago

"A very racist caricature of the prophet Mohammed"

22c 4y ago
Yes a suggestion to OP would be to remove the public feed ASAP, it's too juicy a target for abuse. Even if you try to ban certain terms, people will find a way around it and you'll be in constant game of whac-a-troll. Remove the public feed and they are likely to lose interest.
- _scat 4y ago
  [flagged]
- jackblemming 4y ago
  You know, people used to be ok with having a bit of fun on the internet. Not everything needed to be censored or sanitized, even if it meant you saw a naughty or mean word occasionally.
  - techdragon 4y ago
    More people… more assholes. It’s an unfortunate situation that we can’t all just be better people on all fronts, better at not abusing systems, better at taking a joke, better at accepting the failures of others, better at holding ourselves to account… But at the end of the day, prevention beats cure and it’s not too hard to do, even if it does take a little bit of the spontaneity out of things.
  - rpastuszak 4y ago
    > You know, people used to be ok with having a bit of fun on the internet They still do, but their definition of fun is different than an equivalent of doodling a swastika on a bar urinal. I’ll tell the manager to stop cleaning those. Perhaps spend a moment asking yourself next time what’s so fun here, and at whose expense. Vulgarity is fine, xenophobia isn’t.
  - 22c 4y ago
    I think you misremember the internet. There were and are still very few places on the internet (outside of hosting your own platform) where you can post whatever you want without recourse. No woodworking forum would allow a thread of "black man assaulting asians on the subway". You wouldn't see "jews stealing money" in comp.lang.perl. Removing a submission from your own website that goes against the spirit of the site is not censorship any more than cleaning swastika graffiti off your front door is censorship.
  - dylan604 4y ago
    I think it's less about the intent of people having fun, but other people's reception of that fun.

gedy 4y ago

Thanks for this. Super cool but the public feed shows the true hearts of mankind... lol may want to hide that.

fastball 4y ago

Are you also using Stable Diffusion under-the-hood? Because the results in the public feed look much less... coherent than the examples in the ControlNet Github repo (including examples using sketches as the control system).

herval 4y ago
yes! there's some new params added to the controlnet demo which I didnt have the time to incorporate yet, will do that later today. Makes the images much more coherent.

numpad0 4y ago

It seems public feed isn't updating for an hour, maybe backend is down or rate-limited? 0: https://imgur.com/a/nwqUZHT https://imgur.com/a/nwqUZHT

pfoof 4y ago
By looking at public prompts, I believe the reason is different...

pfoof 4y ago

Very cool tool but the public feed looks like every day on 4chan.

SillyUsername 4y ago

By fast do you mean >3 mins and not showing progress, updating or returning after 4+ minutes?

herval 4y ago
It got overloaded with juvies posting dick pics and went OOM. It takes ~1-5 seconds to generate, normally

thrdbndndn 4y ago

As an automation engineer, I'd say ControlNet is really an unfortunate name.

dylan604 4y ago
would Skynet have more to your liking?

herval 4y ago

Thanks for the load-test, HN! Time to improve the public feed (and the sketch engine) :-)

Larrikin 4y ago

Would be nice to have downloadable images of the original sketch as well, or a side by side. There doesn't seem to be a way to share what you drew and the results with someone

herval 4y ago
working on it :-) thanks for the feedback!

KomoD 4y ago

Every time I visit the site I get: "Application error: a client-side exception has occurred"

herval 4y ago
can you check if there's any error on the web console? tks
- KomoD 4y ago
  Yes: "ReferenceError: user is not defined" Rest on pastebin: https://pastebin.com/CHxFRXd2 https://pastebin.com/CHxFRXd2
  - herval 4y ago
    should be fixed now! thanks for spotting that <3

deleted 4y ago

[deleted]

gus_massa 4y ago

Try to change the title to exactly "Show HN: Convert sketches into images using ControlNet", so it's shown also in the special section.

herval 4y ago
thanks! Just did that :-)
- gus_massa 4y ago
  It needs an space between "Show HN:"
deleted 4y ago
[deleted]

gus_massa 4y ago

I tried to made a few tables. Regular tables are fine, but triangular tables are very confusing for the AI :) I tried to grab a permalink, but there it's not supported https://imgur.com/a/lMcl77S https://imgur.com/a/lMcl77S Also, it would be nice to be able to retrieve my original drawing to see how horrible it is.

dylan604 4y ago
"I know there's meant to be a fourth leg on this thing damnit!!"
- gus_massa 4y ago
  It's strange because one of the common problems of this type of AI is that it can't count, so hands may have an unusual number of fingers. I think my original drawing has only three legs. Two "long" legs in the extremes and a "short" one in the middle.

nnextai 4y ago

For those looking to learn the inner workings of ControlNet, here's a brief introduction. https://nnext.ai/wiki/ControlNet-Explained-3842b7f090a046eda565a55d7a01ec03 https://nnext.ai/wiki/ControlNet-Explained-3842b7f090a046eda...