3 ms·

Hi Lina, Andrew and Sidney, this is awesome. My go-to for checking the edges of video and face identification LLMs are Personas right now -- they're rendered f

Hi Lina, Andrew and Sidney, this is awesome. My go-to for checking the edges of video and face identification LLMs are Personas right now -- they're rendered faces done in a painterly style, and can be really hard to parse. Here's some output: https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/67ece032-f495-43a1-8d50-6fee07fc92cd-Ra0Cg3WWofQlxbOPujQLhuiq26WHY9.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... Source image from: https://personacollective.ai/persona/1610 https://personacollective.ai/persona/1610 Overall, crazy impressive compared to competing offerings. I don't know if the mouth size problems are related to the race of the portrait, the style, the model, or the positioning of the head, but I'm looking forward to further iterations of the model. This is already good enough for a bunch of creative work, which is rad.

lcolucci 2y ago
I didn't know about Persona Collective - very cool! I think the issues in your video are more related to the style of the image and the fact that she's looking sideways than the race. In our testing so far, it's done a pretty good job across races. The stylized painting aesthetic is one of the harder styles for the model to do well on. I would recommend trying with a straight on portrait (rather than profile) and shorter generations as well... it might do a bit better there. Our model will also get better over time, but I'm glad it can already be useful to you!
- vessenes 2y ago
  It's not portrait orientation or gender specific or length related: https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/411c024c-ee88-4b62-98cf-40fbe4acd63e-yUh6ikmSvye80QiZqBdK9jeehxhYa1.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... It's not stylization (alone): here's a short video using the same head proportions as the original video, but the photo style is a realistic portrait. I'd say the mouth is still overly wide. https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/83dec65b-0e5d-4b9c-bb5d-5fa01f215dfb-yysJ2upBY0NRIPyiuQF6REh9cbnA6o.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... I tentatively think it might be race related -- this is one done of a different race. Her mouth might also be too wide? But it stands out a bit less to me. https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/33846a6e-84c7-4be8-8f80-351b9ee8c96e-sgmpYbyAlTUt08N0moJcxZmfUpp003.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... p.s. happy to post a bug tracker / github / whatever if you prefer. I'm also happy to license over the Persona Collective images if you want to pull them in for training / testing -- : feel free to email me -- there's a move away from 'painterly' style support in the current crop of diffusion models (flux for instance absolutely CANNOT do painting styles), and I think that's a shame. Anyway, thanks! I really like this.

lcolucci 2y ago

I didn't know about Persona Collective - very cool! I think the issues in your video are more related to the style of the image and the fact that she's looking sideways than the race. In our testing so far, it's done a pretty good job across races. The stylized painting aesthetic is one of the harder styles for the model to do well on. I would recommend trying with a straight on portrait (rather than profile) and shorter generations as well... it might do a bit better there. Our model will also get better over time, but I'm glad it can already be useful to you!

vessenes 2y ago
It's not portrait orientation or gender specific or length related: https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/411c024c-ee88-4b62-98cf-40fbe4acd63e-yUh6ikmSvye80QiZqBdK9jeehxhYa1.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... It's not stylization (alone): here's a short video using the same head proportions as the original video, but the photo style is a realistic portrait. I'd say the mouth is still overly wide. https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/83dec65b-0e5d-4b9c-bb5d-5fa01f215dfb-yysJ2upBY0NRIPyiuQF6REh9cbnA6o.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... I tentatively think it might be race related -- this is one done of a different race. Her mouth might also be too wide? But it stands out a bit less to me. https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2-clips/33846a6e-84c7-4be8-8f80-351b9ee8c96e-sgmpYbyAlTUt08N0moJcxZmfUpp003.mp4 https://6ammc3n5zzf5ljnz.public.blob.vercel-storage.com/inf2... p.s. happy to post a bug tracker / github / whatever if you prefer. I'm also happy to license over the Persona Collective images if you want to pull them in for training / testing -- : feel free to email me -- there's a move away from 'painterly' style support in the current crop of diffusion models (flux for instance absolutely CANNOT do painting styles), and I think that's a shame. Anyway, thanks! I really like this.