6 ms·

100%. People who have not tried to interact with these models don't understand there is actually quite a lot of skill in navigating the latent space.

mdp2021 4y ago
It really should be just an intermediate phase. Proper, solid control system must be created. First you have Stable Diffusion, then you create ControlNet - and this as just a beginning. This is also one of the reasons why the problem of transparency ("but why and how does it work?") is so important in Machine Learning: allowing control.
- ZephyrBlu 4y ago
  Adding a working control system doesn't take away the fact it requires skill though. It seems like people think that one day they will be able to give a vague prompt and generate exactly what they were looking for, which I don't believe will ever happen. It feels like there will always be some skill in navigating the model.
  - mdp2021 4y ago
    > there will always be some skill There is a world of difference between the needs in governance in just having to write "Those decorations where I marked the area, make them a bit more Hans Holbein as opposed to Klimt", compared to knowing patching tricks so that if you asked the engine to draw a 'T' you were not being served a 'B'. This latter example - ask 'B', get 'T' - I have seen from Midjourney only a few hours ago, in the public showcase. I have examined several prompts and results, and very often they do not overlap but very partially - the engine is like a wild horse. That this is not intrinsic, but just a current stage, is only part of the story: the user will want to have a tool that allows the production of what the artist had exactly in mind (not just "anything nice as /inspired/ by our suggestions"). It is the normal form of the product in its "ready for shipment" state.
    - ZephyrBlu 4y ago
      I disagree it will ever be that easy for people to generate specifically what they’re looking for. I believe there will always be some amount of navigation of the latent space required, and people who don’t understand how to navigate it will struggle. To use your analogy, even if the horse is tame you still need to know how to ride it.
      - mdp2021 4y ago
        > that easy Can you code? Can you illustrate? Can you build? Piece by piece, you sketch, implement, hone, finetune, improve, correct, retry, go into detail, expand... The ease of the process is relative to the tools adopted. Of course there has to be knowledge behind it. Of course you have to know your tools. But "wild horse" and "diligent consultant", axe and scalpel require different degrees areas and modalities of competence. A well built tool will build on the competences already natural for the professional (e.g. a graphic designer will be able to sketch and to instruct the machine in a reasonably standard language). Of course the better you know your tool, the greater the effectiveness. But currently, the systems can be strongly unreliable - they do not respect the request. The user will have to learn to deal with a machine taking natural language as an input just like any manager has to know how to speak to personnel to obtain what is wanted. On the other hand, the implementation of "virtual personnel" which is reliable is a responsibility of the tool developers.
        ZephyrBlu 4y ago
        My point is that even with improved tooling I don't think it will be accessible to a layman. You will need to develop some skill to wield the tool. It seems like you agree based on parts of your comment like "A well built tool will build on the competences already natural for the professional".
        deleted 4y ago
        [deleted]
    - dragonwriter 4y ago
  - throwaway675309 4y ago
    If by "some skill" you mean the basic ability to type vaguely coherent English, than I agree. If anything the recent generative art advancements coming out (Firefly, MJ v5) would seem to refute your point. There was a time when diffusion models require some level of knowledge and skill to use. To get high quality output, it was in your best interest to learn about the different types of samplers, upscalers, hyper networks, textual inversions, understanding denoising etc. etc. Now? I'd say the millions of active users on MJ would seem to prove you wrong. As far as language models like GPT, we've already seen the average level of skill needed come down with each successive release, ChatGPT is easier to use than GPT-3 which is easier to use than GPT-2. Furthermore, devs in the space (Stability, OpenAI, etc) could refine/train pre-processing LLM models that transform more accessible amateur prompts to more professional prompts at real-time.
    - ZephyrBlu 4y ago
      I've used MJ quite a lot. It's easy to create something. It's quite difficult to create a specific thing.
    - mdp2021 4y ago
      > pre-processing LLM models that transform more accessible amateur prompts Dangerous! It looks like instead of driving directly, you are tapping on the shoulders of the one at the steering wheel. Which is useful for children, and frustrating for drivers.
- SkyPuncher 4y ago
  Somethings will be easier, but it's still a lot like programming. You often need to do a lot of setup to accomplish a task. For somethings, there's just no way around creating a specific context. In many ways, it's the difference between using a library like lodash and writing the methods yourself. You still need to understand what's being done, a library just handles more things automatically for you.
- toss1 4y ago
  This sounds like a joke I heard decades ago when I first worked with LISP programmers - when I commented about some unexpected result, he said "ah, you're looking for the DWIM command..." "What?"... "The Do What I Mean command, they're still working on it..." Decades later, and it looks like everyone is still working on the DWIM command...
rasz 4y ago
Just like you wouldnt be able to drive, let alone start Ford T.

mdp2021 4y ago

It really should be just an intermediate phase. Proper, solid control system must be created. First you have Stable Diffusion, then you create ControlNet - and this as just a beginning. This is also one of the reasons why the problem of transparency ("but why and how does it work?") is so important in Machine Learning: allowing control.

ZephyrBlu 4y ago
Adding a working control system doesn't take away the fact it requires skill though. It seems like people think that one day they will be able to give a vague prompt and generate exactly what they were looking for, which I don't believe will ever happen. It feels like there will always be some skill in navigating the model.
- mdp2021 4y ago
  > there will always be some skill There is a world of difference between the needs in governance in just having to write "Those decorations where I marked the area, make them a bit more Hans Holbein as opposed to Klimt", compared to knowing patching tricks so that if you asked the engine to draw a 'T' you were not being served a 'B'. This latter example - ask 'B', get 'T' - I have seen from Midjourney only a few hours ago, in the public showcase. I have examined several prompts and results, and very often they do not overlap but very partially - the engine is like a wild horse. That this is not intrinsic, but just a current stage, is only part of the story: the user will want to have a tool that allows the production of what the artist had exactly in mind (not just "anything nice as /inspired/ by our suggestions"). It is the normal form of the product in its "ready for shipment" state.
  - ZephyrBlu 4y ago
    I disagree it will ever be that easy for people to generate specifically what they’re looking for. I believe there will always be some amount of navigation of the latent space required, and people who don’t understand how to navigate it will struggle. To use your analogy, even if the horse is tame you still need to know how to ride it.
    - mdp2021 4y ago
      > that easy Can you code? Can you illustrate? Can you build? Piece by piece, you sketch, implement, hone, finetune, improve, correct, retry, go into detail, expand... The ease of the process is relative to the tools adopted. Of course there has to be knowledge behind it. Of course you have to know your tools. But "wild horse" and "diligent consultant", axe and scalpel require different degrees areas and modalities of competence. A well built tool will build on the competences already natural for the professional (e.g. a graphic designer will be able to sketch and to instruct the machine in a reasonably standard language). Of course the better you know your tool, the greater the effectiveness. But currently, the systems can be strongly unreliable - they do not respect the request. The user will have to learn to deal with a machine taking natural language as an input just like any manager has to know how to speak to personnel to obtain what is wanted. On the other hand, the implementation of "virtual personnel" which is reliable is a responsibility of the tool developers.
      - ZephyrBlu 4y ago
        My point is that even with improved tooling I don't think it will be accessible to a layman. You will need to develop some skill to wield the tool. It seems like you agree based on parts of your comment like "A well built tool will build on the competences already natural for the professional".
        deleted 4y ago
        [deleted]
  - dragonwriter 4y ago
- throwaway675309 4y ago
  If by "some skill" you mean the basic ability to type vaguely coherent English, than I agree. If anything the recent generative art advancements coming out (Firefly, MJ v5) would seem to refute your point. There was a time when diffusion models require some level of knowledge and skill to use. To get high quality output, it was in your best interest to learn about the different types of samplers, upscalers, hyper networks, textual inversions, understanding denoising etc. etc. Now? I'd say the millions of active users on MJ would seem to prove you wrong. As far as language models like GPT, we've already seen the average level of skill needed come down with each successive release, ChatGPT is easier to use than GPT-3 which is easier to use than GPT-2. Furthermore, devs in the space (Stability, OpenAI, etc) could refine/train pre-processing LLM models that transform more accessible amateur prompts to more professional prompts at real-time.
  - ZephyrBlu 4y ago
    I've used MJ quite a lot. It's easy to create something. It's quite difficult to create a specific thing.
  - mdp2021 4y ago
    > pre-processing LLM models that transform more accessible amateur prompts Dangerous! It looks like instead of driving directly, you are tapping on the shoulders of the one at the steering wheel. Which is useful for children, and frustrating for drivers.
SkyPuncher 4y ago
Somethings will be easier, but it's still a lot like programming. You often need to do a lot of setup to accomplish a task. For somethings, there's just no way around creating a specific context. In many ways, it's the difference between using a library like lodash and writing the methods yourself. You still need to understand what's being done, a library just handles more things automatically for you.
toss1 4y ago
This sounds like a joke I heard decades ago when I first worked with LISP programmers - when I commented about some unexpected result, he said "ah, you're looking for the DWIM command..." "What?"... "The Do What I Mean command, they're still working on it..." Decades later, and it looks like everyone is still working on the DWIM command...

rasz 4y ago

Just like you wouldnt be able to drive, let alone start Ford T.