6 ms·

My chosen profession (3D / filmmaking) feels like being in some kind of combat trench at the moment. Both fascinating and scary

rizky05 3y ago
[dead]
nextworddev 3y ago
What do you ascertain the use case of this in your field? Does it seem high quality? (I have no context)
- zavertnik 3y ago
  I'm not a professional in VFX, but I work in television and do a lot of VFX/3D work on the side. The quality isn't amazing, but it looks like this could be the start of a Midjourney-tier VFX/3D LLM, which would be awesome. For me, this would help bridge the gap between having to use/find premade assets and building what I want. For context, building from scratch in a 3D pipeline requires you to wear a lot of different hats (modeling, materials, lighting, framing, animating, ect). It costs a lot of time to get to not only learn these hats but also use them together. The individual complexity of those skill sets makes it difficult to experiment and play around, which is how people learn with software. The shortcut is using premade assets or addons. For instance, being able to use the Source game assets in Source Filmmaker combined with SFM using a familiar game engine makes it easy to build an intuition with the workflow. This makes Source Filmmaker accessible and its why theres so much content out there made with it. So if you have gaps in your skillset or need to save time, you'll buy/use premade assets. This comes at a cost of control, but that's always been the tradeoff between building what you want and building with what you have. Just like GPT and DALL-E built a bridge between building what you want and building with what you have, a high fidelity GPT for the 3D pipeline would make that world so much more accessible and would bring the kind of attention NLE video editing got in the post-Youtube world. If I could describe in text and/or generate an image of a scene I want and have a GPT create the objects, model them, generate textures, and place them in the scene, I could suddenly just open blender, describe a scene, and just experimenting with shooting in it, as if I was playing in a sandbox FPS game. I'm not sure if MeshGPT is the ChatGPT of the 3D pipeline, but I do think this is kind of content generation is the conduit for the DALL-E of video that so many people are terrified and/or excited for.
  - gavinray 3y ago
    On an unrelated note, could I ask your opinion? My wife is passionate about film/TV production and VFX. She's currently in school for this but is concerned about the difficulty of landing a job afterwards. Do you have any recommendations on breaking into the industry without work experience?
    - kranke155 3y ago
      As a producer? Huh. That’s such a great question. I think producer roles are a little bit less ultra competitive / scarce as they are actually jobs jobs where you have to use excel and planning and budgeting. Being a producer means being on the phone all the time, negotiating, haggling, finding solutions where they don’t seem to exist. Be it in TV, advertising or somewhere in the media space, the common rule is that producers are mostly actually terrible at their jobs, that’s my experience in London. So if she’s really good and really dedicated and learns the job of everyone on set, I’d say she has a shot. The real secret to being good in filmmaking is learning everyone else’s job. Toyota Production System says if you want to run a production line you have to know how it works. If she wants to do VFX production she could start doing her own test scenes, learning basics in nuke and Blender, even understanding the role of Houdini and how that works. If she does that - any company will be lucky to have her.
- deleted 3y ago
  [deleted]
bsenftner 3y ago
So you're probably familiar with the role of a Bidding Producer; imagine the difficulty they are facing: on one side they have filmmakers saying they just read so and so is now created by AI, while that is news to the bidding producer and their VFX/animation studio clients scrambling as everything they do is new again.
sheepscreek 3y ago
Perhaps one way to look at this could be auto-scaffolding. The typical modelling and CAD tools might include this feature to get you up and running faster. Another massive benefit is composability. If the model can generate a cup and a table, it also knows how to generate a cup on a table. Think of all the complex gears and machine parts this could generate in the blink of an eye, while being relevant to the project - rotated and positioned exact where you want it. Very similar to how GitHub Copilot works.
worldsayshi 3y ago
I don't see that LLM's have come that much further in 3D animation than programming in this regard: It can spit out bits and pieces that looks okay in isolation but a human need to solve the puzzle. And often solving the puzzle means rewriting/redoing most of the pieces. We're safe for now but we should learn how to leverage the new tech.
- andkenneth 3y ago
  This is the "your job won't be taken away by AI, it will be taken away by someone who knows how to leverage AI better than you"
  - ChatGTP 3y ago
    Don’t worry , the price of goods will be exponentially cheaper, so you won’t need a job /s
  - johnnyanmac 3y ago
    Probably, but isn't that how most if the technical fields go? Software in particular moves blazing fast and you need to adapt to the market quickly to be marketable.
- adventured 3y ago
  > We're safe for now Some are safe for several years (3-5), that's it. During that time it's going to wreck the bottom tiers of employees and progressively move up the ladder. GPT and the equivalent will be extraordinary at programming five years out. It will end up being a trivially easy task for AI in hindsight (15-20 years out), not a difficult task. Have you seen how far things like MidJourney, Dalle, Stable Diffusion have come in just a year or two? It's moving extremely fast. They've gone from generating stick figures to realistic photographs in two years.
  - ChatGTP 3y ago
    GPT and the equivalent will be extraordinary at programming five years out. Being exceptional at programming isn't hard. Being exceptional at listening to bullshit requirements for 5 hours a day is.
orbital-decay 3y ago
I don't know, 3D CGI has already been moving at the breakneck speed for the last three decades without any AI. Today's tools are qualitatively different (sculpting, simulation, auto-rigging etc etc etc).
- kranke155 3y ago
  3D CGI has gotten faster, but I haven’t seen any qualitative jump for quite some time. IMO the last time a major tech advance was visible was Davy Jones on the Pirates films. That was a fully photorealistic animated character that was plausible as a hero character in a major feature. That was a breakthrough. After that a lot of refinement and speeding up. This is different. I have some positivity about it, but it’s getting hard to keep track of everything that’s going on tbh. Every week it’s a new application and every few months it’s some quantum leap. Like others said, Midjourney and DallE are essentially photorealistic. It seems to me that the next step is generative AI creating better and better assets. And then of course you have video generation which is happening as well…
  - orbital-decay 3y ago
    Both DE3 and MJ are essentially toys for single random pictures, unusable in a professional setting. DALL-E in particular has really bad issues with quality, and while it follows the prompt well it also rewrites it so it's barely controllable. Midjourney is RLHF'd to death. What you want for asset creation is not photorealism, but style and concept transfer, multimodal controllability (text alone is terrible at expressing artistic intent), and tooling. And tooling isn't something that is developed quickly (although there were several rapid breakthroughs in the past, for example ZBrush). Most of the fancy demos you hear about sound good on paper, but don't really go anywhere. Academia is throwing shit at the wall to see what sticks, this is its purpose, especially when practice is running ahead of theory. It's similar to building airplanes before figuring out aerodynamics (which happened long ago): watching a heavier-than-air thing fly is amazing, until you realize it's not very practical in the current form, or might even kill its brave inventor who tried to fly it. If you look at the field closely, most of the progress in visual generative tooling happens in the open source community; people are trying to figure out what works in real use and what doesn't. Little is being done in big houses, at least publicly and for now, as they're more interested in a DC-3 than a Caproni Ca.60. The change is really incremental and gradual, similarly to the current mature state of 3D. Paradigms are different but they are both highly technical and depend on academic progress. Once it matures, it's going to become another skill-demanding field.

rizky05 3y ago

[dead]

nextworddev 3y ago

What do you ascertain the use case of this in your field? Does it seem high quality? (I have no context)

zavertnik 3y ago
I'm not a professional in VFX, but I work in television and do a lot of VFX/3D work on the side. The quality isn't amazing, but it looks like this could be the start of a Midjourney-tier VFX/3D LLM, which would be awesome. For me, this would help bridge the gap between having to use/find premade assets and building what I want. For context, building from scratch in a 3D pipeline requires you to wear a lot of different hats (modeling, materials, lighting, framing, animating, ect). It costs a lot of time to get to not only learn these hats but also use them together. The individual complexity of those skill sets makes it difficult to experiment and play around, which is how people learn with software. The shortcut is using premade assets or addons. For instance, being able to use the Source game assets in Source Filmmaker combined with SFM using a familiar game engine makes it easy to build an intuition with the workflow. This makes Source Filmmaker accessible and its why theres so much content out there made with it. So if you have gaps in your skillset or need to save time, you'll buy/use premade assets. This comes at a cost of control, but that's always been the tradeoff between building what you want and building with what you have. Just like GPT and DALL-E built a bridge between building what you want and building with what you have, a high fidelity GPT for the 3D pipeline would make that world so much more accessible and would bring the kind of attention NLE video editing got in the post-Youtube world. If I could describe in text and/or generate an image of a scene I want and have a GPT create the objects, model them, generate textures, and place them in the scene, I could suddenly just open blender, describe a scene, and just experimenting with shooting in it, as if I was playing in a sandbox FPS game. I'm not sure if MeshGPT is the ChatGPT of the 3D pipeline, but I do think this is kind of content generation is the conduit for the DALL-E of video that so many people are terrified and/or excited for.
- gavinray 3y ago
  On an unrelated note, could I ask your opinion? My wife is passionate about film/TV production and VFX. She's currently in school for this but is concerned about the difficulty of landing a job afterwards. Do you have any recommendations on breaking into the industry without work experience?
  - kranke155 3y ago
    As a producer? Huh. That’s such a great question. I think producer roles are a little bit less ultra competitive / scarce as they are actually jobs jobs where you have to use excel and planning and budgeting. Being a producer means being on the phone all the time, negotiating, haggling, finding solutions where they don’t seem to exist. Be it in TV, advertising or somewhere in the media space, the common rule is that producers are mostly actually terrible at their jobs, that’s my experience in London. So if she’s really good and really dedicated and learns the job of everyone on set, I’d say she has a shot. The real secret to being good in filmmaking is learning everyone else’s job. Toyota Production System says if you want to run a production line you have to know how it works. If she wants to do VFX production she could start doing her own test scenes, learning basics in nuke and Blender, even understanding the role of Houdini and how that works. If she does that - any company will be lucky to have her.
deleted 3y ago
[deleted]

bsenftner 3y ago

So you're probably familiar with the role of a Bidding Producer; imagine the difficulty they are facing: on one side they have filmmakers saying they just read so and so is now created by AI, while that is news to the bidding producer and their VFX/animation studio clients scrambling as everything they do is new again.

sheepscreek 3y ago

Perhaps one way to look at this could be auto-scaffolding. The typical modelling and CAD tools might include this feature to get you up and running faster. Another massive benefit is composability. If the model can generate a cup and a table, it also knows how to generate a cup on a table. Think of all the complex gears and machine parts this could generate in the blink of an eye, while being relevant to the project - rotated and positioned exact where you want it. Very similar to how GitHub Copilot works.

worldsayshi 3y ago

I don't see that LLM's have come that much further in 3D animation than programming in this regard: It can spit out bits and pieces that looks okay in isolation but a human need to solve the puzzle. And often solving the puzzle means rewriting/redoing most of the pieces. We're safe for now but we should learn how to leverage the new tech.

andkenneth 3y ago
This is the "your job won't be taken away by AI, it will be taken away by someone who knows how to leverage AI better than you"
- ChatGTP 3y ago
  Don’t worry , the price of goods will be exponentially cheaper, so you won’t need a job /s
- johnnyanmac 3y ago
  Probably, but isn't that how most if the technical fields go? Software in particular moves blazing fast and you need to adapt to the market quickly to be marketable.
adventured 3y ago
> We're safe for now Some are safe for several years (3-5), that's it. During that time it's going to wreck the bottom tiers of employees and progressively move up the ladder. GPT and the equivalent will be extraordinary at programming five years out. It will end up being a trivially easy task for AI in hindsight (15-20 years out), not a difficult task. Have you seen how far things like MidJourney, Dalle, Stable Diffusion have come in just a year or two? It's moving extremely fast. They've gone from generating stick figures to realistic photographs in two years.
- ChatGTP 3y ago
  GPT and the equivalent will be extraordinary at programming five years out. Being exceptional at programming isn't hard. Being exceptional at listening to bullshit requirements for 5 hours a day is.

orbital-decay 3y ago

I don't know, 3D CGI has already been moving at the breakneck speed for the last three decades without any AI. Today's tools are qualitatively different (sculpting, simulation, auto-rigging etc etc etc).

kranke155 3y ago
3D CGI has gotten faster, but I haven’t seen any qualitative jump for quite some time. IMO the last time a major tech advance was visible was Davy Jones on the Pirates films. That was a fully photorealistic animated character that was plausible as a hero character in a major feature. That was a breakthrough. After that a lot of refinement and speeding up. This is different. I have some positivity about it, but it’s getting hard to keep track of everything that’s going on tbh. Every week it’s a new application and every few months it’s some quantum leap. Like others said, Midjourney and DallE are essentially photorealistic. It seems to me that the next step is generative AI creating better and better assets. And then of course you have video generation which is happening as well…
- orbital-decay 3y ago
  Both DE3 and MJ are essentially toys for single random pictures, unusable in a professional setting. DALL-E in particular has really bad issues with quality, and while it follows the prompt well it also rewrites it so it's barely controllable. Midjourney is RLHF'd to death. What you want for asset creation is not photorealism, but style and concept transfer, multimodal controllability (text alone is terrible at expressing artistic intent), and tooling. And tooling isn't something that is developed quickly (although there were several rapid breakthroughs in the past, for example ZBrush). Most of the fancy demos you hear about sound good on paper, but don't really go anywhere. Academia is throwing shit at the wall to see what sticks, this is its purpose, especially when practice is running ahead of theory. It's similar to building airplanes before figuring out aerodynamics (which happened long ago): watching a heavier-than-air thing fly is amazing, until you realize it's not very practical in the current form, or might even kill its brave inventor who tried to fly it. If you look at the field closely, most of the progress in visual generative tooling happens in the open source community; people are trying to figure out what works in real use and what doesn't. Little is being done in big houses, at least publicly and for now, as they're more interested in a DC-3 than a Caproni Ca.60. The change is really incremental and gradual, similarly to the current mature state of 3D. Paradigms are different but they are both highly technical and depend on academic progress. Once it matures, it's going to become another skill-demanding field.