6 ms·

Ask HN: How is it to work in Visual Effects industry as a developer?

36 points by aprdm 11y ago

Anyone has experience on it? Lots of automation, scalability of the nodes, render farms... Looks like very interesting work.

malux85 11y ago
I have some experience here.. I worked at a studio in London, I was basically a data wrangler, writing python tools to automate everything. The render farm consisted of 250 nodes, each had 48 cores and 256GB of RAM. It was connected to a large storage array, total storage was approx 5PB. In the evenings the desktop machines that the modellers worked on were turned into render nodes. These machines were given lighter scenes to render since we wanted them to complete overnight. The longest frame I have seen has taken a 48 core node 60 hours to render a single frame. What else would you like to know?
- iamcreasy 11y ago
  How do they make sure that they don't have to redo the frame again?
  - malux85 11y ago
    The scheduling software (Qube) is responsible for retrying failed jobs. A frame render can fail for many reasons (e.g. Incorrectly packaged assets, missing textures, out of memory, bad lighting info, missing plugins, random SIGHUPs - rarer) - so if something screws up then Qube will retry N times if it can find available hardware. From the artists point of view, they don't render a frame once, they continuously render and progressively increase the level of detail until they're happy with the scene .. this means the render farm is almost always busy.
- ivraatiems 11y ago
  In what format does a rendered image come out? How large is it? How is editing done?
  - malux85 11y ago
    I worked in VFX - so we were delivered the edited scenes and the artists would render on top of them ... theres very little post-processing done, what comes off the farm is fit for the client. Although the farm schedules individual frames, they are combined into scenes (short clips from 2 - 30 seconds) and the artists get the scenes back. The scene file format is proprietary (and viewed with proprietary software), but I had a sneaky look inside and it seems to be losslessly compressed TGA frames. Scenes are usually short 2-6 seconds, and filesize would be 2-3 GB
- aprdm 11y ago
  That looks very interesting, did you leave the industry (as you said in the past) and if so, why? Do you think that someone having experience with backend / web development with Python would do well in this kind of job? Thanks
- aprdm 11y ago
  Also another question, to do exactly the task you've done, where did you learn about it? Any specific courses or books? Thanks
KaiserPro 11y ago
Depends on where you are, what you want to do. There are two main bits of VFX developer work: tools and RnD. Tools developers are there to allow artists to find, convert, import and send assets to a render queue. Most of this is written in python, as thats the embedded language of choice for most VFX packages (maya, nuke, $other) Then there is RnD: this is most of the stuff you see at sigraph. We need to implement a new type of water caustic that doesn't suck, Model snow more realistically, or trying to make hair systems not an utter pain in the arse to use. If you want to be in the tools department you'll need python, and some experience with maya/nuke/renderman/arnold. If you want to be doing RnD you'll need to figure out what you want to play with and either apply to a company that trying to build their own renderer (MPC) or join a company that is already building one (disney, pixar, the foundry, autodesk) Also, beware that its an industry thats in the second part of a massive globalisation heave. Jobs are clustered in vancouver & london. However there are large changes that mean lots of work is starting to gravitate to china and india. At the moment its low end stuff like paint and roto, but thats where the next generation comes from.
- KaiserPro 11y ago
  I also forgot to mention; at a well established shop everything is automated & integrated. most workstations will have a network home partition, which means that your home directory is everwhere you log in. Its easier and miles faster to write something out to an NFS share than it is to use HTTP/database Ten gig switches get saturated without trying too hard (Place I worked pulled a constant 15 gigabytes a second from storage. bursting to 35. thats bytes, not bits) Render farms have been using cgroups for many years. you'll start to worry about scheduling rate of tasks. The only things that are virtualised are things like the intranet and the wiki. everything else is on real steel rendernodes are big, there are loads of them, but never enough. They are treated like cattle. There are large data pipes. however if you deal with marvel, you'll not have internet access at your desk, it'll be through some horrid remote desktop system. If you're lucky you might get one way copy and paste. Fluid sims have caches that are in the order of 10s of terabytes
  - jkrp 11y ago
deleted 11y ago
[deleted]

36 points by aprdm 11y ago

Anyone has experience on it? Lots of automation, scalability of the nodes, render farms... Looks like very interesting work.

malux85 11y ago

I have some experience here.. I worked at a studio in London, I was basically a data wrangler, writing python tools to automate everything. The render farm consisted of 250 nodes, each had 48 cores and 256GB of RAM. It was connected to a large storage array, total storage was approx 5PB. In the evenings the desktop machines that the modellers worked on were turned into render nodes. These machines were given lighter scenes to render since we wanted them to complete overnight. The longest frame I have seen has taken a 48 core node 60 hours to render a single frame. What else would you like to know?

iamcreasy 11y ago
How do they make sure that they don't have to redo the frame again?
- malux85 11y ago
  The scheduling software (Qube) is responsible for retrying failed jobs. A frame render can fail for many reasons (e.g. Incorrectly packaged assets, missing textures, out of memory, bad lighting info, missing plugins, random SIGHUPs - rarer) - so if something screws up then Qube will retry N times if it can find available hardware. From the artists point of view, they don't render a frame once, they continuously render and progressively increase the level of detail until they're happy with the scene .. this means the render farm is almost always busy.
ivraatiems 11y ago
In what format does a rendered image come out? How large is it? How is editing done?
- malux85 11y ago
  I worked in VFX - so we were delivered the edited scenes and the artists would render on top of them ... theres very little post-processing done, what comes off the farm is fit for the client. Although the farm schedules individual frames, they are combined into scenes (short clips from 2 - 30 seconds) and the artists get the scenes back. The scene file format is proprietary (and viewed with proprietary software), but I had a sneaky look inside and it seems to be losslessly compressed TGA frames. Scenes are usually short 2-6 seconds, and filesize would be 2-3 GB
aprdm 11y ago
That looks very interesting, did you leave the industry (as you said in the past) and if so, why? Do you think that someone having experience with backend / web development with Python would do well in this kind of job? Thanks
aprdm 11y ago
Also another question, to do exactly the task you've done, where did you learn about it? Any specific courses or books? Thanks

KaiserPro 11y ago

Depends on where you are, what you want to do. There are two main bits of VFX developer work: tools and RnD. Tools developers are there to allow artists to find, convert, import and send assets to a render queue. Most of this is written in python, as thats the embedded language of choice for most VFX packages (maya, nuke, $other) Then there is RnD: this is most of the stuff you see at sigraph. We need to implement a new type of water caustic that doesn't suck, Model snow more realistically, or trying to make hair systems not an utter pain in the arse to use. If you want to be in the tools department you'll need python, and some experience with maya/nuke/renderman/arnold. If you want to be doing RnD you'll need to figure out what you want to play with and either apply to a company that trying to build their own renderer (MPC) or join a company that is already building one (disney, pixar, the foundry, autodesk) Also, beware that its an industry thats in the second part of a massive globalisation heave. Jobs are clustered in vancouver & london. However there are large changes that mean lots of work is starting to gravitate to china and india. At the moment its low end stuff like paint and roto, but thats where the next generation comes from.

KaiserPro 11y ago
I also forgot to mention; at a well established shop everything is automated & integrated. most workstations will have a network home partition, which means that your home directory is everwhere you log in. Its easier and miles faster to write something out to an NFS share than it is to use HTTP/database Ten gig switches get saturated without trying too hard (Place I worked pulled a constant 15 gigabytes a second from storage. bursting to 35. thats bytes, not bits) Render farms have been using cgroups for many years. you'll start to worry about scheduling rate of tasks. The only things that are virtualised are things like the intranet and the wiki. everything else is on real steel rendernodes are big, there are loads of them, but never enough. They are treated like cattle. There are large data pipes. however if you deal with marvel, you'll not have internet access at your desk, it'll be through some horrid remote desktop system. If you're lucky you might get one way copy and paste. Fluid sims have caches that are in the order of 10s of terabytes
- jkrp 11y ago

deleted 11y ago

[deleted]