12 ms·

DreamWorks Animation to release MoonRay as open source

440 points by mroche 4y ago

miohtama 4y ago
Would be interesting to know what business considerations lead to the decision to open source MoonRay.
- berkut 4y ago
  Yeah. They've open sourced OpenVDB and other smaller things before, and have contributed things to Embree and OpenSubDiv, but those were libraries/storage formats, not entire production-capable renderers. At the same time however, does their renderer give them that many advantages? As someone who works on a (sort of) competing proprietary renderer, it's a lot of work and effort to do it and support it, and maybe they want to build a community around that from smaller studios and compete with Renderman a bit for mindshare?
  - wlesieutre 4y ago
    If they can get students or freelance animators to learn it, that's a potential hiring pool with a lot less onboarding
  - dimatura 4y ago
    OpenVDB is a really cool tool and a nice C++ codebase. During my PhD I adapted it to manage volumetric data extracted from LiDAR measurements and found it to be much more efficient than more popular alternatives in the robotics world, such as Octomap (which is also a great tool in its own right!).
- KittyCatCuddler 4y ago
  It kinda seems like they might be in cahoots with Microsoft to compete with AWS Nimble. https://aws.amazon.com/nimble-studio/ https://aws.amazon.com/nimble-studio/
  - mchusma 4y ago
    This is not really like Nimble, it's more like Picard renderman
  - berkut 4y ago
    Interesting, seeing as at least on Trolls they were using AWS for some of their compute for rendering. Some Dreamworks employees on the tech side left for AWS as well.
  - khazhoux 4y ago
    Just FYI, Nimble was former Dreamworks animators and engineers.
  - bogwog 4y ago
    This is released under Apache 2.0 though, so Amazon could just do what they always do and repackage/sell it under a different name as an AWS product.
- erichocean 4y ago
  Even small companies can create competitive high-end renderers today so it's not much of a differentiator. Filmmaking is about human talent mostly, not technology. (Same thing happened in live-action filmmaking, anyone can afford the equipment today.) But there's a huge advantage to being the first high-end renderer to go open source, so lots of small studios will adopt it if it's as solid as it looks. It'll become available in Blender. Maintenance will then get extended to the community, including (most likely) support for alternate shading schemes (MaterialX/Lama has decent momentum right now, converging with Sony's OpenShadingLanguage). Apache 2.0 is an ideal license for this kind of project.
- ArtWomb 4y ago
  If we're speculating, Pixar is also announcing it's Renderman 25 XPU cloud native render tech. If gpu cluster & virtualization continues to explode and transform the effects industry. I imagine all the major studios would seek ancillary revenue, farming out their cloud investments during idle hours, to third parties in gov research and academics to use for large scale data viz ;)
  - erichocean 4y ago
    > during idle hours Studios don't really have idle hours on their render farms.
    - samstave 4y ago
      This was 2004. The concepts were still being figured out WRT to making a huge campus for one of the most creative companies... EDIT: I thought you were replying to my comment about the Lucas Presidio Campus
  - samstave 4y ago
    An interesting tidbit of history: When I was on the design team for the Lucas Presidio Campus, we were collapsing all the various divisions to the new campus, aside from Skywalker... I was the LV/Datacenter/network designer for the project. One of the early design requirements was to 1) have fiber to the desktop and 2) have every machine on an interconnect matrix which brought the edge desktop connections as close to the core as possible such that every workstation would become a part of the ender cluster when idle/at night/on-demand etc... This didnt happen but it was a goal back in ~2004 -- I recall them telling us when Jobs came to visit and they had the "Death Star" <-- render cluster they were working on... Lucas apparently saw no future and sold it to Jobs for ~$10mm?? and thats where Pixar got it start. That was an amazing fun project to work on. --- Oh - and during the design sessions, Cisco borked the design review for their (joke of a response) to the RFP, the then CIO for LucasArts in this design meeting with ILM, Lucas, Cisco others -- he says to the entire team in dead seriousness "When can I have power-over-fiber to the desktop" with a look like "Ha! GotchA! And everyone just blinked..
  - 4y ago
- colechristensen 4y ago
  >We expect to see the code base grow stronger with community involvement as DreamWorks continues to demonstrate our commitment to open source Translation: we think making this open source will bring in significant outside contribution so we can get a better renderer with less investment ourselves while not having to pay external providers Open source is a good strategy for the runners up trying to compete, you get the good will from giving something away in exchange for other people improving your product for free.
  - ziddoap 4y ago
    Why are you framing this in such a negative light? They open something up, let people use it, and hope that some people might contribute back. Cool, sounds great to me. They aren't forcing me to contribute back to the project. Seems pretty win-win.
    - colechristensen 4y ago
      It’s clearly business, not self sacrifice. Your interpretation of that as negativity is a reflection of your own judgement not mine.
  - erichocean 4y ago
    > runners up trying to compete MoonRay likely has the best volume rendering in the world, certainly that's open source. Only Weta's internal renderer is in the same league.
    - berkut 4y ago
      Not necessarily: it would depend on what Dreamworks' requirements have been up to now as to the current state: obviously, they use volumetrics, but almost all their stuff is CG and not necessarily photo real, being more artistic-driven. That fact might have allowed them to take short-cuts. How do you know what Manuka's capable of? :)
      - mkaic 4y ago
- desindol 4y ago
  Students get used to your pipeline before you hire them. The technology is nothing groundbreaking but the best that is available for free. Most students will take that opportunity.
  - bombcar 4y ago
    This is huge and people underestimate how valuable it can be for a company. DreamWorks Animation's product is not MoonRay, it is movies and they're commodizing their complement. If every animation studio started using MoonRay because of this, they'd have won.
    - pwdisswordfish9 4y ago
      Good point in your second sentence, but hard see where you're coming from in the last one. Surely the win for DreamWorks is the competitive advantage of having aspiring creators be familiar with their pipeline. If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. (The only way this isn't true is if you subscribe to the belief that more MoonRay use → more MoonRay experts → more DreamWorks movies → more profit. That's probably not sound. This probably won't impact how many movies they put out, just how good they look (after benefiting from open source work) and how easy it is to hire for the movies that they put out.)
      - kanzure 4y ago
        > If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. Other studios could also contribute improvements, which the original studio could benefit from. That's one of my motivations for open-source: I'm selfish and I enjoy the benefits from the efforts of others making my software better, should they find it useful.
        bombcar 4y ago
        And it would mean that they're all on a level playing field - everyone would be using MoonRay so there wouldn't be an obvious technological advantage. (Note that this could mean that if you thought your software was putting you at an advantage, you might not want to open-source it, but you could also believe that even if it is better, your competitors won't / can't switch to it).
- thebeardisred 4y ago
  DreamWorks Animation (aka DWA) is a huge advocate of open source. Over the years they've bankrolled a lot of development around Linux on the desktop and have one of the most astonishing NFS infrastructures I've ever directly layed hands on. They've talked about their love of open source over the years, especially at Red Hat summit. (I spent quite a bit of time working with them and their former CTO is currently a teammate of mine).
  - albatross13 4y ago
    Can you expound a bit on the NFS infrastructure? I'm curious. If you can't, legally, nbd.
    - newman314 4y ago
      I was curious, did a quick search and found this OLD deck. Would be interesting to see how it has evolved since. https://www.usenix.org/legacy/event/lisa10/tech/slides/kamath.pdf https://www.usenix.org/legacy/event/lisa10/tech/slides/kamat...
    - thebeardisred 4y ago
      Fortunately, Mike (Cutler) & Sean (Kamath) documented many things in the USENIX presentation cited by newman314 that I'm still under NDA about. I worked with both of them and they're extremely knowledgeable folks. Now, switching gears to talk about that presentation. ;) I'd call out a few things. 1) Note on slide 17 the call out to _multiple sites_. Yup. You got it. There is a single view of the file store. At a practical level to the end user the sync time between sites is trivial to non-existent. 2) automount, automount, automount (Slide 25). The automount configuration is stored in LDAP providing a convenient way to correlate and sync data which is already integrated with the name service switch/NSS (man 5 nss). 3) Caching (Slides 22-24). This is where a lot of the magic happens. It's this cache tier which _presents_ as if it's a single file store. One conceit I will make (which is glossed over a bit) is that the use of DNS and suffix searching is critical in a situation like this. It allows each site to use configuration management to push out relevant config files (e.g. /etc/resolv.conf) which can then present a search order like "site1.us.example.com us.example.com example.com". Thus, when you do a query for the unqualified name "ldap" (or even better "IN SRV _ldap._tcp") NSS will hand that off to the resolver which will dutifully check for the following (in order) until it gets a positive match or fails: ldap.site1.us.example.com, ldap.us.example.com ldap.example.com. This allows for a DNS based fall back as well as the added benefit of always hitting the local system (instead of a remote site). Also, make sure you check out the stats on Slide 28.... 4 PB... in 2008... before they had to render a movie for each eye... (3D animation!)
- munificent 4y ago
  There is a lot of staffing churn in the film world and many films are built by farming out work to dozens of separate VFX studios. I speculate that having this open source increases its popularity, which makes more likely that any given potential hire or VFX company will have experience with it, which makes it easier for DreamWorks to ramp up new hires or contract out to third party companies. When I was at EA, we switched from a proprietary UI tool (which was quite well suited for consoles) to Flash (which wasn't) entirely because it eased staffing problems even though the technology itself was a worse fit. The best tool is often the one that people you can hire already know.
  - egypturnash 4y ago
    god, Flash wasn't suited for anything and yet somehow it got used for everything until Apple managed to kill it by keeping it the hell off the iPhone. It was kind of okay for so many things and actually great at none of them. It might even still be in use at Cartoon Network, it's been a while since I last heard from my former co-worker in the dialup-delivered Flash cartoon mines who ended up as the animation director for Foster's and Titans but he sure was still using an old version of Flash/Animate long after the browser plugin was retired. There's probably a lot of lessons in Flash's status as a tool that did a ton of different things half-assedly.
    - munificent 4y ago
      It was like the JavaScript of UX and 2D games for a while. It was trash as a technology, but it was a technology that millions of UI designers and indie game developers cut their teeth on and already knew deeply, so it stuck around much longer than it should have. Like grown-ups trying to ride tricycles to work. I remember someone at EA talked to Adobe about getting access to the Flash (.fla) file format so that our UI tools could read it directly to export into the in game format instead of reading the already compiled and compressed .swf format. And they were like, "Uh... you really don't want to do that. It's like a memory dump of the entire application state. It's a giant mess. We're really sorry." By all accounts, it seemed like the Flash codebase was a dumpster fire. But so many amazing little games and UIs were built using it.
manchoz 4y ago
The name actually reminds me of the awesome RenderMan-compliant BMRT https://en.wikipedia.org/wiki/Blue_Moon_Rendering_Tools https://en.wikipedia.org/wiki/Blue_Moon_Rendering_Tools
- codetrotter 4y ago
  For me the name reminded me of Mental Ray. Dunno if there is any connection. https://en.wikipedia.org/wiki/Mental_Ray https://en.wikipedia.org/wiki/Mental_Ray
- boulos 4y ago
  Eh, it traces rays and the moon (and "Moon Boy") are DreamWorks.
pwdisswordfish9 4y ago
- berkut 4y ago
  Where's the acronym in the title?
  - pwdisswordfish9 4y ago
    - berkut 4y ago
      Fair enough. But I suspect if you got your wish there would be a lot less posts...
- erulabs 4y ago
  I suppose they could have said "Monte Carlo Raytracer" but would that have really helped anyone?
  - pwdisswordfish9 4y ago
    Is that even a serious question? Because the answer is (a fairly obvious) "Yes."
    - lynndotpy 4y ago
      This is unreasonably snarky and rude, but I'll bite because I'm curious: Why is this knowledge so useful it should be explained in the title?
      - pwdisswordfish9 4y ago
        Just curious: why is it that not one, but two people have responded in this thread as if it has anything to do with the title?
nixcraft 4y ago
From download page https://openmoonray.org/download https://openmoonray.org/download : "We are currently preparing the packages for public release, and MoonRay will be available for download from Github soon."
mkaic 4y ago
This looks absolutely fantastic. If you're interested in the technical details currently available, check out the about page[0] on the official MoonRay site, or this presentation from SIGGRAPH 2017[1]. I'm particularly excited for the HeatMap render pass, seems like a really practical way to identify what's bogging your scene down. Not to mention the gorgeous volumetric rendering this engine is capable of (see How To Train Your Dragon: The Hidden Realm[2]). Really impressed that Dreamworks is making this move. They've made some of my favorite films of all time (the Dragons franchise genuinely changed my life and is a major part of why I love filmmaking, film scores, and 3D rendering) and I'm glad to see they're doing this. I can't wait to try MoonRay in Blender once a community integration exists! [0]https://openmoonray.org/about https://openmoonray.org/about [1]https://jo.dreggn.org/path-tracing-in-production/2017/MoonrayV3.pdf https://jo.dreggn.org/path-tracing-in-production/2017/Moonra... [2]https://www.awn.com/animationworld/how-moonray-became-hidden-superpower-behind-how-train-your-dragon-hidden-world https://www.awn.com/animationworld/how-moonray-became-hidden... EDIT: I've also come across this excellent blog post written by an engineer who worked on MoonRay for 4 years: http://rendering-memo.blogspot.com/2019/ http://rendering-memo.blogspot.com/2019/ And this paper recommended by erichocean in their reply below: http://www.tabellion.org/et/paper17/MoonRay.pdf http://www.tabellion.org/et/paper17/MoonRay.pdf
- erichocean 4y ago
  http://www.tabellion.org/et/paper17/MoonRay.pdf http://www.tabellion.org/et/paper17/MoonRay.pdf is also a good read.
  - mkaic 4y ago
    Thanks, I'll have to check this out! And to think I thought I was going to be productive at work today before seeing this announcement...
- echelon 4y ago
  > Really impressed that Dreamworks is making this move. It's bone-headed. They could have partnered with a cloud vendor and maintained control and a large percent of the revenue. Instead, Amazon can now turn this into an AWS offering, drive mindshare to the AWS offering, and then bleed DreamWorks of whatever competitive advantage this offered them. AWS makes money, DreamWorks loses talent and leadership that moves over to work at AWS. Open sourcing big software with an enterprise market and giving it a permissive license is stupid, because you can't compete. Look at pretty much every permissively licensed database vendor. Elastic's recent fight comes to mind. In ten years, when this stuff becomes easy enough for small studios to run (something my startup is working on), the need for big studios like DreamWorks to pursue projects funded by institutional capital evaporates. Everyone will be making real time 3D animation. If you look carefully, you can see the genesis of that trend now. If I were in a leadership position at DreamWorks, I would be staffing up how to make these offerings broadly available. How to make tools that put more people in the director seat. That sounds like the mission of a company named "Dream Works". Let more people dream. I guess this does it, just not in a way that they'll be using your company to do so. Big disruptions are coming to the creative fields. This kind of move is like giving away the keys to Amazon for free. (I mean, they are buying Hollywood studios left and right. If you don't see it, you're asleep at the wheel.)
punnerud 4y ago
What can be the real reason for Open Sourcing? That they have a hard time competing with the new machine learning based rendering?
- erichocean 4y ago
  Every studio has been open sourcing high end VFX/CG software libraries for over a decade. This is just a continuation of that.
- deleted 4y ago
  [deleted]
- anaganisk 4y ago
  We are in a world, where left pad was opensource, question must always be why is it closed source. I mean not to say closed is bad, but a question is good.
ramesh31 4y ago
Is this akin to RenderMan? And is this something useful for hobbyists, or is it just something that would run in a render farm?
- bogwog 4y ago
  A hobbyist film maker probably won't use this directly, but it will likely find its way into software that they do use at some point.
- VikingCoder 4y ago
  Yes, roughly akin to RenderMan. Yes, some hobbyists will use this.
pitaj 4y ago
I'm not very well acquainted with this kind of thing. Is this something that could be used with other open-source projects like Blender?
- knolan 4y ago
  Yes, you can use other renderers with Blender. RenderMan for example.
- dry_soup 4y ago
  It will require the development of an add-on for Blender. It might even already exist.
robalni 4y ago
It's good to see companies freeing important software like this. That shows other companies and everyone else that you can do that without too much risk and that it might even give you benefits. I think many companies are afraid of doing it and I think this is how we can show them that there is no problem with doing it (and if there is a problem, we will learn that; we learn something either way). Since free software is so important to many people and one of the biggest questions about how the digital society should work, it's crucial that we explore the possibilities of releasing free software. I'm tired of the current situation where on one side we have a ton of software and on the other side people who can't use that software because of how unethical or impractical it is. We need to find something that works for everyone and this is a step in that direction.
- echelon 4y ago
  Amazon is going to take this and make it SaaS. Then the ecosystem moves to Amazon instead of DreamWorks.
  - zucker42 4y ago
    Do people pay to use DreamWorks renderer? I would think they pay DreamWorks to animate stuff. That hasn't changed.
    - echelon 4y ago
      Individual creators are now building scenes in Blender and Unreal Engine. The floodgates are about to open for individual creators doing animation. An easy to use render farm is a good startup idea.
      - SEJeff 4y ago
        It's really just some shoestring and bubblegum ontop of amazon / google gpu instances. As commodity AND niche as a renderfarm is, it seems like if it was to run, it would have to be via a foundation like ummm the blender foundation.
johnmertic 4y ago
We added MoonRay today to the ASWF Landscape, which is a great tool for seeing all the open source in the visual and special effects industry. Check it out at https://l.aswf.io https://l.aswf.io
- erichocean 4y ago
  That's a very helpful page, thank you for maintaining it!
jarrell_mark 4y ago
Blender integration using Hydra. "By adapting Hydra as a render add-on to Blender, any renderer that supports Hydra [ie MoonRay] can be connected to Blender easily, by plugging-in to the Hydra add-on." - https://gpuopen.com/learn/amd-usd-hydra-blender/ https://gpuopen.com/learn/amd-usd-hydra-blender/

440 points by mroche 4y ago

miohtama 4y ago

Would be interesting to know what business considerations lead to the decision to open source MoonRay.

berkut 4y ago
Yeah. They've open sourced OpenVDB and other smaller things before, and have contributed things to Embree and OpenSubDiv, but those were libraries/storage formats, not entire production-capable renderers. At the same time however, does their renderer give them that many advantages? As someone who works on a (sort of) competing proprietary renderer, it's a lot of work and effort to do it and support it, and maybe they want to build a community around that from smaller studios and compete with Renderman a bit for mindshare?
- wlesieutre 4y ago
  If they can get students or freelance animators to learn it, that's a potential hiring pool with a lot less onboarding
- dimatura 4y ago
  OpenVDB is a really cool tool and a nice C++ codebase. During my PhD I adapted it to manage volumetric data extracted from LiDAR measurements and found it to be much more efficient than more popular alternatives in the robotics world, such as Octomap (which is also a great tool in its own right!).
KittyCatCuddler 4y ago
It kinda seems like they might be in cahoots with Microsoft to compete with AWS Nimble. https://aws.amazon.com/nimble-studio/ https://aws.amazon.com/nimble-studio/
- mchusma 4y ago
  This is not really like Nimble, it's more like Picard renderman
- berkut 4y ago
  Interesting, seeing as at least on Trolls they were using AWS for some of their compute for rendering. Some Dreamworks employees on the tech side left for AWS as well.
- khazhoux 4y ago
  Just FYI, Nimble was former Dreamworks animators and engineers.
- bogwog 4y ago
  This is released under Apache 2.0 though, so Amazon could just do what they always do and repackage/sell it under a different name as an AWS product.
erichocean 4y ago
Even small companies can create competitive high-end renderers today so it's not much of a differentiator. Filmmaking is about human talent mostly, not technology. (Same thing happened in live-action filmmaking, anyone can afford the equipment today.) But there's a huge advantage to being the first high-end renderer to go open source, so lots of small studios will adopt it if it's as solid as it looks. It'll become available in Blender. Maintenance will then get extended to the community, including (most likely) support for alternate shading schemes (MaterialX/Lama has decent momentum right now, converging with Sony's OpenShadingLanguage). Apache 2.0 is an ideal license for this kind of project.
ArtWomb 4y ago
If we're speculating, Pixar is also announcing it's Renderman 25 XPU cloud native render tech. If gpu cluster & virtualization continues to explode and transform the effects industry. I imagine all the major studios would seek ancillary revenue, farming out their cloud investments during idle hours, to third parties in gov research and academics to use for large scale data viz ;)
- erichocean 4y ago
  > during idle hours Studios don't really have idle hours on their render farms.
  - samstave 4y ago
    This was 2004. The concepts were still being figured out WRT to making a huge campus for one of the most creative companies... EDIT: I thought you were replying to my comment about the Lucas Presidio Campus
- samstave 4y ago
  An interesting tidbit of history: When I was on the design team for the Lucas Presidio Campus, we were collapsing all the various divisions to the new campus, aside from Skywalker... I was the LV/Datacenter/network designer for the project. One of the early design requirements was to 1) have fiber to the desktop and 2) have every machine on an interconnect matrix which brought the edge desktop connections as close to the core as possible such that every workstation would become a part of the ender cluster when idle/at night/on-demand etc... This didnt happen but it was a goal back in ~2004 -- I recall them telling us when Jobs came to visit and they had the "Death Star" <-- render cluster they were working on... Lucas apparently saw no future and sold it to Jobs for ~$10mm?? and thats where Pixar got it start. That was an amazing fun project to work on. --- Oh - and during the design sessions, Cisco borked the design review for their (joke of a response) to the RFP, the then CIO for LucasArts in this design meeting with ILM, Lucas, Cisco others -- he says to the entire team in dead seriousness "When can I have power-over-fiber to the desktop" with a look like "Ha! GotchA! And everyone just blinked..
- 4y ago
colechristensen 4y ago
>We expect to see the code base grow stronger with community involvement as DreamWorks continues to demonstrate our commitment to open source Translation: we think making this open source will bring in significant outside contribution so we can get a better renderer with less investment ourselves while not having to pay external providers Open source is a good strategy for the runners up trying to compete, you get the good will from giving something away in exchange for other people improving your product for free.
- ziddoap 4y ago
  Why are you framing this in such a negative light? They open something up, let people use it, and hope that some people might contribute back. Cool, sounds great to me. They aren't forcing me to contribute back to the project. Seems pretty win-win.
  - colechristensen 4y ago
    It’s clearly business, not self sacrifice. Your interpretation of that as negativity is a reflection of your own judgement not mine.
- erichocean 4y ago
  > runners up trying to compete MoonRay likely has the best volume rendering in the world, certainly that's open source. Only Weta's internal renderer is in the same league.
  - berkut 4y ago
    Not necessarily: it would depend on what Dreamworks' requirements have been up to now as to the current state: obviously, they use volumetrics, but almost all their stuff is CG and not necessarily photo real, being more artistic-driven. That fact might have allowed them to take short-cuts. How do you know what Manuka's capable of? :)
    - mkaic 4y ago
desindol 4y ago
Students get used to your pipeline before you hire them. The technology is nothing groundbreaking but the best that is available for free. Most students will take that opportunity.
- bombcar 4y ago
  This is huge and people underestimate how valuable it can be for a company. DreamWorks Animation's product is not MoonRay, it is movies and they're commodizing their complement. If every animation studio started using MoonRay because of this, they'd have won.
  - pwdisswordfish9 4y ago
    Good point in your second sentence, but hard see where you're coming from in the last one. Surely the win for DreamWorks is the competitive advantage of having aspiring creators be familiar with their pipeline. If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. (The only way this isn't true is if you subscribe to the belief that more MoonRay use → more MoonRay experts → more DreamWorks movies → more profit. That's probably not sound. This probably won't impact how many movies they put out, just how good they look (after benefiting from open source work) and how easy it is to hire for the movies that they put out.)
    - kanzure 4y ago
      > If all animation studios use MoonRay, then that advantage gets erased, and the ultimate impact is null. Other studios could also contribute improvements, which the original studio could benefit from. That's one of my motivations for open-source: I'm selfish and I enjoy the benefits from the efforts of others making my software better, should they find it useful.
      - bombcar 4y ago
        And it would mean that they're all on a level playing field - everyone would be using MoonRay so there wouldn't be an obvious technological advantage. (Note that this could mean that if you thought your software was putting you at an advantage, you might not want to open-source it, but you could also believe that even if it is better, your competitors won't / can't switch to it).
thebeardisred 4y ago
DreamWorks Animation (aka DWA) is a huge advocate of open source. Over the years they've bankrolled a lot of development around Linux on the desktop and have one of the most astonishing NFS infrastructures I've ever directly layed hands on. They've talked about their love of open source over the years, especially at Red Hat summit. (I spent quite a bit of time working with them and their former CTO is currently a teammate of mine).
- albatross13 4y ago
  Can you expound a bit on the NFS infrastructure? I'm curious. If you can't, legally, nbd.
  - newman314 4y ago
    I was curious, did a quick search and found this OLD deck. Would be interesting to see how it has evolved since. https://www.usenix.org/legacy/event/lisa10/tech/slides/kamath.pdf https://www.usenix.org/legacy/event/lisa10/tech/slides/kamat...
  - thebeardisred 4y ago
    Fortunately, Mike (Cutler) & Sean (Kamath) documented many things in the USENIX presentation cited by newman314 that I'm still under NDA about. I worked with both of them and they're extremely knowledgeable folks. Now, switching gears to talk about that presentation. ;) I'd call out a few things. 1) Note on slide 17 the call out to _multiple sites_. Yup. You got it. There is a single view of the file store. At a practical level to the end user the sync time between sites is trivial to non-existent. 2) automount, automount, automount (Slide 25). The automount configuration is stored in LDAP providing a convenient way to correlate and sync data which is already integrated with the name service switch/NSS (man 5 nss). 3) Caching (Slides 22-24). This is where a lot of the magic happens. It's this cache tier which _presents_ as if it's a single file store. One conceit I will make (which is glossed over a bit) is that the use of DNS and suffix searching is critical in a situation like this. It allows each site to use configuration management to push out relevant config files (e.g. /etc/resolv.conf) which can then present a search order like "site1.us.example.com us.example.com example.com". Thus, when you do a query for the unqualified name "ldap" (or even better "IN SRV _ldap._tcp") NSS will hand that off to the resolver which will dutifully check for the following (in order) until it gets a positive match or fails: ldap.site1.us.example.com, ldap.us.example.com ldap.example.com. This allows for a DNS based fall back as well as the added benefit of always hitting the local system (instead of a remote site). Also, make sure you check out the stats on Slide 28.... 4 PB... in 2008... before they had to render a movie for each eye... (3D animation!)
munificent 4y ago
There is a lot of staffing churn in the film world and many films are built by farming out work to dozens of separate VFX studios. I speculate that having this open source increases its popularity, which makes more likely that any given potential hire or VFX company will have experience with it, which makes it easier for DreamWorks to ramp up new hires or contract out to third party companies. When I was at EA, we switched from a proprietary UI tool (which was quite well suited for consoles) to Flash (which wasn't) entirely because it eased staffing problems even though the technology itself was a worse fit. The best tool is often the one that people you can hire already know.
- egypturnash 4y ago
  god, Flash wasn't suited for anything and yet somehow it got used for everything until Apple managed to kill it by keeping it the hell off the iPhone. It was kind of okay for so many things and actually great at none of them. It might even still be in use at Cartoon Network, it's been a while since I last heard from my former co-worker in the dialup-delivered Flash cartoon mines who ended up as the animation director for Foster's and Titans but he sure was still using an old version of Flash/Animate long after the browser plugin was retired. There's probably a lot of lessons in Flash's status as a tool that did a ton of different things half-assedly.
  - munificent 4y ago
    It was like the JavaScript of UX and 2D games for a while. It was trash as a technology, but it was a technology that millions of UI designers and indie game developers cut their teeth on and already knew deeply, so it stuck around much longer than it should have. Like grown-ups trying to ride tricycles to work. I remember someone at EA talked to Adobe about getting access to the Flash (.fla) file format so that our UI tools could read it directly to export into the in game format instead of reading the already compiled and compressed .swf format. And they were like, "Uh... you really don't want to do that. It's like a memory dump of the entire application state. It's a giant mess. We're really sorry." By all accounts, it seemed like the Flash codebase was a dumpster fire. But so many amazing little games and UIs were built using it.

manchoz 4y ago

The name actually reminds me of the awesome RenderMan-compliant BMRT https://en.wikipedia.org/wiki/Blue_Moon_Rendering_Tools https://en.wikipedia.org/wiki/Blue_Moon_Rendering_Tools

codetrotter 4y ago
For me the name reminded me of Mental Ray. Dunno if there is any connection. https://en.wikipedia.org/wiki/Mental_Ray https://en.wikipedia.org/wiki/Mental_Ray
boulos 4y ago
Eh, it traces rays and the moon (and "Moon Boy") are DreamWorks.

pwdisswordfish9 4y ago

berkut 4y ago
Where's the acronym in the title?
- pwdisswordfish9 4y ago
  - berkut 4y ago
    Fair enough. But I suspect if you got your wish there would be a lot less posts...
erulabs 4y ago
I suppose they could have said "Monte Carlo Raytracer" but would that have really helped anyone?
- pwdisswordfish9 4y ago
  Is that even a serious question? Because the answer is (a fairly obvious) "Yes."
  - lynndotpy 4y ago
    This is unreasonably snarky and rude, but I'll bite because I'm curious: Why is this knowledge so useful it should be explained in the title?
    - pwdisswordfish9 4y ago
      Just curious: why is it that not one, but two people have responded in this thread as if it has anything to do with the title?

nixcraft 4y ago

From download page https://openmoonray.org/download https://openmoonray.org/download : "We are currently preparing the packages for public release, and MoonRay will be available for download from Github soon."

mkaic 4y ago

This looks absolutely fantastic. If you're interested in the technical details currently available, check out the about page[0] on the official MoonRay site, or this presentation from SIGGRAPH 2017[1]. I'm particularly excited for the HeatMap render pass, seems like a really practical way to identify what's bogging your scene down. Not to mention the gorgeous volumetric rendering this engine is capable of (see How To Train Your Dragon: The Hidden Realm[2]). Really impressed that Dreamworks is making this move. They've made some of my favorite films of all time (the Dragons franchise genuinely changed my life and is a major part of why I love filmmaking, film scores, and 3D rendering) and I'm glad to see they're doing this. I can't wait to try MoonRay in Blender once a community integration exists! [0]https://openmoonray.org/about https://openmoonray.org/about [1]https://jo.dreggn.org/path-tracing-in-production/2017/MoonrayV3.pdf https://jo.dreggn.org/path-tracing-in-production/2017/Moonra... [2]https://www.awn.com/animationworld/how-moonray-became-hidden-superpower-behind-how-train-your-dragon-hidden-world https://www.awn.com/animationworld/how-moonray-became-hidden... EDIT: I've also come across this excellent blog post written by an engineer who worked on MoonRay for 4 years: http://rendering-memo.blogspot.com/2019/ http://rendering-memo.blogspot.com/2019/ And this paper recommended by erichocean in their reply below: http://www.tabellion.org/et/paper17/MoonRay.pdf http://www.tabellion.org/et/paper17/MoonRay.pdf

erichocean 4y ago
http://www.tabellion.org/et/paper17/MoonRay.pdf http://www.tabellion.org/et/paper17/MoonRay.pdf is also a good read.
- mkaic 4y ago
  Thanks, I'll have to check this out! And to think I thought I was going to be productive at work today before seeing this announcement...
echelon 4y ago
> Really impressed that Dreamworks is making this move. It's bone-headed. They could have partnered with a cloud vendor and maintained control and a large percent of the revenue. Instead, Amazon can now turn this into an AWS offering, drive mindshare to the AWS offering, and then bleed DreamWorks of whatever competitive advantage this offered them. AWS makes money, DreamWorks loses talent and leadership that moves over to work at AWS. Open sourcing big software with an enterprise market and giving it a permissive license is stupid, because you can't compete. Look at pretty much every permissively licensed database vendor. Elastic's recent fight comes to mind. In ten years, when this stuff becomes easy enough for small studios to run (something my startup is working on), the need for big studios like DreamWorks to pursue projects funded by institutional capital evaporates. Everyone will be making real time 3D animation. If you look carefully, you can see the genesis of that trend now. If I were in a leadership position at DreamWorks, I would be staffing up how to make these offerings broadly available. How to make tools that put more people in the director seat. That sounds like the mission of a company named "Dream Works". Let more people dream. I guess this does it, just not in a way that they'll be using your company to do so. Big disruptions are coming to the creative fields. This kind of move is like giving away the keys to Amazon for free. (I mean, they are buying Hollywood studios left and right. If you don't see it, you're asleep at the wheel.)

punnerud 4y ago

What can be the real reason for Open Sourcing? That they have a hard time competing with the new machine learning based rendering?

erichocean 4y ago
Every studio has been open sourcing high end VFX/CG software libraries for over a decade. This is just a continuation of that.
deleted 4y ago
[deleted]
anaganisk 4y ago
We are in a world, where left pad was opensource, question must always be why is it closed source. I mean not to say closed is bad, but a question is good.

ramesh31 4y ago

Is this akin to RenderMan? And is this something useful for hobbyists, or is it just something that would run in a render farm?

bogwog 4y ago
A hobbyist film maker probably won't use this directly, but it will likely find its way into software that they do use at some point.
VikingCoder 4y ago
Yes, roughly akin to RenderMan. Yes, some hobbyists will use this.

pitaj 4y ago

I'm not very well acquainted with this kind of thing. Is this something that could be used with other open-source projects like Blender?

knolan 4y ago
Yes, you can use other renderers with Blender. RenderMan for example.
dry_soup 4y ago
It will require the development of an add-on for Blender. It might even already exist.

robalni 4y ago

It's good to see companies freeing important software like this. That shows other companies and everyone else that you can do that without too much risk and that it might even give you benefits. I think many companies are afraid of doing it and I think this is how we can show them that there is no problem with doing it (and if there is a problem, we will learn that; we learn something either way). Since free software is so important to many people and one of the biggest questions about how the digital society should work, it's crucial that we explore the possibilities of releasing free software. I'm tired of the current situation where on one side we have a ton of software and on the other side people who can't use that software because of how unethical or impractical it is. We need to find something that works for everyone and this is a step in that direction.

echelon 4y ago
Amazon is going to take this and make it SaaS. Then the ecosystem moves to Amazon instead of DreamWorks.
- zucker42 4y ago
  Do people pay to use DreamWorks renderer? I would think they pay DreamWorks to animate stuff. That hasn't changed.
  - echelon 4y ago
    Individual creators are now building scenes in Blender and Unreal Engine. The floodgates are about to open for individual creators doing animation. An easy to use render farm is a good startup idea.
    - SEJeff 4y ago
      It's really just some shoestring and bubblegum ontop of amazon / google gpu instances. As commodity AND niche as a renderfarm is, it seems like if it was to run, it would have to be via a foundation like ummm the blender foundation.

johnmertic 4y ago

We added MoonRay today to the ASWF Landscape, which is a great tool for seeing all the open source in the visual and special effects industry. Check it out at https://l.aswf.io https://l.aswf.io

erichocean 4y ago
That's a very helpful page, thank you for maintaining it!

jarrell_mark 4y ago

Blender integration using Hydra. "By adapting Hydra as a render add-on to Blender, any renderer that supports Hydra [ie MoonRay] can be connected to Blender easily, by plugging-in to the Hydra add-on." - https://gpuopen.com/learn/amd-usd-hydra-blender/ https://gpuopen.com/learn/amd-usd-hydra-blender/