6 ms·

Changing the Culture of Python at Facebook [video]

81 points by sandGorgon 10y ago

sctb 10y ago
We've changed the link from https://youtu.be/nRtp9NgtXiA https://youtu.be/nRtp9NgtXiA. Please submit the full URL, since HN kills links from shorteners as they break the referential integrity of the post.
- sandGorgon 10y ago
  interestingly, this is because of mobile apps. I used the share function in the youtube app, coupled with the "copy link" intent on android that pops up. This is the link that was given to me and i pasted it here. On a mobile device using a native app, it is quite hard/impossible to get a different url. Could you whitelist the youtu.be url ? because I'm guessing this will start to happen more frequently.
  - throw_away 10y ago
    Or, perhaps hn could just auto-dereference the links to a whitelist of the most popular shorteners & then everyone can be happy?
    - sctb 10y ago
      Thanks, that's a good idea that has come up before. We've got it on our list to look at.
      - jimminy 10y ago
        Part of the reason Youtube provides the youtu.be link is that their systems disambiguate between mobile and desktop websites. Mobile versions have the 'm.' subdomain, and provide a less optimized experience for desktop. I would also argue that the youtu.be link is a canonical form, since the identifiers are identical to the source. It is just an optimized canonical proxy.
sandGorgon 10y ago
one of the interesting things in this video is that Facebook pretty much invented an equivalent of Flatpak/Zeroinstall/Click-packages to deploy their python apps.
- krakensden 10y ago
  It's actually not super hard, Unix has pretty good support for this, and a lot of companies have something like this internally. The gulf between "good enough for a company" and "good enough for the world" is pretty huge. Plus, the extent to which people can't do this because they have no/have a terrible build system is generally underestimated.
- xapata 10y ago
  The speaker says, "dependencies are associated with the application (as they should be) rather than with the system" That's certainly a practical strategy, but it reduces flexibility/modularity. Go-lang shares the same strategy for deployment. Perhaps it was inspired by Java-users.
  - tedd4u 10y ago
    There's good and bad there, if you need to patch a security vulnerability you may need to redeploy tons of packages on each host (each package that contains a copy of the bad lib code) versus patching it once. Depending on how you deploy that might be no big deal or it might be.
    - rb2k_ 10y ago
      Then again, patching it 'once' usually involves restarting all processes that are dynamically linked anyway (and really hoping that it loads the new lib). If deployment is fully automated and scales well, the only painful thing is the restart to begin with :)
- justinsaccount 10y ago
  See also https://github.com/pantsbuild/pex https://github.com/pantsbuild/pex I've had great success using pex to bundle up tools that use a few dependencies. It's a lot simpler than tools that try to bundle a full python distribution.
deleted 10y ago
[deleted]
xapata 10y ago
The punchline: - Python is popular at Facebook - Python 3 is the default for new projects - Changing to Python 3 needed an internal advocate to update build and dev tools to push the change - Facebook is a big user of the new ``asyncio`` module Perhaps the most subtle point of the talk is that Facebook managed to change from 2 to 3 from the efforts of just one person. Of course there were more folks changing code than just the speaker, but it appears he was able to heavily influence the culture change. Enabling 3 for the build tools probably made a big difference.
- makmanalp 10y ago
  Plus, teaching new hires to use py3 as default, putting in linter warnings to get rid of old bad habits, adding in 2to3 compat layer libraries by default and switching to 3 as default after that seems to have helped.
  - xapata 10y ago
    All stuff that can be done by 1 person at your company. Go forth and live in the __future__!
    - ludamad 10y ago
      The cost of 1 person varies heavily though :P
- Animats 10y ago
  Changing Python versions is hardly a "culture change". Changing to Go or Rust might be. The main problem is library support, not the language changes. A large company like Facebook can afford to put someone on library compatibility to make this happen.
  - xapata 10y ago
    Apparently that wasn't even necessary. The big change for library compatibility was simply changing the settings of the build system. According to the speaker. In my own experience, most libraries are 3-compatible.
kirkdouglas 10y ago
I haven't seen the video yet, but did they mention contributing back to the community?
yeukhon 10y ago
> - Python 3 is the default for new projects What teams are using Python at Facebook now? So far job requirements seem to be mainly C++ and PHP. Do they use Cython to speed up? What about PyPy?
- fbreduc 10y ago
  not many, just some op-y type jobs
- peterjlee 10y ago
  Instagram uses django
- rb2k_ 10y ago
  The Production Engineering org at Facebook uses a lot of Python. Check out https://code.facebook.com/posts/1040181199381023/python-in-production-engineering/ https://code.facebook.com/posts/1040181199381023/python-in-p...

81 points by sandGorgon 10y ago

sctb 10y ago

We've changed the link from https://youtu.be/nRtp9NgtXiA https://youtu.be/nRtp9NgtXiA. Please submit the full URL, since HN kills links from shorteners as they break the referential integrity of the post.

sandGorgon 10y ago
interestingly, this is because of mobile apps. I used the share function in the youtube app, coupled with the "copy link" intent on android that pops up. This is the link that was given to me and i pasted it here. On a mobile device using a native app, it is quite hard/impossible to get a different url. Could you whitelist the youtu.be url ? because I'm guessing this will start to happen more frequently.
- throw_away 10y ago
  Or, perhaps hn could just auto-dereference the links to a whitelist of the most popular shorteners & then everyone can be happy?
  - sctb 10y ago
    Thanks, that's a good idea that has come up before. We've got it on our list to look at.
    - jimminy 10y ago
      Part of the reason Youtube provides the youtu.be link is that their systems disambiguate between mobile and desktop websites. Mobile versions have the 'm.' subdomain, and provide a less optimized experience for desktop. I would also argue that the youtu.be link is a canonical form, since the identifiers are identical to the source. It is just an optimized canonical proxy.

sandGorgon 10y ago

one of the interesting things in this video is that Facebook pretty much invented an equivalent of Flatpak/Zeroinstall/Click-packages to deploy their python apps.

krakensden 10y ago
It's actually not super hard, Unix has pretty good support for this, and a lot of companies have something like this internally. The gulf between "good enough for a company" and "good enough for the world" is pretty huge. Plus, the extent to which people can't do this because they have no/have a terrible build system is generally underestimated.
xapata 10y ago
The speaker says, "dependencies are associated with the application (as they should be) rather than with the system" That's certainly a practical strategy, but it reduces flexibility/modularity. Go-lang shares the same strategy for deployment. Perhaps it was inspired by Java-users.
- tedd4u 10y ago
  There's good and bad there, if you need to patch a security vulnerability you may need to redeploy tons of packages on each host (each package that contains a copy of the bad lib code) versus patching it once. Depending on how you deploy that might be no big deal or it might be.
  - rb2k_ 10y ago
    Then again, patching it 'once' usually involves restarting all processes that are dynamically linked anyway (and really hoping that it loads the new lib). If deployment is fully automated and scales well, the only painful thing is the restart to begin with :)
justinsaccount 10y ago
See also https://github.com/pantsbuild/pex https://github.com/pantsbuild/pex I've had great success using pex to bundle up tools that use a few dependencies. It's a lot simpler than tools that try to bundle a full python distribution.

deleted 10y ago

[deleted]

xapata 10y ago

The punchline: - Python is popular at Facebook - Python 3 is the default for new projects - Changing to Python 3 needed an internal advocate to update build and dev tools to push the change - Facebook is a big user of the new ``asyncio`` module Perhaps the most subtle point of the talk is that Facebook managed to change from 2 to 3 from the efforts of just one person. Of course there were more folks changing code than just the speaker, but it appears he was able to heavily influence the culture change. Enabling 3 for the build tools probably made a big difference.

makmanalp 10y ago
Plus, teaching new hires to use py3 as default, putting in linter warnings to get rid of old bad habits, adding in 2to3 compat layer libraries by default and switching to 3 as default after that seems to have helped.
- xapata 10y ago
  All stuff that can be done by 1 person at your company. Go forth and live in the __future__!
  - ludamad 10y ago
    The cost of 1 person varies heavily though :P
Animats 10y ago
Changing Python versions is hardly a "culture change". Changing to Go or Rust might be. The main problem is library support, not the language changes. A large company like Facebook can afford to put someone on library compatibility to make this happen.
- xapata 10y ago
  Apparently that wasn't even necessary. The big change for library compatibility was simply changing the settings of the build system. According to the speaker. In my own experience, most libraries are 3-compatible.

kirkdouglas 10y ago

I haven't seen the video yet, but did they mention contributing back to the community?

yeukhon 10y ago

> - Python 3 is the default for new projects What teams are using Python at Facebook now? So far job requirements seem to be mainly C++ and PHP. Do they use Cython to speed up? What about PyPy?

fbreduc 10y ago
not many, just some op-y type jobs
peterjlee 10y ago
Instagram uses django
rb2k_ 10y ago
The Production Engineering org at Facebook uses a lot of Python. Check out https://code.facebook.com/posts/1040181199381023/python-in-production-engineering/ https://code.facebook.com/posts/1040181199381023/python-in-p...