12 ms·
PyCon 2015: Are we still changing the world?
- darkxanthos 12y agoRight now, a huge opportunity for Python seems to be data science. I didn't notice it in the list of places where Python shows up. Also, in that space, it more than just shows up, it brings unique capability to the space with its libraries and the support major tool developers have for it.
- tedmiston 12y agoI agree with you 100%. I also got a comment from a NASA engineer that other strong additions are NASA itself and the Department of Energy. If you know of any specific companies, I'd definitely welcome the suggestion.
- mpdehaan2 12y agoI think folks just don't talk about Python, they get things done. While I think, say, other areas also get things done, Rails developers are more likely to blog and have pretty fonts. Nothing against that, it just seems to be the case. While metrics show there's more Python out there, it's just not exciting, it doesn't change a lot, and it just works. And that's fine too.
- tedmiston 12y agoI find myself thinking (hoping) this is the case too. In fact, writing the post to talk about Python at this level was indirectly inspired by a talk I heard at Code Genius (http://code.genius.com/ http://code.genius.com/), put on by the Ruby-based Rap Genius Engineering team. With respect to metrics, I think having the baseline is important, and you're right. I also found myself looking for more granular metrics then what I was able to find. I really want to be able to answer questions like - "Within Python, how is usage shifting between web apps, scientific work (both research prototyping and in practice), open source, machine learning, etc.?" - "How are the 'frameworks on top of frameworks' like Flask-RESTful and Django Rest Framework changing in adoption over time?" - "Is anyone actually adopting Python 3 as their default yet?" I wasn't quite able to find that data, but if anyone has ideas for first steps in getting there, I'd be happy to contribute (email in profile).
- mpdehaan2 12y agoOn my end (east coast US), I hear the Python/data/analytics things a lot but haven't directly encountered those companies. Python was at least previously big in a lot of local shops doing a lot of systems programming type stuff (management apps) -- including folks like Red Hat and HP, including my old one (Ansible). Startups are still highly likely to use it. There are enough Django shops around, including at least one major Django consultancy-type shop, but it feels like slightly more (but not much more) rails on the web side. I'm not plugged into the industry biotech to know how much Python is floating around, but I think it's probably pretty common, just that those folks travel in slightly different circles and we don't cross polinate enough. "Is anyone actually adopting Python 3 as their default yet?" Almost certaintly someone has, the real question is what percentage and what characterizes them :)
- baldfat 12y agoPython is still over shadowed by R in data science. I stopped using Python for data science and use R and it really worked out great for me. R has gained a lot of momentum and I think that Pandas with Wes leaving isn't being developed as quickly. (Most of it is re-doing what R has done). I am amazed at all the FUD I was told to not use R and just use Python but after hitting so many walls I tired it and I am so glad I did. Your millage may vary.
- jules 12y ago<-- Maturity -- Fundamentals well designed --> R Python Julia
- bsg75 12y agoThe relationship between maturity and design with respect to usefulness is not linear, with respect to "usefulness in production".
- jules 12y agoRight, but it does depend on how long term you view it. Problems in the fundamentals of the core language can't be fixed, a mature library ecosystem can be developed.
- PaulHoule 12y agoR is an awful language with great libraries. My main beef about R (and similar tools) is that I summarize my data I get from Hadoop and find it is still too big for R to handle and I then have to summarize it some more.
- Lofkin 12y agoHave you tried data.table?
- baldfat 12y agoHave you tried the Hadley Universe? Seriously there isn't a data set that Python can do that R can't (Lots of ways to do things as opposed to Python's one correct way). Using dplyr has been a huge change in the way I do any data munging. Panda's has started using some of dplyr with new function assign. Dplyr really handles large data sets well like data.tables (data.table is fast and handles larger sets but I like dplyr's syntax more) By the way i should have stated that Python is a good choice for doing Data Science it is just that R works better for me.
- realusername 12y agoI also have the same feeling. Python is very mathematics-friendly compared to other imperative languages and is used a lot more than other mainstream languages for mathematics related projects. On Github, there is a lot of good quality machine learning and statistics projects built with Python.
- woah 12y agoPython is a very nice language, but a big stumbling block is the poor package management. Contrasting this with node, which makes it as easy as possible to build modular software, python's packages are a big step backwards.
- mpdehaan2 12y agoI'm not following this. Can you give examples of features in node? Given, the quality of things in any community repo, and the namespacing, are often terrible, but this hasn't ever kept me from using good things out of it. I'm also not sure how it can be a step backwards, as Python is quite a bit older :) One thing I don't like about npm is folks are starting to store non-npm things in it, which I think may because of the weird situation of npm getting funding. I don't understand that just as much as I wouldn't understand PyPi or CPAN getting funding :)
- carrja99 12y agoThe key difference between pypi and npm is that if you pull a package down and it depends on something==1.0.2 and you depend directly on the incompatible something==0.8.9 both version live together in peace because only request will use 1.0.2. With pypi (and most other dependency management systems) you'll have to make a choice on which one you want to use and in this example you'll have to choose an older version of the library depending on something==1.0.2.
- abstractbeliefs 12y agoThis has been largely solved with virtualenvs where those collisions happen on the system level, but I concede it's harder when the mutually incompatible requirements are in the same project.
- tedmiston 12y agoIf I'm understanding the question correctly, I agree virtualenv + virtualenvwrapper solves it mostly. For what I consider a decent treatment of some of the shortcomings in pip vs. npm, I found this post helpful: "Things I wish Pip learned from Npm" https://medium.com/@alonisser/things-i-wish-pip-learned-from-npm-f712fa26f5bc https://medium.com/@alonisser/things-i-wish-pip-learned-from...
- vegabook 12y agoFor me, the only place where Python truly excels versus the competition is Numpy. The easy python scaffolding gets you to the heart of a problem in a friendly and flexible way, and then you can interface easily to this big V8 of a compute engine (especially if you combine it with the MKL libraries). The ecosystem around Numpy is astronomically good. I have fallen in love again with Python (2) for this reason. I tried 3 but I don't need all it's accountant-like seriousness. It's less fun to code in. When I need to design for servers or for performance I will do golang or ocaml. When I need simple scaffolding for numerical work, and scripting for prototyping integration of various systems, Python 2.7 is like an old, unfussy, friend, helping me along my long and winding data wrangling road with a minimum of cognitive tax. It just works. Long may it live. In my view the Python powers should look at numerical / data science and throw all of Python's weight behind that, and forget about the web, where they've already lost.
- alexhill 12y agoCan you articulate what you mean by Python 3's "accountant-like seriousness"? Is there more to it than print-as-a-function? I code in both every day and just don't see what the fuss is about. To me, they're barely different dialects; more like different accents.
- jMyles 12y agoThe many more generator-style iterables give it a strange dramatic tone to me. Also, the type hints syntax looks utterly un-fun.
- msellout 12y agoYou can ignore the type hints syntax. I plan to. Python 2: map(func, iterable) # returns a list Python 3: map(func, iterable) # returns an iterator list(map(func, iterable)) # returns a list
- jMyles 12y ago> You can ignore the type hints syntax. I plan to. I feel like that will be quite an ear-plugging, eye-covering endeavor. And that's really the point: it has the potential to create a community in which half the people are eyes-closed, ears-plugged. On your second point: Yeah, I don't mind casting things to a list. But it does seem... I guess, dramatic.
- jaimebuelta 12y agoAs a Python developer myself, I think that the community is very "pragmatic engineering" oriented (explicit better than implicit, get things done, simplicity, maintainability, etc) The focus is on getting things done, today and tomorrow; not be trendy or be on the first page of HN or disrupt the language space using a new paradigm. Python is "silently?" used A LOT. Not only on web development (which has more the spotlight on the Internet) but in areas like systems scripting, data analysis or science research. I think that is a great language with a lot of opportunities (including career opportunities) that is still growing. Yes, it's not the new kid in town. I think of it more as an advantage, really.
- amyjess 12y agoYes, I can confirm that I've done a prodigious amount of system scripting in Python.
- tedmiston 12y agoOur pragmatism is one of the biggest factors that draws me to Python as a dev. I agree about our significant use outside of web dev -- even though I know a lot of it is "out there", I had a hard time gathering it with many major orgs not being on CrunchBase, AngelList, or StackShare, (and some with a footprint that doesn't represent their true size on GitHub). If there's a source for this data out there, I'd love to find it.
- ianbicking 12y agoIs Python changing the world, or Go, or Ruby, or Javascript? If you are asking about "changing the world" that's a big thing to ask for. Simply being useful doesn't make a technology world-changing. I feel very confident that Go will not change the world. People may use Go to make things that change the world, but Go won't change the world. If Python has changed the world – that is the language itself has done so, not just the people using the language – then I'd look mostly to its community, governance, and development process. For Ruby I'd look to its culture, philosophy, and discourse. Javascript actually has a technical claim to changing the world: it's created a programming platform that requires no installation of software, and has brought sandboxed development into the world in a broad and practical way. But if we look at Node.js/IO.js, a Javascript platform with similar scope to Python, then I personally see a generational reinvention: a platform that a new generation of budding developers can call their own, and make their own decisions. Asking a lot of questions that no one actively considers in these older platforms. But there's lots of other things that can change the world besides programming languages. Are programming languages really the best place to hang your hat? Do you want to compare Python to Ruby or Node.js, or compare Python a web application or wiki or the cloud; i.e., entirely different categories?
- tedmiston 12y agoIan -- I'm honored for you taking the time to read through and comment. Your post (understandably) has the highest click-through of anything I linked. At first I was a bit taken back by the words myself (borrowed from the PSF [0]), but as you've pointed out, all of these languages have demonstrably changed the world through different aspects. Thank you for your insight and efforts in sparking the conversation. 0: http://pyfound.blogspot.com/2015/02/psf-python-brochure-sold-out-help-us.html http://pyfound.blogspot.com/2015/02/psf-python-brochure-sold...
- bsg75 12y ago"People may use $LANGUAGE to make things that change the world, but $LANGUAGE won't change the world."
- coldtea 12y ago
- PaulHoule 12y agoI think there is a certain kind of sloppiness in Python culture that holds it back. For instance, the split between Python 2 and Python 3 has kept from me getting more into Python because I have to choose one or the other in terms of which libraries I want to be compatible with (and the libraries are a big reason I'd want to to use Python.) If you look at Java or PHP or .NET there is a lot of concern about keeping compatibility between versions, but going from 2 to 3 they changed a lot of APIs that they didn't have to. The most disturbing thing is that I have been shouted down for years on places like proggit for bringing this issue up but it is only in the last year that I have seen it talked about by pythonistas, and that's a general problem I see with Python, that it ducks the hard questions which Java, in particular, faces before anyone else. (i.e. having a memory model for threads which is mature) On one hand, the GIL means that Python is not scalable in terms of threads. Today 4 core machines are mainstream, even for phones, and with a language that supports scalable threads, you can get a 4x speedup easily for many real apps. The other option is go to the Node route, which is balls-to-the-walls async operations in a single process, but (i) the async people also have had a culture of "kicking the can down the road", (ii) it is hard to get programmers who are used to sync APIs to switch to async (this is one reason why the "Modern" interface in Win 8 was DOA... Microsoft was pushing people to use Javascript because rewriting Win32 apps to be async defeats the purpose of using Win32 which is being able to use all the old stuff and not learning anything new.)
- msellout 12y agoThere are many ways to get around the GIL. Use PyPy. Use processes. And are you really doing that much computation? Threads on standard CPython are perfectly fine for I/O.
- actsasbuffoon 12y agoPyPy still has a GIL: http://pypy.readthedocs.org/en/latest/faq.html#does-pypy-have-a-gil-why http://pypy.readthedocs.org/en/latest/faq.html#does-pypy-hav...
- msellout 12y ago
- rkuykendall-com 12y agoPython is changing the world more than ever before, by becoming the new default. I know undergrad CS students, masters CS students, non-CS professionals well into their career, and non-CS hobbyists. And they are all writing Python. Five years ago when I was doing my undergrad, that would have been Java, or C++. It's not exactly an exciting change from a HackerNews viewpoint, but it's pretty amazing for programming as a whole. The people I know are so much more excited to be learning. There is frustration, but much less than when I knew people learning on C++. Obviously that's still a fantastic language, but I'm happy Python is the new default.
- tedmiston 12y agoI was excited when MIT switched to Python for setting the trend for other schools to follow that it's okay to teaching computing from the top down.
- EToS 12y agoIm currently migrating a large FTSE100 web application stack from Java/SpringMVC over to Python/Django.. Its been a long journey (4 years!) to unhinge Java as the given option, but Pythons proven time and time again to save millions per project, and often with better results because developers can concentration on product rather than framework problems.
- rwhitman 12y agoIs the Python 2 vs Python 3 thing ever going to be resolved? If you look at Python adoption from the perspective of someone picking it up for the first time, the very first issue they encounter is the "which version of Python should I use" debate, which pretty much frames the experience from that point onward. The pragmatic attitude that's kept Python 2.X afloat is a key part of why Python has had longevity but it also doesn't set a great tone for "changing the world" or keeping up with the latest sexy fads
- techdragon 12y agoUbuntu 16.04 LTS is going to be Python 3 by default. I switched to Python 3 only a year ago. The war is over, Python 3 won, and now it's just the "too big to fail" guys who still think Python 2 has a future. Google, Dropbox and a few others who will probably give it another5 years before they make the switch using 3.5's type hints to help automate the process of moving to Python 3.
- xorcist 12y agoThere was never a war, and Python 3 didn't win it. We're still not seeing any mainstream Python 3 adoption (no, I don't count your own project). PyPI downloads was still < 5% last I checked. The big web frameworks are still better tested and better performant with 2. Scientific computing is stuck with 2. Same with Ansible. And the big companies you mention have no (public) intention to migrate. The core developers decided long ago that 3 is what everyone should use, so I have no doubt that will be the case. The open question is how much is left of the ecosystem by then. It's commonly discussed at conferences but there is no straightforward answer. At least we got the TLS stuff backported to 2 which might be indicative of a coming thawing.
- jackmaney 12y ago> Scientific computing is stuck with 2. False. NumPy, SciPy, scikit-learn, Pandas, and friends all work great on Python 3.
- streamline92 12y agoThe Python GIL problem could been solved in Python3 by just moving all global interpreter instance variables into a single struct and changing the C API to take a pointer to this interpreter struct as the first argument in all C API functions. No thread local variable hacks need be used and all interpreter instances could have been completely independent of each other, allowing for seamless and efficient share-nothing multi-threading. Python3 was not intended to be 100% compatible with Python2 anyway, so it was a perfect opportunity to fix this long standing design bug. But this opportunity was lost. Strangely, the Google V8 team repeated Python's API design error and also relies on global interpreter variables and thread-local variables to swap interpreter contexts, or "isolates" in V8-speak. Although my understanding is that the V8 team recognizes the problem and are (slowly) addressing it and changing their APIs accordingly with each successive release.
- copsarebastards 12y agoTo be honest, I think the influence of the Ruby/Rails ecosystem is hurting Python. In the last few years, it's become impossible to run many Python projects without doing a pip install -r requirements.txt. That in itself would not be a problem, except that many pip packages are buggy and poorly maintained. Few people take the time to vet such libraries because the effort involved in a pip install is so low. The result is that we often spend hours configuring a buggy library instead of writing code that would do a better job solving our problems. "Convention over configuration" is no less important than it always was, but it's certainly less fashionable. There are exceptions. Django rest framework, for example, is an extremely mature, well-made library. But these are becoming less of a percentage of the whole and are harder to find in the chaff. And there are a few projects still carrying the torch--but typically these are old projects that have been around for a long time, still maintained by older developers who were writing Python when Python was young. We still have at least a decade before these old masters start retiring, so I don't think Python will die any time soon. But it has become more important than ever to be careful what you let into your dependencies.
- tedmiston 12y agoThis one resonates with me. Though I haven't used either myself yet, I've noticed the trend in automated code quality tooling becoming more popular in Python. Specifically, the ones that come to mind are: Codacy, Code Climate (both free for open source); and Landscape. Personally I'm still using strong classics like coverage and pylint, but plan to explore the new gen stuff soon.