7 ms·
Are Google employees being discouraged from using Python for new projects?
- iseff 17y agoIt sounds less like they're being discouraged from using Python, and more like they're being encouraged to think critically about what sorts of projects Python would excel at. Put another way, they're being encouraged to use the right tool for the job.
- amichail 17y agoWhat's wrong with prototyping a service using Python and if it takes off port it to a better language for scalability?
- 8plot 17y agoThe problem, as pointed out in the article/thread, they want products that will _immediately_ scale if the need arises.
- lacker 17y agoThe problem is that at Google, "taking off" is often equivalent to "launching". When Google launches stuff, it can have millions of users on day 1.
- aristus 17y agoThis. Working for a company with a large audience means that your problem often is no longer getting people to pay attention. Your problem is getting it right, at scale, in multiple languages and locales. This can be alleviated somewhat with internal tests, invite-only alphas, bucket testing, and "labs" features.
- rbanffy 17y agoThat and good monitoring. I always try to launch services with very detailed server monitoring - I want to know how much memory is being used with what, how much I/O and how much time the CPUs spend doing non-application stuff. I want to monitor response times, queue and dataset sizes and anything that helps me say if we will need more servers, different servers or what parts of the application we should port to amd64 assembly.
- blasdel 17y agoAmazon has an interesting approach to this for extant products: A/B testing for infrastructure. Just as they test UI tweaks by funneling percentages of live customers through them and comparing conversions, they also send live traffic to both production and development servers (with the user only seeing the production responses), checking to see if they produce the same output after refactoring.
- Tichy 17y ago"I don't think it's possible to make an implementation like CPython as fast as an engine like V8 or SquirrelFish Extreme that was designed to be fast above all else." Are they saying that JavaScript is already faster than Python? I've only recently started using some Python, and while it seems to be a decent language, I have not yet seen much that would make me prefer it over JavaScript. Some things are a bit smoother in Python, others are smoother in JS. Here's hoping that JavaScript will win :-)
- jimmybot 17y agoAnyone have numbers on how much faster V8 is than CPython?
- spamizbad 17y agohttp://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=python&lang2=v8&box=1 http://shootout.alioth.debian.org/u32/benchmark.php?test=all... CPython is between 32 faster and 14 times slower than Google's V8. Most of the time, however, V8 is 2-6 times faster than Python. Overall, V8 is quicker. The authors of Unladen Swallow have talked a bit about performance challenges in optimizing - a big one is the fact that they're not breaking anything that depends on the CPython interpreter- include C-based modules, and even esoteric Python modules that modify python bytecode on the stack.
- alecco 17y agoWatching V8 and Python development progress, I bet V8 performance will improve significantly in the next couple of years. Also Python comes with the baggage of a much more complex language and feature set.
- igouy 17y ago> CPython is between 32 faster When the CPython program uses GMP and the JavaScript program doesn't.
- rbranson 17y agoV8 is very impressive. It is one of the fastest virtual machines for dynamic languages. Look where it scores on the shootout. http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all&box=1 http://shootout.alioth.debian.org/u32/benchmark.php?test=all... It is nearly 10x faster than Ruby 1.8 and 5x faster than CPython. Of the shootout, it is the fastest dynamically typed language other than LuaJIT.
- staunch 17y agoThe biggest risk Google faces is not developing some awesome new thing. There is very little risk that they will develop some awesome new thing, but not be able to make it scale. If discouraging developers from using Python means that one great project doesn't happen it's probably a net lose for them.
- lucifer 17y agoPrecisely what category of "awesome new thing[s]" needs to be developed in Python and can not be implemented in Java or C?
- staunch 17y agoI'm not saying there's anything technically wrong/bad about Java/C++. Awesome new things come from people's brains. Some people's brains are sensitive to things like motivation/happiness. Python makes some people's brains more motivated/happy than Java/C++ does. An new thing may never get off the ground, or be taken far enough to become awesome if the developer doesn't have the motivation to work on it.
- rbanffy 17y agoA tongue-in-cheek reply could be "something that has to be done in a short time?" A programmer using Python (and Ruby, Perl, Lisp, Smalltalk) is usually more productive than one with comparable brain-size using Java or C.
- lucifer 17y agoProductive in this context means 'coming up with an idea that Google can turn into a product'. /tongue-in-cheek
- rbanffy 17y agoA nice approach is to develop in your VHLL of choice, profile and rewrite the more important parts in a less expressive (but faster/leaner) language like Java or C (or Verilog)
- lacker 17y agoIt just depends on what sort of project it is. If you know that you will get millions of users per day starting from day 1, then you have to design for some level of scale from the beginning. Prototypes in Python are common at Google, but usually it will have to be rewritten before launch.
- jxcole 17y ago<sarcasm>Maybe they should just develop everything in lisp!</sarcasm> PS Here's to hoping the W3C adopts the sarcasm tag.
- coffeemug 17y agoThe scalability argument doesn't make sense to me. When you're dealing with Google's scale, you need to parallelize horizontally, and you need to design your software in a way which lends itself to horizontal parallelization. For this type of software how much you squeeze out of a given machine is almost irrelevant to scalability (other than the cost of maintaining the extra machines). Of course once the cost goes up too much, you can always rewrite in C.
- coliveira 17y agoAnd using Python in this scenario is better than using Java, where you may have faster execution but a huge memory use due to the VM.
- wmf 17y agoYou're assuming that Java uses more memory than Python; why?
- gaius 17y agoExperience. Or you can look at the numbers for yourself on the shootout site. Java using 10-100x as much memory is not uncommon.
- SwellJoe 17y agoThe JVM memory cost is generally paid only once. Python doesn't have an effective thread implementation, but Java does. And, of course, CPU usage of Python is pretty much unarguably higher than Java for most things, assuming well-written code in each language.
- rbanffy 17y agoI thought that in any modern OS you would have the libraries loaded only once and the only thing that is multiplied across processes (or threads) is the working data for that specific thread or process. The overhead should be minimal. If not, it's an OS problem outside the domain of the Java and Python maintainers. What we are apparently seeing is that the working data is larger in Python.
- deleted 17y ago[deleted]
- euroclydon 17y agosearch for unladen swallow
- rbanffy 17y agoAs unladen as it is, it is still very compatible with Python modules. If Unladen Swallow is to go faster, it would break backwards compatibility with modules written in C. And that wouldn't be as bad an idea as it seems. Sometimes, what you really need is a good API reboot. Also, a good approach is to prototype in a VHLL, profile and rewrite the rest in lower level languages such as Java or C.
- megaman821 17y agoI wonder how much stuff is written in C for speed and how much is just a Python wrapper for C code. The first problem isn't so bad since speed is what is increasing. The second problem is pretty big because new wrappers or Python equivalents would have to be written.
- draegtun 17y agoHow does "unladen swallow" fit into running Perl, Ruby and other dynamic languages?
- elq 17y agothat would be nice... oh, wait a minute http://code.google.com/p/unladen-swallow/wiki/ProjectPlan http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
- draegtun 17y agoYou're missing my point. I'm talking about combining efforts with all the language communities (ie. Python, Perl, Ruby, etc). Focusing the energy rather than dissipating it over many VM projects. Parrot is designed with multiple languages in mind where as "Unladen swallow" is designed for Python.
- fjabre 17y agoJava instead of python..? ick.. Also, with the resources that Google has I'm a little baffled as to why they couldn't devote some serious effort into getting python up to spec performance wise.
- rgoddard 17y agoThey are, hence the Unladen Swallow project. But the improvements are that are possible are constrained by the language and having to maintain backwards compatibility. If Google were to just take the language and do with it what they want, they would risk alienating the community which would weaken the utility of the language. So while improvements are being made, the types of improvements are limited.
- fjabre 17y agoThanks. Makes a little more sense now. So basically: it's complicated.
- draegtun 17y agoNow if "all" the opensource dynamic languages could run off the same opensource VM then that would be something? The combined effort of all communities working towards producing a scalable & fast VM could make a big difference? Is it time to think Parrot?
- easp 17y agoAre you posting this from 2005?
- draegtun 17y agoAre you posting from 1991? ;-)
- draegtun 17y agoHmmm... HN has done a funny here. It deleted this from top of its original thread and put it elsewhere (with slightly different timestamp)?? Here's two replies to original thread where it says parent is deleted. http://news.ycombinator.com/item?id=933718 http://news.ycombinator.com/item?id=933718 http://news.ycombinator.com/item?id=933710 http://news.ycombinator.com/item?id=933710 Very strange.
- maigret 17y agoSo (at least small) Python programs tend to have less lines than Java ones - OK. That makes them maybe less bug prone. But Java, being laughed at all the time, has got to be one of the most mature and supported programming languages out here. It means it have an outstanding toolset on every platform, and that the Virtual Machine is extremely well optimized. Damn, Java is almost as fast as C++ when written well. After that, it depends on the programmers skill. That's another discussion :)
- megaman821 17y agoIf Google had a choose their development stack today without worry about any legacy development, what do you think it would look like. I think Python/C/Erlang or Jython/Java/Erlang would be a good fit.
- mkelly 17y agoNo. Next question?
- Virax 17y agoQuestion popped into my head: why is Colin Winter wasting his time with idle chit-chat like this? Is he bored of Unladen Swallow already?
- yason 17y agoWhere comes the imperative that Google ought to always consider using Python for new projects? Or more Python in general? They already contribute to Python and apparently use it in production, too, but I don't see that translating to the above.
- Kirby 17y agoAlso, keep in mind that you have to get _very_ big before this becomes an issue. If you're at Yahoo, Google, MSN - yes, language issues can become a performance design consideration. If you're at merely a big site, like Ticketmaster, IMDb, or Livejournal, with good software design you can handle a lot of load with reasonable responsiveness. (All those three sites are written in perl, in fact. I've worked for one of them.) If your page views per day on your project aren't peaking in the billions, you're probably better off optimizing for the language that your team is most competent in.
- fauigerzigerk 17y agoNo you don't have to be _very_ big, not even big, for the language to become a design consideration. You just have to do more than ship a few strings back and forth between the browser and the database.