8 ms·
They should put energy labels on programming languages. It’s convenient to use Python, but a waste of cpu cycles
by stabbles 4y ago
They should put energy labels on programming languages. It’s convenient to use Python, but a waste of cpu cycles
- lylejantzi3rd 4y agoWhy hasn't anybody created a more efficient version of Python? Slap some static types on that baby, get rid of the heavy runtime, and you've got a good start toward something special.
- doodlesdev 4y agoThere are more efficient interpreters and compilers, problem is they don't work with the existing libraries. If you mean another language, there is Lua for scripting purposes and Julia for math stuff. JavaScript also exists, which despite its fame it is actually MUCH faster than Python, and you can even run it in a runtime such as bun.js and get even more out of it. IMO the biggest problem is that the Python libraries ecosystem is just too good to miss out on. I absolutely hate Python but even I have to admit it is much easier to do some stuff in Python compared to other languages simply because of the libraries available to do basically anything you could dream of.
- ravenstine 4y agoWouldn't surprise me if a subset of Python compiled to asm.js would run faster in V8 than cpython.
- JonChesterfield 4y agoProbably doesn't have to be a subset. Call back to cpython for the really dynamic madness, much like a jit tiers back out of machine code when assumptions don't hold.
- dragonwriter 4y ago> Wouldn’t surprise me if a subset of Python compiled to asm.js would run faster in V8 than cpython. I’m certain that there exists some subset of python for which this is true, but so what?
- bombolo 4y agoBut would this "subset of python" be actually useful? Once you break compatibility stuff doesn't work… and a fast python that can't run my software is less useful than a slow python that runs the software.
- biomcgary 4y agoPython performance, both single core and concurrent, seem to be its primary Achilles heel. However, Python is unmatched for developer speed in compute heavy AI / ML since it uses C, etc., under the hood. I don't really like Python but it's impossible to avoid in data science because you don't have to re-invent the wheel. Several languages have persisted beyond their natural shelf life due to their library ecosystems. For a period of time, CPAN was one of the best arguments for using Perl. More recently, Java's ecosystem is also a pretty good reason to use Java right now, even if you don't like the language. Languages seem to thrive early due to initial ease of use (Python, particularly, but also relative uptake of Go vs Rust), but can wane due to a thousand cuts (or sigils in Perl's case).
- lylejantzi3rd 4y ago> However, Python is unmatched for developer speed in compute heavy AI / ML since it uses C, etc., under the hood. John Carmack seems to think otherwise. > "That means that in the time that Python can perform a single FLOP, an A100 could have chewed through 9.75 million FLOPS" https://twitter.com/id_aa_carmack/status/1503844580474687493?lang=en https://twitter.com/id_aa_carmack/status/1503844580474687493...
- cldellow 4y agoAm I missing something? The OP's comment is that while Python's native interpreter is slow, Python code as a whole can still be fast since people will delegate the hot spots to non-Python libraries optimized for speed. To refute that, you linked to John Carmack quoting Horace He saying that Python's native interpreter does CPU-based math slower than a GPU. But...that's why people writing Python use libraries to delegate math-intensive work to a GPU, which is the OP's point.
- grumpyprole 4y agoIf one is forced into writing code that needs to be fast in a language other than Python, then I'm not sure I understand why Python gets to claim any credit?
- rahen 4y agoThat's the spot filled by Go, and previously Pascal. Native code with good efficiency while still being fairly easy and productive.
- pphysch 4y agoGo and Python have a quite different set of strengths. Python heavily leverages operator overloading and other metaprogramming features that Go does not support (well). If you pick up Go expecting it to be a "more efficient Python", you will be sorely disappointed.
- bombolo 4y agoIn my experience go's libraries are generally not so high quality
- pphysch 4y agoHard to judge. In Python it's very easy to create nice porcelain APIs for powerful frameworks because you can hide complex behaviors behind operators. Imagine Pandas or Django without overloaded access operators. This is not the case in Go.
- bombolo 4y agoI didn't mean how beautiful the api looks like… In my experience authors of go libraries are somewhat noobs on linux/terminal stuff and their libraries fail to take things into account.
- nnadams 4y agoNim [0] might be one similar example. Similar syntax as Python and trying to be just as extendable as Lisp. [0] https://nim-lang.org/ https://nim-lang.org/
- spijdar 4y agoThere are several "close to python" languages like this, actually, I remember one popping up on HN semi-recently. If you expand into just "kind of Python-like", you have languages like Nim that approximately fill the hole of "efficient Python". My impression, though, is a lot of the popular frameworks/libraries for Python lean heavily on runtime reflection in Python (e.g. Django). So people end up stuck to the full Python interpreter/runtime.
- Simran-B 4y agoI consider Ruby to be quite similar to Python. However, it's kind of like Python's weird brother, with optional parentheses for calls without arguments, auto-returning the result of the last expression, and some odd-ball syntaxes. It manages to be even less efficient, though.
- quesera 4y agoAs someone who probably has more experience with Ruby than you do, and probably less with Python, I'd characterize the relationship as inverted -- Ruby is more consistent than Python. In Ruby, parens are optional for all methods, and only required if you need to force non-default precedence. Returning the value of the last expression seems reasonable to me (just like a UNIX shell). In Python, I'm bothered by the little inconsistencies. Why is str.len (or length, or size, or count, etc) not a method? Why len(str) instead? And Python has plenty of oddball syntax: significant whitespace, triple quotes?? You get used to it of course. Ruby and Python are about equally efficient -- if you're optimizing for code efficiency, both are bad choices. In many domains, that's not a critical metric, so we talk about developer efficiency. I'll argue that developers are more efficient when they are more happy, and for me that selects Ruby by a mile over Python. But I won't pretend that it's more than a matter of taste. :)
- wood-porch 4y agoI feel very similarly about Python vs. Ruby inconsistencies! I remember starting a job and used python for the first time coming from Ruby, and being extremely disappointed in the inconsistencies in the standard library and the sheer… lack of features in the Stdlib compared to Ruby. I’ve yet to use a language with such a magnificent stdlib and documentation. I main python nowadays, and I find the ecosystem and typing to be better than Ruby, but Ruby is the best language I’ve ever used for many reasons
- deleted 4y ago[deleted]
- adsharma 4y agohttps://github.com/py2many/py2many/blob/main/doc/langspec.md https://github.com/py2many/py2many/blob/main/doc/langspec.md Reimplement a large enough, commonly used subset of python stdlib using this dialect and we may be in the business of writing cross platform apps (perhaps start with android and Ubuntu/Gnome)
- tlavoie 4y agoAlso languages like OCaml, which while less known, are still getting improvements all the time. This was an interesting blog post series from a few years back: https://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-retrospective/ https://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-re...
- kuang_eleven 4y agoBecause it honestly doesn't matter most of the time. In the majority of use cases, your runtime is dominated by I/O, and for the remaining use-cases, you either have low-level functions written in other languages wrapped in Python (numpy, etc.) or genuinely have a case Python is a terrible fit for (eg. low-level graphics programming or embedded). Why bother making a new variant language with limitation and no real benefit?
- lylejantzi3rd 4y agoThis perspective is a common one, but it lacks credibility. https://twitter.com/id_aa_carmack/status/1503844580474687493?lang=en https://twitter.com/id_aa_carmack/status/1503844580474687493...
- pdonis 4y agoJohn Carmack is hardly an unbiased source. In any case, if your program is waiting on network or file I/O, who cares whether the CPU could have executed one FLOP's worth of bytecode or 9.75 million FLOPs worth of native instructions in the meantime?
- baq 4y agoHis code has had 16.6ms to execute since before a lot of people here had been born. Of course Python is hopeless in his domain. It’s creator and development team will be the first to admit this.
- KyeRussell 4y agoYou’ve posted this multiple times in this thread, and not once has it been relevant to the point being made. You are sticking your fingers in your ears and deferring to a contextless tweet by a celebrity.
- lylejantzi3rd 4y agoIf you want somebody to engage in a serious discussion, insulting them is not the way to go. The post is highly relevant. Next time, if you don't understand why, just ask.
- mrkeen 4y agoThere are plenty of statically-typed languages which are a bit further along than 'a good start'.
- deleted 4y ago[deleted]
- cb321 4y agoYou may be interested in https://en.wikipedia.org/wiki/Cython https://en.wikipedia.org/wiki/Cython which was a fork of Pyrex which dates back to the very early noughties. It is basically as you describe and even has a way to "bypass the GIL" in extension modules. There is also an easy way to create a "compiled script" using `--embed`. This is essentially a gradually typed system [1] like the Common Lisp `declare` facility. Cython even has a warning system to tell you what you forgot to declare, and an annotated HTML generator to highlight "more Python C API heavy" portions of your code. Personally, I think going all the way to Nim [2] is more satisfying than a "gradually typed" system, though. [1] https://en.wikipedia.org/wiki/Gradual_typing https://en.wikipedia.org/wiki/Gradual_typing [2] https://nim-lang.org/ https://nim-lang.org/
- pdonis 4y agoYou might take a look at Nim.
- deleted 4y ago[deleted]
- dragonwriter 4y ago> Why hasn’t anybody created a more efficient version of Python? Slap some static types on that baby, get rid of the heavy runtime, and you’ve got a good start toward something special. The biggest strength of Python is an ecosystem built on a blend of python code and native extensions, leveraging Python’s strength as a glue language. Dumping the runtime while remaining compatible with the extension ecosystem is quite a challenge, as is adding static typing and taking full advantage of it while also maintaining compatibility with existing ecosystem of code which is a mix of untyped and typed in the existing (optional) type system. Which is probably why most “more efficient” Python efforts don’t try to do both of those things. Some did, or at least replacing the runtime, but most of them fizzled, not because they didn’t offer benefits, but because burning access to the existing ecosystem wasn’t worth those benefits. OTOH, what we see now is efforts to improve the base interpreter while maintaining compatibility, and opt-in compilation tools that can be used within Python, with a subset of Python or a statically-typed Python-like language that builds to extensions for the Python runtime. Examples include mypyc, Cython, Numba, etc.
- jray 4y agohttps://nim-lang.org/ https://nim-lang.org/
- grumpyprole 4y agoOCaml would have been awesome for scientific computing, had it got enough adoption.
- rahen 4y agoIf so, there would only be room for C and assembler. Give me at least ksh and awk, they don't use much resources and are infinitely handy.
- nnadams 4y agoA group of researchers published a paper on this topic, "Energy Efficiency across Programming Languages" [0]. They have tables listing the relative energy usage and memory consumption. It's just one paper of course, but the energy results weren't too surprising. C was the baseline. C++, Rust, and Java close behind. Languages like C# and Go near the middle, and then Python, Ruby, and Perl at the bottom for most energy used. [0] https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sle...
- awjlogan 4y agoThis got a reasonable amount of attention, but it’s not particularly relevant. The example solutions would never be written in pure interpreted Python - you would import numpy and use the numeric functions from there, effectively using C rather than Python.
- bee_rider 4y agoI only skimmed it, but it looks like they forgot to account for the extra energy used to run the coffee pot, for the programmers using less productive languages.
- type-r 4y agothe brain is pretty expensive to run
- heresjohnny 4y agoYou seem to favor human brain cycles over CPU cycles. Conversely, time spent on memory management and pointer arithmetic is time that could’ve been used to add value to the world.
- jnsaff2 4y agoWell for code that gets executed billions of times spending more human cycles seems a fair trade off. For boring old corner shop crud, maybe not.
- heresjohnny 4y agoMost code isn’t fire and forget though. Bugs occur, requirements change. The human cycles are a perpetual investment. And Python is simply very, very maintainable. I do agree that there is a point where the scale tips. I wonder where that is, and what kind of application we’re talking about then. Many applications are at their basic level old corner shop crud, really.
- Turing_Machine 4y ago> Well for code that gets executed billions of times spending more human cycles seems a fair trade off. Do you feel the same way about (e.g.) modern mechanized farming compared to classic farming that used human labor (generally unfree and unpaid labor, such as serfs or slaves)? "Efficiency" is not the only goal that matters.
- jnsaff2 4y agoI don’t get your analogy. Comparing manual labor to industrialized farming is comparing (paper) accounting books to programming. Whether you add gps based differential seeding to just doing it by ‘feel’ but still with machines is the difference of programmer brain cycles. Efficiency is not all obviously and there are different efficiencies one can find relevant. Why is this controversial?
- Turing_Machine 4y ago
- deleted 4y ago[deleted]
- duckmysick 4y agoOr to go one step further, phase out (outlaw) energy-inefficient programming languages like we do with high-emission cars. I'm not advocating for it, but it's an interesting thought experiment.
- thorncorona 4y agoPointless when heavy industry utterly dwarfs computing. Might be nice for embedded devices though, but they already run C/C++...
- tmtvl 4y agoDoing a good thing does not benefit from whataboutery.
- bee_rider 4y agoTax carbon and let the market sort it out. Generally I’d expect less efficient languages to be more user friendly (well, a language which has neither advantage is just bad, why is anybody using it?). So, hypothetically these companies should get away with spending fewer programmer-hours on things. They won’t have to run the coffee pots, and air conditioners as late.
- jmull 4y agoGeneral-purpose processing units are convenient to use but waste so much electricity and materials compared to dedicated hardware. …A silly proclamation, of course.
- fulafel 4y agoOn the other hand, when you spend more programmer hours on less productive languages, those programmers end up having a larger energy and natural resources footprint with the increased amounts of money paid out. 99% of programming projects aren't "web scale" and the development costs far outweigh the operating hardware costs.