3 ms·
Python has already become a lot more than a scripting language. To say today that scripting is it's core identity seems naive at best. Yes, it has roots but h
by metadat 4y ago
Python has already become a lot more than a scripting language. To say today that scripting is it's core identity seems naive at best. Yes, it has roots but has object oriented and functional facets which do not exist in awk or bash.
Pandas, numpy, scipy, tensorflow. All of these go way beyond what is possible with a scripting language.
Since when is the runtime performance of a script a serious concern? Why is it a problem if this aspect gets slightly slower if it brings real concurrency support?
- pclmulqdq 4y agoAwk is a functional language with a surprising number of features. Bash, not so much. Developers really do care about runtime performance of scripting languages: They often wait for scripts to finish running before doing other work, and if the sum of the time it takes to write and execute a script is too long, they will look for an alternative. All of the libraries you have cited are from the ML and stats communities, and they are not core language features. From what I understand, ML folks like Python because it is fast to play with and get results. In other words, they like it because it is a scripting language. Personally, I like that Python has kept the GIL so far because I would never run a 24/7 server in Python and I am happy to use it very frequently for single-threaded scripting tasks. Edit: I didn't decide that Python was a scripting language. The Python maintainers did. The point is that the identity of a project doesn't flow down from its use cases. Edit 2: I should have said "the identity of a project doesn't flow down from its users."
- Supermancho 4y ago>> To say today that scripting is it's core identity seems naive at best. Yes, it has roots but has object oriented and functional facets which do not exist in awk or bash. Pandas, numpy, scipy, tensorflow. All of these go way beyond what is possible with a scripting language. No, it is not. It's what you can do with a scripting language, which is a fairly straightforward functional categorization. If you are very familiar with it and really want to just ignore the performance issues and don't mind strange workarounds (eg elaborate libraries like Twisted), you probably use python. > The point is that the identity of a project doesn't flow down from its use cases. Use cases are what determines the identity of a language. It takes a massive and ongoing marketing campaign to convince people otherwise, with limited success without use-cases. Python is popular because it's a scripting language with a relatively low learning curve (one true way to do things philosophy). That's it. It will improve over time, but it's been slow going...and that's in comparison to the new Java features! Haskell is a shining example of how the language identity is foisted upon the public until you are convinced enough to try it. It doesn't take long to learn it's a nightmare of incomplete features and overloaded idioms for common tasks, so it isn't used. There aren't good use-cases for a language with haskell's problems, so the developer community and industry-at-large avoids it.
- ghshephard 4y ago"Personally, I like that Python has kept the GIL so far because I would never run a 24/7 server in Python and I am happy to use it very frequently for single-threaded scripting tasks." Just as a side-note - my prior gig used Python on both the Server and Data Collection industrial systems. It was very much a 24x7x365 must-never-go-down type of industrial application, and, particularly when we had a lot of data-sources, was very multi-processing. Was not unusual to see 32 processes working together (we used sqlite and kafka as our handoff and output of processes) running on our data collection appliances. Our core data modelling engine would routinely spin up 500 worker pods to complete the work needed to be done in 1/500th of the time, but we would still see some of the long term historian runs take upwards of a week to complete (many hundreds of machines for multiple years with thousands of tags coming in every 5 seconds is just a lot of data to model). I say this mostly to demonstrate that people and companies do use python in both large-processing intensive environments as well as industrial-must-never-stop-24x7 mission critical appliances. I don't ever recall any of our engineers looking at Python as "a scripting language" - it was no different to them than Java, C#, Rust, Go, C++ or any other language.
- pclmulqdq 4y agoI know that people do use python for 24/7 "must not fail" applications. I'm just not smart enough to write python that I would trust like that. Python comes with a tremendous number of foot guns and you have to find them because there is no compiler to help, and it can be a real pain to try to understand what is happening in large (>10,000 line) python programs.
- ghshephard 4y agoI think the key here is to see using the Python language as an engineering discipline like any other, and just take the classes, read the literature, learn from more Senior Engineers, and projects, before attempting to develop these types of systems on your own. I don't think anybody expects a recent graduate from computing science, with maybe 4 or 5 classes that used python under their belt (and maybe a co-op or two) to be writing robust code (perhaps in any language). But, after working with Sr. Engineers who do so, and understanding how to catch your exceptions and respond appropriately, how to fail (and restart) various modules in the face of unexpected issues (memory, disk failures, etc...) - then a python system is just as robust as any other language. I speak from the experience of running them in factories all over the planet and never once (outside of power outages - and even there it was just downtime, not data loss) in 2+ years seeing a single system go down or in any way lose or distort data. And if you want more performance? Make good use of numpy/pandas and throw more cores/processes at the problem. Just being aware of every exception you can throw (and catching it) and making robust use of type hinting takes you a long way. Also - and this may be more appropriate to Python than other languages that are a bit more stable, an insane amount of unit and regression testing helps defend quite a bit from underlying libraries like pandas changing the rules from under you. The test code on these project always seemed to outweigh the actual code by 3-5x. "Every line of code is a liability, Every test is a golden asset." was kind of the mantra. I think that what makes python different from other languages, is that it doesn't enforce guardrails/type checking/etc... As a result, it makes it trivial for anyone who isn't an engineers to start blasting out code that does useful stuff. But, because those guardrails aren't enforced in the language, it's the responsibility of the engineer to add them in to ensure robustness of the developed system. That's the tradeoff.
- pjmlp 4y agoExcept most of them are bindings written in native languages, being scripted from Python.
- p10_user 4y agoBut all the “magic” that makes the scientific stack so great is largely due to numpy (and many other Fortran or c driven code) being fantastic. Python is the glue scripting language for organizing and calling procedures. That’s,IMO, it’s original and core purpose. Now the fact that you can do some fun metaprogramming shenanigans in Python just speaks to how nice it is to write.