5 ms·

I’ve worked on Python codebases >100k LOC with no issues. What actually is your point?

toolslive 8y ago
having been on the receiving end of a large python project, I can say that refactoring is (trying to stay polite here) a challenge.
- pmiller2 8y ago
  I don’t see how that has anything to do with Python per se. It’s more of an issue with large codebases.
  - toolslive 8y ago
    Dynamically typed languages make it harder to refactor a large codebase than statically typed languages do.
    - jernfrost 8y ago
      In some ways and some ways not. Making a global change is much harder in a dynamic language. However static languages by their very nature tie the code together more tightly. It is much easier to decouple something in a dynamic language. The re-factoring needs in a dynamic language is thus entirely different.
      - TimJYoung 8y ago
        Yes, but these are two sides of the same coin: being able to inject code dynamically in a way that cannot be statically checked at compilation time results in code that will cause runtime errors if there are issues (even basic issues). Also, I would argue that your use of the words "tie the code together" implies something that isn't actually occurring. With a static type system, verifying that code is using the correct types for classes, methods, functions, etc. is not binding them together in any way. If you drop down to assembler, you can do anything you want with any of it, just as you would with a dynamic language. A static type system is simply a compile-time integrity check on the validity of your code. Also, the de-coupling and modularization of functionality has been a thing long before dynamic languages became so popular. You simply need an FFI, and then the compiler makes sure that everything is straight before you get anywhere near deployment.
  - adrianN 8y ago
    Type systems generally are thought to help with large scale refactoring. Unless you have perfect path coverage with your unit tests, there can always be paths lurking after refactoring your duck objects to be geese.
- est 8y ago
  refactoring a equivalent Java project is way more challenging. I'd argue. Refactoring a python project is usually one man's job. Refactoring a Java project takes a whole department and process management crew.
  - qalmakka 8y ago
    Java is absolute garbage, and it is definitely not the kind of language I've in mind, which is something like Go, Rust, C#, Kotlin, Swift, ocaml... Even C++ is perhaps easier to understand than Java, if not because it tends to have half boilerplate than a similarly sized Java project.
  - potatoyogurt 8y ago
    Why do you think Java is so much harder to refactor? Java and python are not that much different, and Java has static types which aid refactoring as well as a really strong tooling ecosystem. I haven't worked with large python codebases, but it hasn't really been my experience that refactoring Java is unusually hard, except insofar as average code quality among projects written in Java is probably lower.
    - toolslive 8y ago
      Java IDEs have good support for refactoring since the late 90s: - rename/move class (and autoupdate all references) - change method signature most of it is just stating what you want and the tool does it for you. I don't see that level of support from python IDEs. https://news.ycombinator.com/item?id=9671783 https://news.ycombinator.com/item?id=9671783
  - senorjazz 8y ago
    > refactoring a equivalent Java project is way more challenging. I'd argue. Sorry but you are wrong. Just plain wrong. There are benefits for both languages. But to say python is easier to refactor is like saying Java is more concise. It isn't. >Refactoring a Java project takes a whole department and process management crew. Has nothing to do with the language, but the company. A small company will refactor and have an update out same day in Java. An enterprise using some python will be as you say
- tom_ 8y ago
  It's impossible. Once your program gets past ~1,000 lines, you're stuck with it, basically :( I've always found C++ much easier to work with, if not to actually write.
- fabioz 8y ago
  Personally, I think that refactoring Python is straightforward (and many refactorings you need to do are much simpler and easier to do without breaking existing APIs -- doing refactorings in Java usually needs changing much more code even for simple things). I think that the real problem is not having a good test coverage, and I agree that bad programs written in Python may be harder to work with than bad programs written in a statically typed language (but in a good codebase, even if large, I'd rather be working with Python). -- although note that in the latest incarnation of Python you can try to use Mypy (http://mypy-lang.org/ http://mypy-lang.org/) along with https://github.com/Instagram/MonkeyType https://github.com/Instagram/MonkeyType or https://github.com/dropbox/pyannotate https://github.com/dropbox/pyannotate to add types to you and go from there if you really miss types too much and this could make a bad codebase bearable (I'm personally waiting for PEP 544 before adding any types though as I'm not a fan of the type system implemented in PEP 484 very much).
  - scarface74 8y ago
    Personally, I think that refactoring Python is straightforward (and many refactorings you need to do are much simpler and easier to do without breaking existing APIs -- doing refactorings in Java usually needs changing much more code even for simple things). Refactoring in a non statically typed language cant be as straightforward - many refactors in statically typed language can be automated and guaranteed safe.
qalmakka 8y ago
Like someone else said, I personally find that very large codebases written in dynamically typed languages tend to discourage refactoring. You may try to mitigate the issue by writing the whole application as standalone, independent modules, but we all know that a bit of interdependence always creeps inside even the best designed application. I love Python; that's the first language I've learnt and I can't really live without its REPL. I only think that the industry has a tendency to excessively obsess towards certain languages, even when they are not the best tool for the job. For instance, I strongly believe that the academia's morbid obsession with Java has crippled a whole generation of new programmers, which are now unable to think and code in no other way but the Java™ Way. I won't say that python is as bad as Java in this regard (it isn't even close), but it annoys me to death when it looks like we're heading once again towards a language monoculture; at least this time the language isn't utter crap.
- devxpy 8y ago
  > looks like we're heading once again towards a language monoculture Isn't that's only true for data science? What about the JavaScript monoculture :( And academia still has courses for C java C++ etc. No respectable University teaches a single language!
  - qalmakka 8y ago
    In many Universities, C introductory courses are taught by people that hasn't touched C in at least two decades, or has close to zero experience to how programming modern C is. More often than not, this C is taught by people that have only used Java in years; I've seen horrible stuff, such as C89 still being thought like it's the only standard available, or gets() being used by professors that have no idea whatsoever of what's wrong with them.
    - devxpy 8y ago
      Here in India, 90% of teachers haven't touched what they're teaching, like ever. But you still _have_ to study it because you need to give the exam... Not like it solves anything -- just that there's this side of the argument too.. > I've seen horrible stuff, such as C89 still being thought like it's the only standard available, or gets()... Youre absolutely right -- although one could argue that the curious will have the decency to look this stuff up (like myself) The teaching system is still pretty rekt on the python side of things. Its not like you'll find people who actually code teaching at Universities anyway :( Like now we're just going into how bad the teaching system is. The original argument that python is reaching is a monoculture seems weird to me, when I can't even run the damn thing on a browser..

having been on the receiving end of a large python project, I can say that refactoring is (trying to stay polite here) a challenge.

pmiller2 8y ago
I don’t see how that has anything to do with Python per se. It’s more of an issue with large codebases.
- toolslive 8y ago
  Dynamically typed languages make it harder to refactor a large codebase than statically typed languages do.
  - jernfrost 8y ago
    In some ways and some ways not. Making a global change is much harder in a dynamic language. However static languages by their very nature tie the code together more tightly. It is much easier to decouple something in a dynamic language. The re-factoring needs in a dynamic language is thus entirely different.
    - TimJYoung 8y ago
      Yes, but these are two sides of the same coin: being able to inject code dynamically in a way that cannot be statically checked at compilation time results in code that will cause runtime errors if there are issues (even basic issues). Also, I would argue that your use of the words "tie the code together" implies something that isn't actually occurring. With a static type system, verifying that code is using the correct types for classes, methods, functions, etc. is not binding them together in any way. If you drop down to assembler, you can do anything you want with any of it, just as you would with a dynamic language. A static type system is simply a compile-time integrity check on the validity of your code. Also, the de-coupling and modularization of functionality has been a thing long before dynamic languages became so popular. You simply need an FFI, and then the compiler makes sure that everything is straight before you get anywhere near deployment.
- adrianN 8y ago
  Type systems generally are thought to help with large scale refactoring. Unless you have perfect path coverage with your unit tests, there can always be paths lurking after refactoring your duck objects to be geese.
est 8y ago
refactoring a equivalent Java project is way more challenging. I'd argue. Refactoring a python project is usually one man's job. Refactoring a Java project takes a whole department and process management crew.
- qalmakka 8y ago
  Java is absolute garbage, and it is definitely not the kind of language I've in mind, which is something like Go, Rust, C#, Kotlin, Swift, ocaml... Even C++ is perhaps easier to understand than Java, if not because it tends to have half boilerplate than a similarly sized Java project.
- potatoyogurt 8y ago
  Why do you think Java is so much harder to refactor? Java and python are not that much different, and Java has static types which aid refactoring as well as a really strong tooling ecosystem. I haven't worked with large python codebases, but it hasn't really been my experience that refactoring Java is unusually hard, except insofar as average code quality among projects written in Java is probably lower.
  - toolslive 8y ago
    Java IDEs have good support for refactoring since the late 90s: - rename/move class (and autoupdate all references) - change method signature most of it is just stating what you want and the tool does it for you. I don't see that level of support from python IDEs. https://news.ycombinator.com/item?id=9671783 https://news.ycombinator.com/item?id=9671783
- senorjazz 8y ago
  > refactoring a equivalent Java project is way more challenging. I'd argue. Sorry but you are wrong. Just plain wrong. There are benefits for both languages. But to say python is easier to refactor is like saying Java is more concise. It isn't. >Refactoring a Java project takes a whole department and process management crew. Has nothing to do with the language, but the company. A small company will refactor and have an update out same day in Java. An enterprise using some python will be as you say
tom_ 8y ago
It's impossible. Once your program gets past ~1,000 lines, you're stuck with it, basically :( I've always found C++ much easier to work with, if not to actually write.
fabioz 8y ago
Personally, I think that refactoring Python is straightforward (and many refactorings you need to do are much simpler and easier to do without breaking existing APIs -- doing refactorings in Java usually needs changing much more code even for simple things). I think that the real problem is not having a good test coverage, and I agree that bad programs written in Python may be harder to work with than bad programs written in a statically typed language (but in a good codebase, even if large, I'd rather be working with Python). -- although note that in the latest incarnation of Python you can try to use Mypy (http://mypy-lang.org/ http://mypy-lang.org/) along with https://github.com/Instagram/MonkeyType https://github.com/Instagram/MonkeyType or https://github.com/dropbox/pyannotate https://github.com/dropbox/pyannotate to add types to you and go from there if you really miss types too much and this could make a bad codebase bearable (I'm personally waiting for PEP 544 before adding any types though as I'm not a fan of the type system implemented in PEP 484 very much).
- scarface74 8y ago
  Personally, I think that refactoring Python is straightforward (and many refactorings you need to do are much simpler and easier to do without breaking existing APIs -- doing refactorings in Java usually needs changing much more code even for simple things). Refactoring in a non statically typed language cant be as straightforward - many refactors in statically typed language can be automated and guaranteed safe.

qalmakka 8y ago

Like someone else said, I personally find that very large codebases written in dynamically typed languages tend to discourage refactoring. You may try to mitigate the issue by writing the whole application as standalone, independent modules, but we all know that a bit of interdependence always creeps inside even the best designed application. I love Python; that's the first language I've learnt and I can't really live without its REPL. I only think that the industry has a tendency to excessively obsess towards certain languages, even when they are not the best tool for the job. For instance, I strongly believe that the academia's morbid obsession with Java has crippled a whole generation of new programmers, which are now unable to think and code in no other way but the Java™ Way. I won't say that python is as bad as Java in this regard (it isn't even close), but it annoys me to death when it looks like we're heading once again towards a language monoculture; at least this time the language isn't utter crap.

devxpy 8y ago
> looks like we're heading once again towards a language monoculture Isn't that's only true for data science? What about the JavaScript monoculture :( And academia still has courses for C java C++ etc. No respectable University teaches a single language!
- qalmakka 8y ago
  In many Universities, C introductory courses are taught by people that hasn't touched C in at least two decades, or has close to zero experience to how programming modern C is. More often than not, this C is taught by people that have only used Java in years; I've seen horrible stuff, such as C89 still being thought like it's the only standard available, or gets() being used by professors that have no idea whatsoever of what's wrong with them.
  - devxpy 8y ago
    Here in India, 90% of teachers haven't touched what they're teaching, like ever. But you still _have_ to study it because you need to give the exam... Not like it solves anything -- just that there's this side of the argument too.. > I've seen horrible stuff, such as C89 still being thought like it's the only standard available, or gets()... Youre absolutely right -- although one could argue that the curious will have the decency to look this stuff up (like myself) The teaching system is still pretty rekt on the python side of things. Its not like you'll find people who actually code teaching at Universities anyway :( Like now we're just going into how bad the teaching system is. The original argument that python is reaching is a monoculture seems weird to me, when I can't even run the damn thing on a browser..