9 ms·

The hardest part of debugging Python is "hitting the wall" when you come to a native library (compiled C code). And Python has achieved a lot of speed-ups from

• PhilipRoman 3y ago
  Are python debuggers really unable to integrate with something like GDB? I have no problem debugging native calls from Java. During development I've never seen a case where library source was unavailable. Even proprietary components from other companies come with source included.
  • ben-schaaf 3y ago
    GDB on its own does a reasonable job debugging python: https://wiki.python.org/moin/DebuggingWithGdb https://wiki.python.org/moin/DebuggingWithGdb
  • uranusjr 3y ago
    I feel it’s less about instrumentation but more about Python developers understanding C code.
    • throwaway2037 3y ago
      No, as a "fluent C speaker", the instrumentation part is much harder than understanding the C code.
• elnatro 3y ago
  Do you think that relying less in C code could be better for Python and it’s community? I mean less focus on being “glue” between C code and more focus con optimizing the interpreter?
  • generichuman 3y ago
    It is unlikely people would use Python if it didn't rely on C code. If your ML model takes hours when it can take minutes, or takes days when it can take hours, you will move away. You could move away to another language or a faster interpreter but that's a different discussion. > more focus con optimizing the interpreter This is good but there's an upper bound to performance of interpreted languages. Maybe the Python interpreter could be as fast as V8, but it is unlikely to be fast as JVM. People will need to drop down to C / Fortran for whatever compute intensive work they're doing.
    • robertlagrant 3y ago
      > If your ML model takes hours when it can take minutes, or takes days when it can take hours, you will move away. You could move away to another language or a faster interpreter but that's a different discussion. How much of this is done in Python, vs constructing instructions in Python to run on a GPU at uber speed?
      • generichuman 3y ago
        Widely used libraries for python such as SciPy, PyTorch, TensorFlow, Numpy all drop down to lower level languages (C/C++/Fortran). I haven't seen many people doing ML in Python without one of these libraries, so I'd say it is mostly constructing instructions in Python and offloading the actual intensive work to these libraries (which may or may not run on GPU depending on your hardware).
  • wheelerof4te 3y ago
    If you asked me, Python not only shouldn't rely on (third-party) C library code, but it shouldn't even be used for cases where pure Python is not optimal.
• deleted 3y ago
  [deleted]
• criley2 3y ago
  On the other hand, you still have to learn and use Java. I think learning Python and a touch of C is easier than trying to learn Java. Heck I'd go so far as to say C is a lot easier than Java. Frankly I don't understand why having to learn a little bit of C to debug a python program is such a red flag or wall to you, considering the sheer amount of learning one has to do to use Java at all. You're setting up an uber-sophisticated IDE to debug "ancient auth packages" instead of just... learning a little C and potentially fixing an up-to-date and beloved library?
  • post-it 3y ago
    > You're setting up an uber-sophisticated IDE You just install it. > learning a little C and potentially fixing an up-to-date and beloved library A romantic thought, but 99% of the time I'm just going to do a workaround or a local patch. I don't use Java anymore, but I don't hate it. I think it has some verbose conventions, but I vastly prefer it to C's extremely terse conventions. Nowadays I try to do as much in TypeScript as I can, because I find it a pleasure to use, and it has the same property where you can dive into any lib when debugging.
    • throwaway2037 3y ago
      Monkey patching in-house proprietary libraries without source code is also such a nice feature of Java. (Can this be done with C#? I assume yes.)
    • throwaway2037 3y ago
      uber-sophisticated IDE I love this. As if any developer is not much more productive with a 4GB+ RAM IDE churning away at their code base and suggesting all sorts of things as you write code. OMG: See ClangTidy! All C++ IDEs these days are either directly incorpating ClangTidy, or copying its features. When I use CLion, as a medium-level C++ programmer, it is scary how good are the suggestions from ClangTidy!
  • throwaway2037 3y ago
    This post assumes that I don't already know C. I do. The real problem is friction. Being able to debug from Python code into C code is super hard, even in 2023. If you are in the same language (and debug session), the friction is so much lower.
• zigzag312 3y ago
  Likewise, in C#, almost everything is written in pure C#. It also gives you more control over memory allocations than Java, which is nice when trying to squeeze more performance out of the code.
  • heywhatupboys 3y ago
    if you need speed ups like that, don't program in either. I strongly doubt any production C# code is significantly faster than its JAVA counterpart.
    • VBprogrammer 3y ago
      When it comes to languages you can draw up a loose hierarchy of potential performance where languages like C# and Java are vastly equivalent. Attempting to refine it any further descends into a bikeshedding farce.
      • WhereIsTheTruth 3y ago
        Java is faster than C#, that's not "bike-shedding farce", that's a documented, and benchmarked fact JSon serialization: 20+% faster: https://www.techempower.com/benchmarks/#section=data-r21&test=json https://www.techempower.com/benchmarks/#section=data-r21&tes... Single query: 20+% faster: https://www.techempower.com/benchmarks/#section=data-r21&test=db https://www.techempower.com/benchmarks/#section=data-r21&tes... Multiple query: 20+% faster: https://www.techempower.com/benchmarks/#section=data-r21&test=query https://www.techempower.com/benchmarks/#section=data-r21&tes... Data update: 15% faster: https://www.techempower.com/benchmarks/#section=data-r21&test=update https://www.techempower.com/benchmarks/#section=data-r21&tes...
        • throwaway2037 3y ago
          Why is this being downvoted? Sure the opening line isn't the most gentle, but these are some crazy benchmarks. I never saw this website before. First link: How is it possible that Java can achieve same perf as C??? I write a lot of Java, and none of my stuff is close to C. I guess about 50% the speed -- which fine for my needs. One reason (totally unscientific, of course) I think Java is faster: The virtual machine has been open source for longer. So more academics have looked at, run experiments, written papers... that are then read by the core Java team and sometimes implemented. C# is a bit behind, but should catch-up one day.
• spacephysics 3y ago
  Mojo seems like a very promising solution, despite it being AI oriented. Lex Friedman had a great podcast with the founder, its first version was released about a month ago, and in two weeks got 10k discord users. A superset of python that, once finished, can run all native python code, but also adds language features to get closer to the “metal”, getting speeds of C libraries with the ability to properly debug and step through it. If this project goes the way it’s promised (the guy has great experience with language development), this could further cement python as it would solve its biggest criticisms. Very excited for it https://youtu.be/pdJQ8iVTwj8 https://youtu.be/pdJQ8iVTwj8 https://www.modular.com/mojo https://www.modular.com/mojo https://en.m.wikipedia.org/wiki/Chris_Lattner https://en.m.wikipedia.org/wiki/Chris_Lattner
  • throwaway2037 3y ago
    Crikey: LattBot strikes again! This guy's footprint on compilers and languages in one generation cannot be understated. His PhD work eventually grew into LLVM and Clang which lit a fire under GCC and MSVC. The C and C++ compiler ecosystem is now far more competitive and healthy. He has the magic touch. What you describe sounds amazing. And having LattBot behind it is even better. I have watched one of his episodes with Lex before. He is absurdly humble about his accomplishments.
• mikece 3y ago
  > The hardest part of debugging Python is "hitting the wall" when you come to a native library (compiled C code). Aren’t most of these native libraries open source? I’m a C# dev so maybe this is a naive question based on my experience but is there not a way to bring in the source of the C library and debug into it in these “hit the wall” situations?
  • tkuraku 3y ago
    You can, but then you have to setup and build the libraries in debug mode. You can then debug with gdb or visual studio. It is just a lot more work than debugging pure python.
• Qem 3y ago
  > The hardest part of debugging Python is "hitting the wall" when you come to a native library (compiled C code). This is one big reason I love Pharo and other Smalltalk-language implementations. Being mostly written in themselves down to the VM. You can take a deep dive and inspect everything without being afraid of smashing your head against C bedrock underneath. And they still manage doing this while being reasonably performant and dynamic.
• westurner 3y ago
  > The hardest part of debugging Python is "hitting the wall" when you come to a native library (compiled C code). And Python has achieved a lot of speed-ups from Py2 to Py3 by adding more compiled C code. "Debugging a Mixed Python and C Language Stack" (2023) https://news.ycombinator.com/item?id=35710350 https://news.ycombinator.com/item?id=35710350