13 ms·

I agree 100%. I wish every software engineer would spent at least a little time writing some programs in bare C and running them to get a feel for how fast a na

• jcelerier 4y ago
  > I agree 100%. I wish every software engineer would spent at least a little time writing some programs in bare C and running them to get a feel for how fast a native executable can start up and run. It is breathtaking if you're used to running scripting languages and VMs. Conversely when 99.9% of the software you use in your daily life is blazing fast C / C++, having to do anything in other stacks is a complete exercise in frustration, it feels like going back a few decades in time
  • p1esk 4y ago
    Conversely when 99.9% of the software you use in your daily life is user friendly Python, having to do anything in C/C++ is a complete exercise in frustration, it feels like going back a few decades in time
    • bena 4y ago
      I kind of feel both statements. I like writing things in python. It honestly feels like cheating at times. Being able to reduce things down to a list comprehension feels like wizardry. I like having things written in C/C++. Because like every deep magic, there's a cost associated with it.
    • bayindirh 4y ago
      As a person who uses both languages for various needs, I disagree. Things which takes minutes in optimized C++ will probably take days in Python, even if I use the "accelerated" libraries for matrix operations and other math I implement in C++. Lastly, people think C++ is not user friendly. No, it certainly is. It needs being careful, yes, but a lot of things can be done in less lines then people expect.
      • l33t2328 4y ago
        > Things which takes minutes in optimized C++ will probably take days in Python, even if I use the "accelerated" libraries for matrix operations and other math I’m gonna need an example because I do not believe this whatsoever.
        • bayindirh 4y ago
          I'd rather open the code and show what I'm talking about, however I can not. Let's say I'm making a lot of numerical calculations which are fed from a lockless queue with atomic operations to any number of cores you want, where your performance is limited by the CPU cores' FPU performance and the memory bandwidth (in terms of both transfer speed and queries that bus can handle per second). As I noted below, that code can complete 1.7 million complete evaluations per core, per second on older (2014 level) hardware, until your memory controller congests with all the requests. I need to run benchmarks on a newer set of hardware to get new numbers, however I seriously lack the time today to do so and provide you new numbers.
          • necovek 4y ago
  • tester756 4y ago
    Is performance inversely proportional to dev experience? because what you wrote could be said about using C++ in the context of dev experience 10 compilers, IDEs, debuggers, package managers and at the end of the day LLVM compiles 30min and uses tens of GBs of RAM on average hardware I don't believe that this is the best we can get.
    • bhauer 4y ago
      > Is performance inversely proportional to dev experience? No. I feel there is great developer experience in many high performance languages: Java, C#, Rust, Go, etc. In fact, for my personal tastes, I find these languages more ergonomic than many popular dynamic languages. Though I will admit that one thing that I find ergonomic is a language that lifts the performance headroom above my head so that I'm not constantly bumping my head on the ceiling.
    • pjvsvsrtrxc 4y ago
      Huh? "10 compilers, IDEs, debuggers, package managers" what are you talking about? (Virtually) No one uses ten different tools to build one application. I don't even know of any C++-specific package managers, although I do know of language-specific package managers for... oh, right, most scripting languages. And an IDE includes a compiler and a debugger, that's what makes it an IDE instead of a text editor. "and at the end of the day LLVM compiles 30min and uses tens of GBs of RAM on average hardware" sure, if you're compiling something enormous and bloated... I'm not sure why you think that's an argument against debloating?
      • throwaway894345 4y ago
        I assume the parent was talking about the fragmentation in the ecosystem (fair point, especially regarding package management landscape and build tooling), but it's unclear.
      • tester756 4y ago
        >No one uses ten different tools to build one application. I meant you have a lot of choices to make Instead of having one strong standard which everyone uses, you have X of them which makes changing projects/companies harder, but for solid reason? I don't know. >"and at the end of the day LLVM compiles 30min and uses tens of GBs of RAM on average hardware" sure, if you're compiling something enormous and bloated... I'm not sure why you think that's an argument against debloating? I know that lines in repo aren't great way to compare those things, but .NET Compiler Infrastructure: 20 587 028 lines of code in 17 440 files LLVM: 45 673 398 lines of code in 116 784 files The first one I built (restore+build) in 6mins and it used around 6-7GB of RAM The second I'm not even trying because the last time I tried doing it on Windows it BSODed after using _whole_ ram (16GBs)
• michaelchisari 4y ago
  I wish product designers took performance into consideration when they designed applications. Engineers can optimize until their fingers fall off, but if the application isn't designed with efficiency in mind (and willing to make trade-offs in order to achieve that), we'll probably just end up right back in the same place. And a product which is designed inefficiently where the engineer has figured out clever ways to get it to be more performant is most likely a product that is more complicated under the hood than it would be if performance were a design goal in the first place.
• zozbot234 4y ago
  Please don't write programs in bare C. Use Go if you're looking for something very simple and fast-enough for most uses; it's even memory safe as long as you avoid shared-state concurrency.
  • bb88 4y ago
    Please don't write programs in go. Sure it looks awesome on the surface but it's a nightmare when you get a null pointer panic in a 3rd party library. Instead use Rust. See here for more info: https://getstream.io/blog/fixing-the-billion-dollar-mistake-in-go-by-borrowing-from-rust/ https://getstream.io/blog/fixing-the-billion-dollar-mistake-...
    • tomcam 4y ago
      Or, you know, whatever the fuck language you care to use.
    • michaelsshaw 4y ago
      Nothing wrong with any of these languages, especially C. It's been around since the early 70s and is not going anywhere. There's a very good reason it (and to an extent C++) is still is the default language for doing a lot of things since everyone understands it.
      • KronisLV 4y ago
        C and C++ both have excellent library support, perhaps the best interop of any language out there and platform support that cannot be beat. That said, they're also challenging to use for the "average" (median) developer who'd end up creating code that is error-prone and would probably have memory leaks sooner or later. Thus, unless you have a good reason (of which, admittedly, there are plenty) to use C or C++, something that holds your hand a bit more might be a reasonable choice for many people out there. Go is a decent choice, because of a fairly shallow learning curve and not too much complexity, while having good library support and decent platform support. Rust is a safer choice, but at the expense of needing to spend a non-insignificant amount of time learning the language, even though the compiler is pretty good at being helpful too.
• throwaway894345 4y ago
  Yeah, I've written static site generators in Go and Rust among other languages (it's my goto project for learning a new language). Neither needed incremental builds because they build instantly. The bottlenecks are I/O. I've also worked in Python shops for the entirety of my career. There are a lot of Python programmers who don't have experience with and thus can't quite believe how much faster many other languages are (100X-1000X sounds fast in the abstract, but it's really, really fast). I've seen engineering months spent trying to get a CPU-bound endpoint to finish reliably in under 60s (yes, we tried all of the "rewrite the hot path in X" things), while a naive Go implementation completed in hundreds of milliseconds. Starting a project in Python is a great way to paint yourself into a corner (unless you have 100% certainty that Python [and "rewrite hot path in X"] can handle every performance requirement your project will ever have). Yeah, 3.11 is going to get a bit faster, but other languages are 100-1000X faster--too little, too late.
  • necovek 4y ago
    Python is slow in many things like pure looping and arithmetic, even though there are workarounds to make that 1-10x slower rather than 100-1000X (eg. C-based implementations, including all the itertools stuff). I am sometimes frustrated that I can't just loop over a string character by character and not get crappy performance, but the "problem" you (and me) are seeing in existing codebases is that Python is very inviting to beginners, and they are not frustrated with this because they don't know it :) But as you note, bottleneck is the I/O, and a program waiting for I/O in Python and I/O in C will wait the same time after the computation is done. If you are writing software that can parallelize well independently (eg. web apps) and your memory pressure is not the most important thing, you simply run multiple Python processes to max out the CPU (this avoids the GIL unlike async Python). And you keep your dependencies low.
    • throwaway894345 4y ago
      > even though there are workarounds to make that 1-10x slower rather than 100-1000X (eg. C-based implementations, including all the itertools stuff). These only apply for specific problems, and very few applications are purely CSV parsing or purely matrix math operations. In the real world, you often spend more time marshaling your Python data to C than you save by doing your computation in C. > But as you note, bottleneck is the I/O, and a program waiting for I/O in Python and I/O in C will wait the same time after the computation is done. The bottleneck in a static site generator is I/O. The fact that Python, Ruby, etc based implementations take tens of seconds or more while Go and Rust finish instantly for an I/O bound problem is pretty damning. > If you are writing software that can parallelize well independently (eg. web apps) and your memory pressure is not the most important thing, you simply run multiple Python processes to max out the CPU (this avoids the GIL unlike async Python). The goal isn’t to saturate the CPU as much as it is to complete requests in a timely fashion. If it’s just some light translation between HTTP and database layers, Python is fine, but if you have to do anything computationally significant at all, it can range from “a huge pain” to “virtually impossible”. I gave the example earlier of a web service that was struggling to complete requests in even 60s (despite using Numpy under the hood where possible) while a naive Go implementation completed in hundreds of ms.
• latenightcoding 4y ago
  off topic but I initially didn't notice your username but the second I read "I wrote my own template language in Dart" I knew who it was.
  • munificent 4y ago
    Haha, yes, I definitely realized I was doing some extremely on-brand yak shaving when I did it.
• kllrnohj 4y ago
  Rather than bare C something like C++, Rust, or even Haskell would be better. C isn't the fastest, especially not with normal code. C++ templates get a bad rep, but if you want to go fast they are extremely hard to beat. Also those languages show you don't actually have to give up modern features or even that much convenience in order to get blazing fast speeds.
• fullstackchris 4y ago
  While I'll take a bite at this, I think it's also fair to say how poorly portable C is. Can an mobile or web engineer quickly take some C code and use it in their stack somehow? I would guess not. While it's indeed an important lesson to see the speed of some of these 'close to the metal' languages, the question of how practical they are to use is a different question.
  • loup-vaillant 4y ago
    There is a class of C code that can be made extremely portable: pure computations. This allows you to write self contained code with zero dependencies, and if you're willing to give up on SIMD you can stick to fully conforming C99. It's not applicable for everything, but we do have some niches where it comes in handy: cryptographic libraries (I've written one), parsers and encoders of all kind, compilers… For instance can a mobile on web engineer quickly take TweetNaCl or Monocypher and use it in their stack? Yes. They may need to write some bindings themselves, but if they can run C code at all it's fairly trivial.
• nyanpasu64 4y ago
  I wish C++ compilers written in C++ were blazing fast too.
  • kllrnohj 4y ago
    They are. They've just chosen to spend all their speed gains on more optimization passes and static analysis, to produce ever faster outputs than to produce an output faster.
    • wruza 4y ago
      Their fundamental model is one translation unit per time, while developers decided that writing all library code in headers is a good idea. Which makes them parse and DCE literally kilometers of mostly irrelevant code again and again. You’re not wrong, but it’s not the complete point. C++ development is slow as a whole, and compilers/standards do nothing to fix that. It’s a kind of F1 engine in a tractor situation.
      • gpderetta 4y ago
        > and compilers/standards do nothing to fix that C++20 finally standardized modules. Whether they will improve things significantly is still anyone guess.
      • munificent 4y ago
        > while developers decided that writing all library code in headers is a good idea. It wasn't developers who designed C++'s template model which requires generic code to be fully defined in header files. Inheriting C's textual include file based "module" system and then bolting compile-time specialized generics is a choice the C++ committee made, not C++ users. It was probably the right choice given C++'s many very difficult constraints, but that's what directly leads to huge compile times, not dumb C++ users.
• Aeolun 4y ago
  This was about my experience switching from webpack to ESBuild for Javascript. Why do incremental builds if rebuilding the whole thing takes just 2s (as opposed to 90+ with webpack).
• necovek 4y ago
  In a sense, knowing this can also hurt you. At all my recent jobs, I grow frustrated with how slow running a single unit test is locally on a codebase. We are talking 5+ seconds for even the most trivial of trivial unit tests (say, purely functional arithmetic unit test). And this is even with dynamic languages like Python (you see pytest reporting how your unit test completed in 0.00s, and wall time is 7s). And then I get grumpy if they don't let me go and fix it because I am the only one who is that annoyed with this :D
  • hinkley 4y ago
    How on earth are you getting 5 seconds for simple tests? Simple tests should be running in 8ms, and those are my 2015 numbers that I've been too lazy to update.
    • necovek 4y ago
      Have you worked on a recent idiomatic development setup (dockerised local development, top level imports of everything and plenty of setup at the top level too, people unfamiliar with how to manage .pyc files so they simply disable them...)? Common libraries like requests or sqlalchemy take 300-500ms to import (eg. try `time python3 -c 'import requests'` and contrast just `time python3 -c ''` which is python startup overhead). As I said, tests run in sub 10ms, but from issuing pytest to completion it's usually 5-15s.
      • hinkley 4y ago
        Ah, I see, so the setup time is very slow. I don't work in python much but I've worked in a few other languages with slow startup, and amortization is your friend. It's hard though when you have a small module with 'only' 300 tests and your test is 6ms of code that works out to 40ms once setup and teardown are included. I haven't had many opportunities to have the "well maybe you should be making bigger modules" conversation but I am ready for that moment to arise. This is usually the point at which I pull out a 'watch' implementation, since the 5 seconds it's going to take me to switch windows and hit 'up' the right number of times counts too, if we're comparing apples to apples. That said, one of the last times I had a unit testing mentor, I walked into a project that ran 3800 tests in about 7 seconds, and then started poking around trying to figure out who was materially responsible. (He didn't know much more than me from an implementation standpoint, but boy was he good at selling people on test quality.) If that had been 20 seconds it would have still been lovely, but it wouldn't have grabbed my attention quite as much.