6 ms·
Ask HN: Is low level progarmming job more satisfying?
I think general assumption is that a high level programming language/tool:
1. protects you from many technical dangers;
2. empowers you to relatively easy implement complex business logic;
But does it also strike back with these downsides:
1. limits your ability to create "novel" or technically non-trivial systems;
2. moves you closer to the business and thus forces you to deal with many levels of management that influence not only "what" you program, but sometimes also "how" you do it;
Having enough experience with high-level stuff I have this stupid idea that going closer to the OS/metal and wisely choosing tools can somehow protect an engineer from politics and provide an environment with more grunted, real, technical constraints.
Anyone did such a switch? Please, let me know how this corresponds with your experience.
- CogitoCogito 6y agoI think it all depends on what you're interested in. I don't enjoy programming that I feel is "overly abstract" (of course this term has no meaning outside of my own opinionated viewpoint of software engineering). I also enjoy ripping apart the layers of systems to understand the constituent parts. So for me low-level programming is more satisfying, but I'm sure there are people who feel exactly opposite for the same reason. Different folks different strokes.
- ineedasername 6y agoforces you to deal with many levels of management that influence not only "what" you program, but sometimes also "how" you do it" That's because there's a lot more choice when you're working at a higher level. If you're working with an embedded system with an AVR, you're choices may be C/C++ and Assembly-- No need to get into an argument over the latest shiny framework. You'll never really be isolated from office politics or poor management though. Things like feature creep or changing specs etc. also won't go away.
- contingencies 6y agoNothing protects you from management except running your own company, and generally the further you are from business the less value you can obtain. The best you can hope for as an employee is a good manager to shield you from hassle. I have friends who stuck with low level work and they are either aging in security jobs that ceased to stimulate them or relegated to a shifting sands array of temporary employers with minimal financial security.
- nicoburns 6y ago> Nothing protects you from management except running your own company And then you're at the mercy of your customers!
- adzicg 6y ago> And then you're at the mercy of your customers! Not necessarily. With consumer-facing products, especially if they're mass market and the price of a single subscription is low, the customers can't make any big demands (in fact, they can, but you can safely ignore them). I'm part of a very small team that has a reasonably successful SaaS product aimed directly at consumers. We intentionally priced it very low so we can stay in control of the product, and not be at the mercy of our customers.
- nicoburns 6y agoLow level programming can definitely be satisfying, but don't expect it to shield you from politics.
- monocasa 6y agoI'm someone who went from high-level to low-level. Software guy that does firmware, HDL, disassembly RE, some board design and layout, and have done a he high level, react frontend connected to a modern distributed backend with all the kubernetea and what have you I enjoy the low-level work more, but that's more of a personal preference. You can for sure solve high-level problems easier, being below it on the abstraction hierarchy, but your work just comes with new problems. I have never beat my head harder against the wall the trying to figure out something that was a chip bug that eventually becomes new errata (if you're lucky enough for the chip vendor to answer your emails in the first placeti get the errata reported). It doesn't really protect you from politics unless you're the only low-level person in your shop and no one understands what you do, in which case you just stagnate instead. If you're not careful you tend to get paid less too since high-level folk tend to be closer to the flow of money. I like the work, and wouldn't trade it, but it's not a panacea to engineering org issues in your career.
- rhn_mk1 6y agoOften programming in both realms, I find that the low level gets stuck on the "how" part, and if we're talking about technically non-trivial systems, that can drain you before you even start to approach the non-trivial problem you wanted to solve in the first place. The best way to code IMO is to find a way that allows doing low level coding but includes the possibility to jump into the high level whenever needed.
- bartvk 6y agoI did this. But after the learning phase, it wasn't more satisfying. Turns out it's the learning process itself, that makes the work joyful for me.
- Ma8ee 6y agoThat’s my major problem. After about two years of whatever job I have I feel I don’t learn anything interesting any more and get bored.
- bluefirebrand 6y agoThis is a realization I had recently and I have started to consider job hopping more actively once I'm bored, or maybe just moving to shorter term contracting as a rule.
- amelius 6y agoLow-level plumbing can become boring quite easily, though.
- the_only_law 6y agoYeah, honestly, I looked into trying to go into embedded software for a little bit and after a while I kinda figured that a majority of the work would end up just as dull and dissatisfying. At this point I'm just hoping I can one day find a job that will allow me to work on something I find interesting, no matter what level I'm working at.
- anonuser123456 6y agoThere is little difference in my experience. The question is what problem are you solving, and do you have the correct tool for the job? It is frustrating to solve problems with the wrong tool. It is a joy when the tool fits the job. Need to write a web server? Not super fun in C. Need to access raw memory and machine intrinsics directly? Not fun in python.
- pjc50 6y agoThe last time I was working on a web server it was in C++, and could be absurdly performant up to the limits of the operating system. But that was ZWS, before Linux started shipping its own kernel webserver and before the final death of commercial UNIX distributions. https://en.wikipedia.org/wiki/Zeus_Web_Server https://en.wikipedia.org/wiki/Zeus_Web_Server
- Leherenn 6y agoI don't know if the technical constraints are more real, just different. When you start interacting with hardware, a lot of easy stuff can become more difficult (testing, deployment, debugging, ...) and some difficult stuff becomes easier (number of platforms to support, control over the product, scope of the project, ...). Everything is generally slower when you have to interact with hardware. Politics is still there as others have talked about, and the complaints about the tools become more "do we really have to code in ANSI C with a proprietary toolchain?". I'm not saying don't go for it, it's a really nice domain and IMO there's a satisfaction that's unmatched by pure software when you see your hardware project comes to fruition; but don't expect the paradise.
- cjbprime 6y agoI went from being a kernel engineer to a JavaScript application engineer, and JS was more satisfying to do over a long period of time for me -- the "recompile, reboot, retest" loop taking several minutes each time for testing kernel changes that can't be tested in a kernel module was extremely frustrating, compared to the amazing browser devtools for modern JS.
- jjjeii3 6y agoLucky, you don't work with Angular. Compiling a tiny TODO-app is slower than compiling the entire Linux kernel: https://github.com/angular/angular/issues/37293 https://github.com/angular/angular/issues/37293 I really starting hating Angular due to performance issues.
- almost_usual 6y agoAlso feel that pain linking large projects in Rust.
- mike_k 6y agoWow, these are two quite different jobs. What about the the differences in project management, planning and a sense of accomplishment though? (Sure, it depends on the other factors too.)
- cjbprime 6y agoThere was a difference, but it was smaller than the job satisfaction increase from better tooling for me.
- the8472 6y ago> the "recompile, reboot, retest" loop taking several minutes Surely full-time kernel developers must have tons of shortcuts that can speed this up compared to the regular build process, no? There just was an article about booting a VM in hundreds of milliseconds on the frontpage for example.
- cjbprime 6y ago
- aWidebrant 6y agoIn terms of the OSI model, I've gone from working with software on top of layer 7 to working on software and hardware at the bottom of layer 1. It's been a fun and challenging transition, but my job satisfaction has mostly been determined by the people I work with rather than the subject matter. Specifically to your points, I haven't really seen any major difference in the ratio between routine and creative programming at the high and at the low level. The amount of politics and business requirements has also been quite similar.
- Geminidog 6y agoI just did the switch from 10 years of python to C++. Low level programming is way harder with the C++ stack. I blame mostly the language rather then the level. C++ is just a pain in the ass. If you get a low level programming job that uses rust things might be better. Basically at the higher level, when you hit an error it is (generally) much more trivial to solve... not so with C++. With C++ you can hit crashes and not know where it even came from, this is much more rare in higher level languages. I know several people who vehemently hate C++ with a passion and if you're going into low level stuff, very likely you'll have to deal with C++.
- nicoburns 6y ago> I blame mostly the language rather then the level. C++ is just a pain in the ass... With C++ you can hit crashes and not know where it even came from, this is much more rare in higher level languages. Yep, it's definitely the language. You can do low-level programming with Rust and pretty much never have this problem (never say never, but it's certainly not a common issue).
- Geminidog 6y agoUnfortunate that as of right now, most low level stuff is basically C++.
- microtherion 6y ago> With C++ you can hit crashes and not know where it even came from As somebody who writes more C++ than Python, I have pretty much the opposite perspective: If I hit a reproducible crash in C++, I can drop into a debugger, and work out where it came from. If necessary, I can look at the disassembled code, because it's reasonably feasible to map assembly language to C++ constructs. Python code generally doesn't crash that way, it's true (unless it interfaces with C modules). But when errors occur, I have to debug them on a Python level. Dropping down the stack is nearly hopeless because the Python interpreter is just too thick an abstraction layer between assembly language and the Python code. And I find myself having similar problems with Swift, for all its elegance. The mapping into assembly language is just considerably more complex than in C++, so you lose a low level debugging tool.
- foobiekr 6y agoEmbedded is a lot of fun. A lot of fun. It's "real coding" to me, though I haven't done any in fifteen years. But .. embedded vendors can suck. A lot. Terrible docs, epic levels of errata (e.g., a PowerPC vendor who, at one time, shipped a bigger errata document than the entire manual set for an x86 of the same era), and just.. marginal everything. In some cases, they are really interesting - the Broadcom Fastpath SDK was actually a good mix of excellent and terrible. But the SDKs can be absolute trash even from name-brand vendors. Without naming names, I'll give an example: SDK contains a linux distribution from an CPU ASIC company shipped as a tar file. You have no indication whatsoever what the reference point for the linux kernel is, no idea what patches it has or does not have, and so on. When you ask them, they don't even understand why you care. Get ready for the fun of massive diffing. The network stacks on the really low-end can be shockingly bad. From a career standpoint, embedded is a bit of a niche. You will probably always have a job - especially in the IoT era, but even without that. It is typically a somewhat solitary role - you do the code for the XXX block (if drivers) or for some set if functions (IoT-style embedded), but you don't end up doing "broaden your scope" things as much - no system-level nor distributed systems work, which can be somewhat limiting.
- cjbprime 6y ago> It's "real coding" to me, though I haven't done any in fifteen years. For what it's worth, as someone who's done both (Linux kernel maintainer and web developer), this sentiment reads as a kind of chauvinism to me -- the implication that higher-level work isn't real coding. If I had to choose which was more difficult, I'll probably go with web development: the kernel simply had much less that you had to know to be effective, compared to the massive stack of protocols and frameworks that even a simple web app runs on top of and demands competency with.
- erickj 6y agoOP made a judgement call and ranked his/her preferences. So be it. I understand the sentiment. Comparing work I do, the higher up I go the less it can feel like coding. Javascript, Python, and Ruby frameworks at times can feel like a paint by numbers puzzle - and I don't mean that to be complimentary under any interpretation. That doesn't make good web development "easy", nor does it diminish the value of good ideas and fast product development.
- alkonaut 6y agoI want to be in some sort of sweetspot. I don't want to do anything where drivers or hardware keeps sucking the life out of me. I also don't want to do anything where I need to read a dozen spells into some cloud shell in order to get a hello world from an app running on sixteen different machines. I just want to code algorithms and data structures and solve hard domain problems. Not solve difficult programming and operations problems in order to deliver on what is often (at least superficially!) simple domain problems. I think that sweet spot is in games and desktop development.
- dvtrn 6y agoHow can I reach out to you? I don't have a job offer, I just wanna be nearby when you find this job so I can come with ;) I don't want to do anything where drivers or hardware keeps sucking the life out of me. I also don't want to do anything where I need to read a dozen spells into some cloud shell in order to get a hello world from an app running on sixteen different machines. This sang to me, friend. Someone suggested consulting once, to me, and I see it quite a bit in forums where the topic of engineering malaise is brought up. Problem for me is that puts me a few layers closer to the business types and represents the exact opposite direction I want to go in terms of day to day interactions. Not that I think they're bad folks, I understand they have a part to play in the business same as I do; it's just there's a massive misalignment between "the business" and myself as an engineer in terms of...I guess comparative levels of tolerance for bullshit? Something, ain't it? Been doing this for 22 years, management for the last 8 and I still can't find a better way to put that.
- alkonaut 6y agoThis (I assume) is most jobs in “technical” desktop software (cad, sound editing, automation...) and game tech. I had a job like this my whole career. I’m describing the area I want to stay in :)
- CodeGlitch 6y agoI spent 11 years in the games industry - there is certainly something great about working within the confines of a games console. You have a set amount of memory and processing power, and that's it. It's a shame the industry itself sucks. Take a look at EA-spouse and the latest crunch for Cyberpunk - it sucks all enthusiasm out of you.
- CodeGlitch 6y agoHigh-level does have the advantage that your work is more easily understood by non-technical / management types - and that could have a positive effect on promotions. Let's say you glue a bunch of AWS services together to solve a business problem: in contrast to solving some technical hurdle to do with obscure hardware. The former will definitely be easily understood by more people, and I hazard to say actually has a large impact on a business (in most cases?) Low-level programming does scratch that itch of "hacking" stuff, although I have had satisfaction with glueing stuff together with Python in interesting ways.
- tinktank 6y agoI do it for a living, IMO, no. We spend a lot of our time just putting plumbing in place, not much time left for creativity.
- thih9 6y agoCould you elaborate? What do you program, what language do you use, what exactly do you mean by putting plumbing? (making sure libraries talk to each other? something else?)
- whateveracct 6y agoas a CompE by training, i'll say that people fetishize low-level programming in general, everything's exactly the same imo so if you're interested in the field, go for it
- BossingAround 6y agoI've heard the same from multiple people focusing on the low level layers.
- djmips 6y agoI sometimes wonder if these people never actually worked in high level programming and just 'think' it's probably the same.
- whateveracct 6y agoi pretty much exclusively work in high level languages - it is the same. people just think it's different because they like being "closer to the computer" which isn't a real thing but instead a mental block. i feel plenty close to the computer pretty much constantly no matter my level of abstraction.
- pjc50 6y agoSome versions of my CV say "full stack developer from the transistors upwards", given that at one time or another I've done everything from chip design to a webserver to PIC16 assembly to actual frontend web design (although a long time before React). Software is, by and large, software. The things that can make or break satisfaction are to me: - coworkers, clients, and the political environment. This is completely independent of the technology and project requirements. As a consultant I did very different projects with the same co-workers. - iteration and debuggability. The best for this, for me, is C# in Visual Studio. GDB is an acceptable substitute for the C family. Web applications .. the browser is OK, but now your app is always and forever in two pieces, and you can't cross-stacktrace. Many embedded environments are bad at this; you may find breakpoint debugging and printf debugging are both impossible. Or you have to do an Elon Musk and recreate the crash cause from the smoking debris field. - fundamentally interesting problem and ownership of work. This can be tricky, but when you get to do things your way and produce great results that can be very satisfying. Again, completely independent of language; depends what impresses you and your circle. Do you want something you can talk about at parties? I met the last surviving maintainer of the Aston Martin Lagonda's bonkers 1980s CRT-based 6V digital logic dashboard. I have only a very few times got to choose the language for a new project, or indeed start a new greenfield project at all.
- almost_usual 6y ago> OS/metal and wisely choosing tools can somehow protect an engineer from politics and provide an environment with more grunted, real, technical constraints. Lots of “that’s a good idea but it’s impossible to predict all the side effects of that change so no.” You’ll encounter managers who think you should be shipping at the velocity of a high level programmer. Overall I enjoy it more but the politics don’t go away.
- pkaye 6y agoIts fun but there it sucks is 1. you have the frustration of working around hardware issues and 2. some resource restricted products you spend a lot of time just squeezing things in instead of just writing up the code.
- OnlyOneCannolo 6y agoI disagree with all 4 of your assumptions. Usually you're working on different types of problems. The technical dangers are different, the type of business logic you're implementing is different, and the types of new things you'd create are different. There's also a huge ecosystem of tools and techniques that you probably haven't even heard about. If you're just going to make a CRUD app in C or something, odds are you're going to experience the drawbacks you mentioned without much extra benefit. But if you work on actual low-level problems, it can be a completely different world where programming isn't even half the story.
- mike_k 6y agoI do aknowledge that everything will be different ) (There was no attempt to compare different languages or their "abstractness" levels and to apply them to the same class of problems. Of course, that almost never the dilemma people are facing.)
- qwertywert_ 6y agoThis is the major dilemma of my career right now... move close or farther away from embedded.. 1. Tools are shit sometimes 2. fixing bugs no-one else understands (and so you don't receive proper praise - because manager might not even understand what you did) 3. Literally spending weeks on insane hardware issues, that requires sifting through 1000 page documentation. 4. harder work - less pay (depending on company) The major positive is it's extremely satisfying when you do fix the things..
- chme 6y ago"Low-level" is, like all areas of computer sciences, pretty huge and diverse so your experience will depend on what machine you are programming for. FPGAs are fun, because you can design logic circuits and hardware, but with some exception you will have to deal with huge proprietary toolchains and closed source 'compilers'. It offers a pretty unique way to solve problems, which is fun to do. Like playing with a tiny but powerful electrical circuit toolbox. With micro controllers you might have to deal with proprietary compilers, libraries and IDEs but to a lesser amount than with FPGAs. Still fun to get something to blink, move or a display to work. If you work on micro processors, then you could work on bootloaders, where you more or less concern yourself just to get something working enough to boot another system, or with OS kernels, where you work in a bigger system infrastructure in order to allow as much of the hardware as possible to be available for the user-land by writing drivers. Its challenging because you work on a small gear in the middle of a huge clockwork, that is mostly not written by yourself, and you will struggle to understand. There are also many special processors you could implement stuff for like DSPs. There you will feel the language limitations but they are very interesting and odd beasts to work with. Low-Level is mostly about reading hardware and often software documentation (and finding errors in those or the hardware itself) to figure out how to do certain things, and then if something doesn't work, printing out registers to figure out which bit is wrong. Or having to use oscilloscopes or logic analyzers to debug. This certainly isn't everyones cup of tea, but still very rewarding when you actually start to understand how the pieces fit together and things work.
- NationalPark 6y agoA lot of people are trying their best here to make CRUD work sound interesting, but the answer to your question is basically yeah, it's more fun to work on lower level stuff, because it's more technically challenging and you get to employ more theoretical concepts. Unfortunately it's also more political, because the stakes are higher (bigger impact) and for whatever reason more difficult (but talented) personalities seem to get a pass in that world compared to application development work.
- psyc 6y agoHow low level? I don’t know how to design silicon or write in hardware languages. I don’t fully understand how to write drivers, even if I understand a lot of the concepts. I abhor web UI and server side business logic. I adore user interaction, optimization, and graphics programming on client devices of all kinds. My sweet spot is coding against relatively low level OS and graphics APIs in user space. I wrote C and C++ code for a decade, but these days I prefer the simplicity of C#. Hope that helps!
- pornel 6y agoI've moved from web apps in JS to applications and network-level plumbing in Rust. It is more satisfying to work at machine's native speed. Computers are _fast_, especially when you use all CPU cores. ES6 is not bad, but Rust is joy to use. Tooling is solid, and doesn't need periodical `rm -rf node_modules`. It's easier to peel off and debug compile-time abstractions than runtime/VM abstractions. However, you mention "politics" and freedom how you program. I don't think that has anything to do with whether you write high-level or low-level code. If you work for an organization, you'll have to deal with management, whether you write JS widgets or firmware for a microwave. Programmers bikeshed at all levels, if it's not whether to use CSS-in-JS, then whether to use spinlocks. The real, technical constraints of bare metal have sparked decades-long microkernel flamewars.
- pjmlp 6y agoNah, it is going to be east const vs west const for hours. :)
- the__alchemist 6y agoRust needs a periodical `rm -rf target/` if you have finite storage.
- juancn 6y agoI do software for other engineers almost exclusively, what you would call "low level". Frameworks, APIs, compilers, query engines, highly concurrent code, distributed systems, etc. The "hard" stuff. (I've done embedded too) Building tools is what I truly love to do, they support business use cases, and I understand them, and I build my tools accordingly. For me it's the most fulfilling kind of software development. You need to know your stack to the metal, but you may not need to tinker with it. For example, I do a lot of Java, but high performance Java, I understand CPU cache access times, and roughly what hotspot will generate, what the OS will do, etc. and have the tools to check my assumptions. I don't do the low level stuff unless absolutely necessary, but I use the tools I have to their limit. You get deep knowledge of software stacks, and get to do experiments, and get to use all the cool algorithms and techniques, and even develop a few new ones. The part I love most, is that you keep learning. There's so much to learn and it's wicked fun. I've done high-level stuff, but I end up liking things like rendering engines or layout engines rather than building a dialog box. I'm ranting, so TLDR: learn your stack, fully, to the transistors if you can, and pick the spot where you have most fun. Money will follow.
- rramadass 6y agoSo You Wanna Be an Embedded Engineer: The Guide to Embedded Engineering, From Consultancy to the Corporate Ladder by Lewin Edwards.
- braddx 6y agoI guess it's some kind of fetish, to fit a genius piece of code (art) into such small device and have do great thing.
- blackcats 6y agoYes it is more satisfying. Programming C++ like 20 years ago. Not invent another way to solve a solved problem every year (JavaScript)
- robin21 6y agoThe higher up you are there are so many layers of abstraction that you need to rely on. To get good performance there are so many layers that can be a bottle neck. When you are doing low level stuff it can be liberating because you don’t have to rely on so many abstractions. I find most of programming dealing with bad abstractions or choosing a library/language that does 90% of what you want but the final 10% is extremely difficult or requires you to abandon some library/language and start from scratch.
- mike_k 6y agoI should have expressed this in a more clear way: the most interesting thing to understand for me here is the non-technical aspect of different jobs. It is not specifically about embedded, just more systems and low level than working with (as opposed to implementing from scratch) APIs, workers, queues, DBs. To rephrase the original question: it is not easy to imagine an "Agile" approach to a kernel/network or other kind of systems programming. I would be glad to know there is a chance for a more natural, R&D driven development and bit less fuss about corporate "values". Thanks guys. Your answers provide some really good and unexpected insights!
- the__alchemist 6y agoI'm having a lot of fun programming microcontrollers in Rust. It's nice being able to make something that responds to user input in a way that feels instantaneous. When you strip out the complexities of Operating systems, and the software they run that competes for resources, you can make programs that are a pleasure to use, and do a specific job well. I've been working mostly with Arm Cortex-M, like STM32. Give it a shot - it'll at least broaden your horizons. The Rust toolchain and peripheral access crates make getting started easier than with C toolchains. It's easy to spin embedded into a multi-domain learning experience. Ie, firmware, PCB design and ordering, enclosure machining, PC-based update tools, part choosing etc. It feels...liberating: If I think of an electronic device I'd like to exist, I have the tools available to make it happen. Nothing I've done in Python, Javascript, etc provided this.
- markus_zhang 6y agoI'm actually wondering where you get the luxury to work with C/C++ and Python at the same time. It's probably going to be one of the best things in the world.