4 ms·

The bearishness here surprises me. The more a company spends on infrastructure, the more they need low-level people. A good low-level programmer can reduce cos

The bearishness here surprises me. The more a company spends on infrastructure, the more they need low-level people. A good low-level programmer can reduce cost requirements 10x or more. Any company spending millions or billions on infrastructure can make enormous savings by hiring the right people. Crucial for Google, Amazon, Facebook, and even midsize startups can see a big improvement. And its difficult to fill those jobs. Which means, there's opportunity there.

throwaway777836 10y ago
Because those companies, and their mindsets, represent a fraction of all software jobs. The performance improvements that come from having good systems programmers appeal to only a small subset of companies. Even if the opportunity is there, features are what matter primarily to most companies. Only the ones with deep pockets and forward thinking will shell out for talent who will improve the infrastructure in a way that leads to often incremental and unnoticed or "unnecessary" improvements.
chocolatebunny 10y ago
Can you clarify your statement?What does a low-level person do to improve infrastructure?
- bitwiseand 10y ago
  Writing and designing code that is mindful of latencies (Amdahl's law). Temporal / Spatial locality, avoiding needless copies etc.. This approach needs to start from design and also follow into implementing the code. Moreover, this is done without compromising on readability.
  - CalChris 10y ago
    Umm, Amdahl's law is actually a negative result. It's saying that if you infinitely improve some magical hotspot that is 1% of your workload, the best you can only ever get to is 99%. If anything, Amdahl is telling us not to grind on trivialities.
    - analognoise 10y ago
      That's not how I understood it - I thought it was about the upper limit of theoretical parallelism. But I'm an EE, so maybe I misunderstood some finer points?
      - CalChris 10y ago
        An upper limit on the benefits. To me, an upper limit is a negative result. When Leon (in Blade Runner) finds out that he only has 4 years to live (an upper bound) he takes it pretty hard. Amdahl: A fairly obvious conclusion which can be drawn at this point is that the effort expended on achieving high parallel processing rates is wasted unless it is accompanied by achievements in sequential processing rates of very nearly the same magnitude. http://www-inst.eecs.berkeley.edu/~n252/paper/Amdahl.pdf http://www-inst.eecs.berkeley.edu/~n252/paper/Amdahl.pdf A point made when we read the paper was that Seymour Cray always made sure that his computers were also the fastest scalar computers even though they were sold as vector processors.
        analognoise 10y ago
        Thank you for the excellent explanation; I will consider it more thoroughly. I think mathematicians consider an upper limit a positive bound - positive, in the sense of being well defined; you're using negative in the other sense? I actually like that quite a bit.
    - bitwiseand 10y ago
      If a system performance is profiled as being dominated by 80% A , 19% B and 1 % C. I'd focus on working on A first to get maximum gains. Amdahl's law gives you the backing as to why you should do this. As a real world example, if an operation involves a network call and you see the RTT dominating the time. You may want to think of ways to avoid the call (caching etc..) if possible to get really good gains.
- StillBored 10y ago
  Gosh... Lets say you have to purchase X servers at Y dollars, where X*Y>1E8. How much does an engineer making $200k a year have to reduce X by before he pays for himself? Not much, and simply finding a hotspot and doubling the performance by aligning a data structure or whatnot could pay for his salary for a decade. The old adage that computers are cheap and engineers are expensive is about 15 years out of date for a large portion of the software being written today. That is when companies stopped being able to generally externalize the costs of shitty code. Plus, as microcontrollers ate the world, the ability to ship a $1 micro controller instead of a $10 one on a product run of 100k units can mean the difference between a competitive growing company and a dying one.
CalChris 10y ago
My Facebook iPhone app weighs in at 241 MB. It has massive redundancy and you can say that again. http://blog.timac.org/?p=1707 http://blog.timac.org/?p=1707 Still, I have to say that there is indeed an opportunity for a few low level people there.
- deleted 10y ago
  [deleted]
bitwiseand 10y ago
I fully agree with you. It's not just infrastructure but also devices and the surrounding ecosystem. I work in this space and let me assure you there are many jobs with GREAT pay. I"ll go on and make another bold claim. Focusing on lower level stuff and systems concepts lay an excellent foundation for designing complex systems regardless of the language used. Once you understand how a program is executed from the ground up i.e. right from the program counter to page table lookups to cache hits to cache coherency, all abstractions are easier to deconstruct and understand. I work up and down the stack and this has been my, quite possibly just anecdotal, experience.
- dbancajas 10y ago
  how much we talkin?
badloginagain 10y ago
Game development also needs people going low. Both in terms of cranking every last bit of performance from an end users machine, to packing as much information into a packet as possible to stream to the server.
- rectang 10y ago
  True, but the game industry is legendary for horrible working conditions (thanks to an oversupply of labor).
  - badloginagain 10y ago
    > oversupply of labor The funny thing is, its the opposite for the low level stuff. Nobody does it, so there's a huge demand, which results in too much work for those who are filling those roles. Game development is broken for a litany of reasons, but I resist the urge to go into it to keep things on topic. I will say that it gets kids in STEM, and for the youngin's, getting into the low level stuff is a safe bet for a profitable career.
    - yoklov 10y ago
      > The funny thing is, its the opposite for the low level stuff. This was not my experience or the experience of people I know at all. I imagine that it depends on which companies you work with, or what country you're in, but my experience was that low level skills often didn't give you very much in the way of extra job security, and certainly didn't save you from crunch, low pay (much lower than everywhere else in software), etc.
jackmott 10y ago
Even if you work in domains where you don't need to do low level programming, knowing how the hardware works at a low level helps you do a better job. When you know what the stack and the heap are, when you know how memory locality affects performance, when you know the full cost of allocations, you can write better Java/C#/OCaml/Haskell, you can check on the assembler they output, you will be ready to occasionally write small C libraries if portions or slow, etc. Ignoring this stuff completely leads to text editors made in electron.
- nojvek 10y ago
  Tbh the text editors in electron do try to squeeze as much as they can from the web API's. The VSCode tokenizer putting dense info in typed arrays as bitfields is a good example of someone understanding how memory access works under the hood. However I do agree with you, getting into web dev is easy. Doing things closer to metal is harder. I would definitely love to do more low level C programming.
Zooper 10y ago
The creation of that which is novel is another allure.

throwaway777836 10y ago

Because those companies, and their mindsets, represent a fraction of all software jobs. The performance improvements that come from having good systems programmers appeal to only a small subset of companies. Even if the opportunity is there, features are what matter primarily to most companies. Only the ones with deep pockets and forward thinking will shell out for talent who will improve the infrastructure in a way that leads to often incremental and unnoticed or "unnecessary" improvements.

chocolatebunny 10y ago

Can you clarify your statement?What does a low-level person do to improve infrastructure?

bitwiseand 10y ago
Writing and designing code that is mindful of latencies (Amdahl's law). Temporal / Spatial locality, avoiding needless copies etc.. This approach needs to start from design and also follow into implementing the code. Moreover, this is done without compromising on readability.
- CalChris 10y ago
  Umm, Amdahl's law is actually a negative result. It's saying that if you infinitely improve some magical hotspot that is 1% of your workload, the best you can only ever get to is 99%. If anything, Amdahl is telling us not to grind on trivialities.
  - analognoise 10y ago
    That's not how I understood it - I thought it was about the upper limit of theoretical parallelism. But I'm an EE, so maybe I misunderstood some finer points?
    - CalChris 10y ago
      An upper limit on the benefits. To me, an upper limit is a negative result. When Leon (in Blade Runner) finds out that he only has 4 years to live (an upper bound) he takes it pretty hard. Amdahl: A fairly obvious conclusion which can be drawn at this point is that the effort expended on achieving high parallel processing rates is wasted unless it is accompanied by achievements in sequential processing rates of very nearly the same magnitude. http://www-inst.eecs.berkeley.edu/~n252/paper/Amdahl.pdf http://www-inst.eecs.berkeley.edu/~n252/paper/Amdahl.pdf A point made when we read the paper was that Seymour Cray always made sure that his computers were also the fastest scalar computers even though they were sold as vector processors.
      - analognoise 10y ago
        Thank you for the excellent explanation; I will consider it more thoroughly. I think mathematicians consider an upper limit a positive bound - positive, in the sense of being well defined; you're using negative in the other sense? I actually like that quite a bit.
  - bitwiseand 10y ago
    If a system performance is profiled as being dominated by 80% A , 19% B and 1 % C. I'd focus on working on A first to get maximum gains. Amdahl's law gives you the backing as to why you should do this. As a real world example, if an operation involves a network call and you see the RTT dominating the time. You may want to think of ways to avoid the call (caching etc..) if possible to get really good gains.
StillBored 10y ago
Gosh... Lets say you have to purchase X servers at Y dollars, where X*Y>1E8. How much does an engineer making $200k a year have to reduce X by before he pays for himself? Not much, and simply finding a hotspot and doubling the performance by aligning a data structure or whatnot could pay for his salary for a decade. The old adage that computers are cheap and engineers are expensive is about 15 years out of date for a large portion of the software being written today. That is when companies stopped being able to generally externalize the costs of shitty code. Plus, as microcontrollers ate the world, the ability to ship a $1 micro controller instead of a $10 one on a product run of 100k units can mean the difference between a competitive growing company and a dying one.

CalChris 10y ago

My Facebook iPhone app weighs in at 241 MB. It has massive redundancy and you can say that again. http://blog.timac.org/?p=1707 http://blog.timac.org/?p=1707 Still, I have to say that there is indeed an opportunity for a few low level people there.

deleted 10y ago
[deleted]

bitwiseand 10y ago

I fully agree with you. It's not just infrastructure but also devices and the surrounding ecosystem. I work in this space and let me assure you there are many jobs with GREAT pay. I"ll go on and make another bold claim. Focusing on lower level stuff and systems concepts lay an excellent foundation for designing complex systems regardless of the language used. Once you understand how a program is executed from the ground up i.e. right from the program counter to page table lookups to cache hits to cache coherency, all abstractions are easier to deconstruct and understand. I work up and down the stack and this has been my, quite possibly just anecdotal, experience.

dbancajas 10y ago
how much we talkin?

badloginagain 10y ago

Game development also needs people going low. Both in terms of cranking every last bit of performance from an end users machine, to packing as much information into a packet as possible to stream to the server.

rectang 10y ago
True, but the game industry is legendary for horrible working conditions (thanks to an oversupply of labor).
- badloginagain 10y ago
  > oversupply of labor The funny thing is, its the opposite for the low level stuff. Nobody does it, so there's a huge demand, which results in too much work for those who are filling those roles. Game development is broken for a litany of reasons, but I resist the urge to go into it to keep things on topic. I will say that it gets kids in STEM, and for the youngin's, getting into the low level stuff is a safe bet for a profitable career.
  - yoklov 10y ago
    > The funny thing is, its the opposite for the low level stuff. This was not my experience or the experience of people I know at all. I imagine that it depends on which companies you work with, or what country you're in, but my experience was that low level skills often didn't give you very much in the way of extra job security, and certainly didn't save you from crunch, low pay (much lower than everywhere else in software), etc.

jackmott 10y ago

Even if you work in domains where you don't need to do low level programming, knowing how the hardware works at a low level helps you do a better job. When you know what the stack and the heap are, when you know how memory locality affects performance, when you know the full cost of allocations, you can write better Java/C#/OCaml/Haskell, you can check on the assembler they output, you will be ready to occasionally write small C libraries if portions or slow, etc. Ignoring this stuff completely leads to text editors made in electron.

nojvek 10y ago
Tbh the text editors in electron do try to squeeze as much as they can from the web API's. The VSCode tokenizer putting dense info in typed arrays as bitfields is a good example of someone understanding how memory access works under the hood. However I do agree with you, getting into web dev is easy. Doing things closer to metal is harder. I would definitely love to do more low level C programming.

Zooper 10y ago

The creation of that which is novel is another allure.