5 ms·

Well, if you ask how to design “large” systems then the answer is already “don’t”. And besides that’s not a technical issue, our issues are organizational and p

Well, if you ask how to design “large” systems then the answer is already “don’t”. And besides that’s not a technical issue, our issues are organizational and political. Also, yes there is an universal pattern and that is called a competent and experienced software engineer. We lack those. We have a lot of kids with fancy degrees.

marginalia_nu 4y ago
There's large systems that do useful things though that don't decompose into smaller systems. Shouldn't we have those? Compilers. VMs. Operating systems. Search engines. LLMs...?
- SanderNL 4y ago
  The core tech of LLMs is famously simple. All the rest decomposes just fine. I know we suck at it and there often nontechnical forces that steer us into other directions, but I don’t see big fundamental issues AI can’t already help with. I know that there is complexity, but the essential part of it is not our job. That is my point. We are accidental complexity managers. I know we like to think working on compilers and browsers is magic, but compared to other fields it’s nothing special. And by the way, you and I know that exactly 6.78% of devs are employed in those domains and of those 1.12% work on core parts. I am sorry if I come off annoying and stupid because I am both. You of course raise an interesting point it’s just that I don’t feel like going into wide(r) tangents.
  - marginalia_nu 4y ago
    Why are you working on the boring parts, just gluing together prefab components?
    - SanderNL 4y ago
      See my last paragraph. Also it’s all glueing prefab parts. Software dev is a dead end job anyway, all parts of it.
      - marginalia_nu 4y ago
        Huh. I don't glue prefab parts when I develop.
        SanderNL 4y ago
        You do OS development?
        marginalia_nu 4y ago
        No, but I've built an internet search engine.
        SanderNL 4y ago
        Oh right, nice one btw, but seriously how are you not glueing prefabs? Are you writing your own db engine? Network layer?
        marginalia_nu 4y ago
        Yes on the db engine, at least for the document indexing part. I'm not building absolutely everything from scratch, a significant part of it is bespoke. There's just not a lot of off-the-shelf stuff that's built to deal with this type of application. You do occasionally see attempts at building internet search engines out of like elasticsearch or stuff like that, but it just doesn't scale.

marginalia_nu 4y ago

There's large systems that do useful things though that don't decompose into smaller systems. Shouldn't we have those? Compilers. VMs. Operating systems. Search engines. LLMs...?

SanderNL 4y ago
The core tech of LLMs is famously simple. All the rest decomposes just fine. I know we suck at it and there often nontechnical forces that steer us into other directions, but I don’t see big fundamental issues AI can’t already help with. I know that there is complexity, but the essential part of it is not our job. That is my point. We are accidental complexity managers. I know we like to think working on compilers and browsers is magic, but compared to other fields it’s nothing special. And by the way, you and I know that exactly 6.78% of devs are employed in those domains and of those 1.12% work on core parts. I am sorry if I come off annoying and stupid because I am both. You of course raise an interesting point it’s just that I don’t feel like going into wide(r) tangents.
- marginalia_nu 4y ago
  Why are you working on the boring parts, just gluing together prefab components?
  - SanderNL 4y ago
    See my last paragraph. Also it’s all glueing prefab parts. Software dev is a dead end job anyway, all parts of it.
    - marginalia_nu 4y ago
      Huh. I don't glue prefab parts when I develop.
      - SanderNL 4y ago
        You do OS development?
        marginalia_nu 4y ago
        No, but I've built an internet search engine.
        SanderNL 4y ago
        Oh right, nice one btw, but seriously how are you not glueing prefabs? Are you writing your own db engine? Network layer?
        marginalia_nu 4y ago
        Yes on the db engine, at least for the document indexing part. I'm not building absolutely everything from scratch, a significant part of it is bespoke. There's just not a lot of off-the-shelf stuff that's built to deal with this type of application. You do occasionally see attempts at building internet search engines out of like elasticsearch or stuff like that, but it just doesn't scale.