14 ms·

Becoming a Systems Architect

238 points by tetek 4y ago

fwr 4y ago
Even John Carmack - an ultimate prototype for a nerd - has evolved into a CTO (and from the linked Fridman interview he sounds like a really good one). What is it about tinkerers being put into these positions as a natural progression? It always seemed to me that people with more of a managerial background would be better managers - is software/system development the only field where masters of their craft ultimately become directors?
- deleted 4y ago
  [deleted]
- guhidalg 4y ago
  Software isn't the only field where practitioners get promoted to management, but it is one of the fields where technically incompetent managers and executives will kill a company with unsound decisions.
- dnadler 4y ago
  > What is it about tinkerers being put into these positions as a natural progression? I don't know enough to say whether this is a common pattern or not, but if it is, it could perhaps be that tinkerers tend to gather a vast breadth of knowledge that can be very useful when making strategic decisions. They reach the point where they know enough to understand what questions to ask on many topics, even if they are an expert in only a few (or none) of them.
  - convolvatron 4y ago
    I'm not the worlds greatest manager - but after you've worked with hundreds or thousands of people and seen seen hundreds of projects come and go you start to develop some insight about how things go and how they can go bad. I don't know how else you pick up the skills to be an effective technical manager
- PainfullyNormal 4y ago
  > What is it about tinkerers being put into these positions as a natural progression? Because those tinkerers get to a point where they want to be the ones making the decisions, controlling the culture, technology, and direction of the company. Without position you have no power and without power you can't affect change.
- moonchrome 4y ago
  You get to scale up your impact by managing people.
  - powerhour 4y ago
    In my case my impact plummeted. Sometimes going in to management means having your hands tied -- you are the ones allowing higher ups to scale their impact. It was awful. I'd love to be able to find folks that can do what I do -- it's probably our #1 issue holding us back -- and direct them to meet some business and technical goals, but I've yet to work at an organization that supports that mode of management.
  - oaiey 4y ago
    And here your organization has failed you. In our org we split engineering career at a certain point into three careers: engineering, people and architects. Up to the VP level their grading run in parallel. The architects are optimized for tech and business impact while the people managers are good in managing people. and yes engineering tracks and architecture track optimize certain skills but are not necessarily a hard separation
- theshrike79 4y ago
  When a manager promoted or advances through the tech track to a CTO or Architect position, they still keep their base-level knowledge. It's a lot easier to spot developers or contractors bullshitting when you've been in their shoes.
- hungryforcodes 4y ago
  Tinkerers are basically technology generalist, and have tinkered in probably everything. So they can map out solutions and have good ideas of what will work and won't. People with managerial backgrounds have....what? Nothing really. They have to guess at any plan presented by technical people, are always suspicious they are being screwed over on estimations and real problem areas, and are unable to correctly identify when people are doing good work -- thus also being unable to set a healthy engineering culture for success. That's why most managers are demoralizing for engineers. They just don't get it. It's interesting to note that alot of the most successful startups in SV are not from MBA's but engineers with masters or PhDs....it's not a coincidence I think. They have the practical experience to lead a real world venture to success. Managers are good at managing departments like insurance claim processing or bad debt collections, which any human can learn in a few weeks fundamentally.
  - jrvarela56 4y ago
    This is an overly cynical view of management. Distrust and "screwed by estimations" are signs that the dynamic needs tweaking, not that all dynamics are like this. People with managerial backgrounds can become quite adept at helping you: - Identify blindspots in your biases and behavior that keep you from peak performance - Avoid working on stuff that's not valuable to your team - Settle disputes within a group - Motivate you and keep you engaged/fulfilled with your work - Get unstuck with personal problems This is not an exhaustive list and you don't have to have a 'managerial background' to master stuff like this. I am an engineer who has had to learn management as a startup founder. I used to distrust the whole management thing but that kept me from growing as a teammate. Management is not only useful in 'non technical' jobs, it's useful in all human endeavors it's why we study it so much and why it has so much leverage.
    - jpfr 4y ago
      As a startup founder you are automatically at the top of the company food-chain. Managers in the middle of the food-chain are all about power-struggle and social games they play to out-compete each other. That is especially the case for non-technical managers who don’t have a sense for the underlying technical challenges. They have all their bandwidth available for political positioning and social games. It would be great if all managers where “serving the team” in the sense of your bullet points. But alas many don’t see it that way. IMHO many startups are successful because they have a technical person at the top. Who is capable to understand, evaluate and positively reward technical work within their organization.
- bob1029 4y ago
  > What is it about tinkerers being put into these positions as a natural progression? Probably ambition. At a certain level of experience, you realize you cannot create your technological dreamscape by yourself. Becoming lord of a technology company and directing its resources is like programming the ultimate computer.
- _gabe_ 4y ago
  > Even John Carmack - an ultimate prototype for a nerd - has evolved into a CTO I didn't get the impression that he's your typical CTO and completely hands off with code and development. If anything, it sounded like he's still very much in the trenches but has learned how to delegate work well and pick which problems are worth the time investment. When it came to his work on Oculus and the work he's about to do in the field of AGI it sounded like he'll definitely be making direct contributions. It's entirely possible that I misunderstood his stance throughout the interview though and he's a more hands off guy now.
- larve 4y ago
  I feel that turning to leadership (tech leadership, but also pure engineering management) can be a natural "tinkerer" progression. If you want bigger impact, you turn to more "abstraction", and use tooling and patterns to fill out the details. In this case, you can consider that your role is to give broad directions. They are very important directions, because they shape the "design space" of the problem you are trying to solve. After that, you have a set of "programming tools" (I don't want it to sound mechanistic, because it is the opposite of that), which are your team and reports, and they will be able to fill in the details (and the details here can be significant pieces of design and architecture by themselves). And your role is to choose the right tools, and allow them to work to their full potential. This means clearing obstacles, clear communication, technical help at times, mentorship, aligning expectations and giving them clear paths for growth. All these things can be considered engineering at a larger scale. You want to get a really big system shipped and productive? This is the work, these are the skills you need.
- edmundsauto 4y ago
  It's not they are tinkerers necessarily - it's that they want their impact to scale larger than what they themselves can create. Solo coder -> tech lead -> CTO is like going from single CPU -> multiple CPU -> distributed systems.
- robertlagrant 4y ago
  My impression is lawyers and doctors also get promoted for core skills rather than management skills.
- hcarvalhoalves 4y ago
  > It always seemed to me that people with more of a managerial background would be better managers The distinction dates back to the industrial revolution, where you had manufacturing line workers and line managers. A manager would usually be an owner's relative or someone they trusted – more loyal to the company than the unions. This distinction perpetuated well into current age, just notice how much implicit bias there is about "programmers don't have people skills" to keep workers accepting a career ceiling. Most managers aren't skilled either, and not respected by the workers due to it, but companies won't keep them from managing because they need someone to be responsible for plans, estimates, OKRs, etc. I guess in software it's "more" common due to survivorship bias – the business of software is so messed up and people have so little idea how to manage it, that companies without experienced leaders have a smaller chance. Strong companies and teams in the field have leaders with enough hands-on experience to have natural authority.
- zenlot 4y ago
  Who's programming 12 hours a day. He never ever said in that interview, that he's going away from programming. He's been "CTO" in the early days too. You misunderstood the whole conversation if you think he's turning into an "architect" only.
- ebcode 4y ago
  > is software/system development the only field where masters of their craft ultimately become directors? Theatre and Cinema are two other fields where folks have been known to work their way up to director/producer/megalomaniac... I think Weinberg in "The Psychology of Computer Programming" talks about the performative nature of writing code.
- baby 4y ago
  Carmack did end up moving from Meta VR (although he still works there one day a week). I presume it is because he was not doing enough engineering work. I used to read all of his posts internally and he really did seem to be an amazing CTO.
sarchertech 4y ago
I’m currently a principal engineer, but I’ve had architect titles in the past. One worry I’ve had with taking jobs with that title is that the salary range seems significantly lower on average. Even if the specific job paid well, I worry about pigeonholing myself into those roles.
OnionBlender 4y ago
Does anyone know of any good books for getting better at System Design that isn't web focused? Every time I look for "System Design" the books are about creating a highly distributed web app with SQL or NoSQL database, load balancers, and other web centric stuff. I've spent most of my career fixing bugs or improving existing apps/systems in C++. Every time I interview somewhere they want me to design a (usually offline) program or service and I don't know how best to do that. At this rate I'll be stuck at Senior Engineer for the rest of my career. (I'm already 14 years in).
- larve 4y ago
  I learned a lot by writing "prototypes." If you look at system design and architecture from afar, they're all very simple patterns that can be written up in a self contained python or javascript program: * RPC = function call * API design = function signature * data schema = simple structs * message queue = list * load balancer = simple facade object * database = hash table / in memory SQLite Once you have a rough sketch in place, you can run your prototype and test different behaviours. It is a bit "tricky" to tackle asynchronous and concurrency, which you should do from the start. I prefer to do everything based on callbacks, and run my simulation from a static input list of events, ticking every component with a tick() method. This way I can generate different scenarios, save them to JSON, replay them. Here's some examples: - https://github.com/wesen/summer-pasture-netflix https://github.com/wesen/summer-pasture-netflix cell.py is a kubernetes prototype (blackboard system), while app.py is the start of the netflix event notification system. I am thinking of writing a course / book about the subject, and have a bunch of (wild) notes here: https://publish.obsidian.md/manuel/ZK/Structure+Notes/2+-+Software/2i0+-+Prototypes/SK+-+2i0+-++2022-08-13+-+Building+prototypes+for+software+architecture https://publish.obsidian.md/manuel/ZK/Structure+Notes/2+-+So...
  - pramodbiligiri 4y ago
    This sounds like a great idea. If you can help simulate non-determinism and faults that will take it to the next level.
    - 4y ago
CSMastermind 4y ago
> no matter how many smart people you hire, and how many nice frameworks are used - if the organization and processes are not in order, you will keep losing time and money. I disagree. In my experience hiring the right people limits the need for organization and process. Obviously you can't eliminate those completely but if you hire the right people your org will generally succeed in spite of a lack of organization and process. I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team.
- tetek 4y ago
  You are not wrong, my experience comes from what I think might be a niche. A startup doing both hardware and software, and does so with an offshore team. For 'simple' systems, Systems Engineer is not needed. Typically Software Engineers share architectural responsibilities. I recommend listening to this https://youtu.be/pSfZutP9H-U?t=386 https://youtu.be/pSfZutP9H-U?t=386 from 6:30 to 10min. I find it very valuable. > I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team. I agree 100%.
- Cwizard 4y ago
  Yes but if you hire a lot of smart people and then have a process where they cannot make any decisions but instead someone from sales decides every little thing you will still not get anywhere regardless of how many smart people you have. But I agree if you have a lot of smart people you can go for a lean process, and let everyone mostly guide themselves. But you do have to provide that freedom.
- spaetzleesser 4y ago
  Exactly. There is a difference between "right people" and "smart people". The right people are usually smart but smart people aren't always right for the team. If you have the right people you don't much process or organization imposed from top. They will organize themselves.
angarg12 4y ago
> I can see, that I’m indeed a Systems Architect, and not a Software Engineer anymore. I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do? I prefer how my company career ladder, where there are no architect titles. Each level is 'fractal', containing almost the same responsibilities but at different scales and scope. So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service, while as a Staff Engineer you will oversee the architecture of a system spanning several teams. This way architecture is everyone's responsibility, and people have a natural path to grow their skills.
- tetek 4y ago
  Of course I'm not just architecting everything by myself, and making decisions all day long. Every Software Engineer still makes a lot of architectural decisions on different levels - exactly how you described. There's just a lot of other activities that Systems Engineer does (and still consults them with the team, but is at the end responsible for doing them). (all the bullet points in "The realization")
- cryptozeus 4y ago
  No, architects also code. Google “lean architect”
- humbleMouse 4y ago
- Cwizard 4y ago
  Did you read the article? The description the OP gives does not sound like the situation you describe
- pjmlp 4y ago
  We never left those days on the enterprise space.
- swatcoder 4y ago
  It might be harmful if it was universal, but it's really just another facet of the normal, perpetual tension between generalization and specialization. Large organizations working on grand projects benefit from slicing up roles so that experienced specialists can anticipate and prepare solutions for the uncommon challenges that arise in those projects (and so that the bureaucracy of large orgs can be managed at all). Meanwhile, smaller organizations, or groups working on more modestly scoped projects, benefit from generalists who can hold much of the project in their head and develop highly cohesive solutions that can be delivered efficiently. A skyscraper requires seasoned architects or its going to crumble under its own weight, but even an extensive residential home renovation can be planned by a general contractor and done way cheaper than if many specialists were brought in. It sounds like there are some organizations/projects you wouldn't want to be involved with, but that doesn't mean that those organizations/projects are doing something harmful or regressive. They just don't suit the way you want to approach work.
balentio 4y ago
Most folks define their title by what they are paid to do. It follows then that if you are not paid to do anything, then you don't have the title, right? Well, if that's not quite right, then you haven't really become anything but rather you are someone who is mucking about with systems and happen to be being paid for it.
spaetzleesser 4y ago
Whenever I hear the title "Architect" I have to think about certain segments of the IT department at my company and also the participants at my scrum master training. Both were full of "senior architects" that were good at talking but were so out of touch with reality that they couldn't contribute anything real. It's pretty easy to architect a system if you ignore the little, annoying details that make an implementation difficult. And to know what's hard you need hands-on implementation experience. These guys get away with their nonsense because often the implementation is outsourced to developers who just implement what they are told to do without asking questions. Never mind that the bad design decisions make the system much slower than it has to be and takes way more effort to implement.
- hu3 4y ago
  Gotta love those over-engineered projects where to add a text field to an existing form requires editing more than 15 files. So may layers of indirection that debugging is a nightmare and callstacks look like the full lorem ipsum.
  - oaiey 4y ago
    That is overengineering and bad technical design. Can happen completely independent of the architecture profession.
    - contingencies 4y ago
      The problem with object-oriented languages is they've got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. - Joe Armstrong ... via https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
- oaiey 4y ago
  I am an architect. Proud of my profession. I code only for fun. I do solution architecture in a scope with 100s of impacted developers. I highly suspect that dozens of them are describing us exactly like you do. However, once they challenge us and we elaborate the problem to them, their solution usually falls apart and they align with our solution. Why: because in that scale of complexity we operate in not every engineer can maintain the necessary knowledge. I highly respect the approach of architecture in the team/agile environment and seniority in engineering. But at a given scale, someone is there to orchestrate all of it and does the overall plan. That person is a solution architect.
Melatonic 4y ago
In my opinion "architect" should probably really be the most senior role of people that are doing infrastructure. It can be applied to more specific software engineering roles but it just seems more appropriate considering the infrastructure is what everyone is using as a tool to run their various projects.
agluszak 4y ago
Greetings from "the legendary MIM faculty" ;)
joeamroo 4y ago
I prevent cross-site scripting, I monitor for DDoS attacks, emergency database rollbacks, and faulty transaction handlings. The Internet heard of it? Transfers half a petabyte of data every minute. Do you have any idea how that happens? All those YouPorn ones and zeroes streaming directly to your shitty, little smart phone day after day? Every dipshit who shits his pants if he can't get the new dubstep Skrillex remix in under 12 seconds? It's not magic, it's talent and sweat. People like me, ensuring your packets get delivered, un-sniffed. So what do I do? I make sure that one bad config on one key component doesn't bankrupt the entire fucking company. That's what the fuck I do.
- nonethewiser 4y ago
  - Michael Scott, The Office.

238 points by tetek 4y ago

fwr 4y ago

Even John Carmack - an ultimate prototype for a nerd - has evolved into a CTO (and from the linked Fridman interview he sounds like a really good one). What is it about tinkerers being put into these positions as a natural progression? It always seemed to me that people with more of a managerial background would be better managers - is software/system development the only field where masters of their craft ultimately become directors?

deleted 4y ago
[deleted]
guhidalg 4y ago
Software isn't the only field where practitioners get promoted to management, but it is one of the fields where technically incompetent managers and executives will kill a company with unsound decisions.
dnadler 4y ago
> What is it about tinkerers being put into these positions as a natural progression? I don't know enough to say whether this is a common pattern or not, but if it is, it could perhaps be that tinkerers tend to gather a vast breadth of knowledge that can be very useful when making strategic decisions. They reach the point where they know enough to understand what questions to ask on many topics, even if they are an expert in only a few (or none) of them.
- convolvatron 4y ago
  I'm not the worlds greatest manager - but after you've worked with hundreds or thousands of people and seen seen hundreds of projects come and go you start to develop some insight about how things go and how they can go bad. I don't know how else you pick up the skills to be an effective technical manager
PainfullyNormal 4y ago
> What is it about tinkerers being put into these positions as a natural progression? Because those tinkerers get to a point where they want to be the ones making the decisions, controlling the culture, technology, and direction of the company. Without position you have no power and without power you can't affect change.
moonchrome 4y ago
You get to scale up your impact by managing people.
- powerhour 4y ago
  In my case my impact plummeted. Sometimes going in to management means having your hands tied -- you are the ones allowing higher ups to scale their impact. It was awful. I'd love to be able to find folks that can do what I do -- it's probably our #1 issue holding us back -- and direct them to meet some business and technical goals, but I've yet to work at an organization that supports that mode of management.
- oaiey 4y ago
  And here your organization has failed you. In our org we split engineering career at a certain point into three careers: engineering, people and architects. Up to the VP level their grading run in parallel. The architects are optimized for tech and business impact while the people managers are good in managing people. and yes engineering tracks and architecture track optimize certain skills but are not necessarily a hard separation
theshrike79 4y ago
When a manager promoted or advances through the tech track to a CTO or Architect position, they still keep their base-level knowledge. It's a lot easier to spot developers or contractors bullshitting when you've been in their shoes.
hungryforcodes 4y ago
Tinkerers are basically technology generalist, and have tinkered in probably everything. So they can map out solutions and have good ideas of what will work and won't. People with managerial backgrounds have....what? Nothing really. They have to guess at any plan presented by technical people, are always suspicious they are being screwed over on estimations and real problem areas, and are unable to correctly identify when people are doing good work -- thus also being unable to set a healthy engineering culture for success. That's why most managers are demoralizing for engineers. They just don't get it. It's interesting to note that alot of the most successful startups in SV are not from MBA's but engineers with masters or PhDs....it's not a coincidence I think. They have the practical experience to lead a real world venture to success. Managers are good at managing departments like insurance claim processing or bad debt collections, which any human can learn in a few weeks fundamentally.
- jrvarela56 4y ago
  This is an overly cynical view of management. Distrust and "screwed by estimations" are signs that the dynamic needs tweaking, not that all dynamics are like this. People with managerial backgrounds can become quite adept at helping you: - Identify blindspots in your biases and behavior that keep you from peak performance - Avoid working on stuff that's not valuable to your team - Settle disputes within a group - Motivate you and keep you engaged/fulfilled with your work - Get unstuck with personal problems This is not an exhaustive list and you don't have to have a 'managerial background' to master stuff like this. I am an engineer who has had to learn management as a startup founder. I used to distrust the whole management thing but that kept me from growing as a teammate. Management is not only useful in 'non technical' jobs, it's useful in all human endeavors it's why we study it so much and why it has so much leverage.
  - jpfr 4y ago
    As a startup founder you are automatically at the top of the company food-chain. Managers in the middle of the food-chain are all about power-struggle and social games they play to out-compete each other. That is especially the case for non-technical managers who don’t have a sense for the underlying technical challenges. They have all their bandwidth available for political positioning and social games. It would be great if all managers where “serving the team” in the sense of your bullet points. But alas many don’t see it that way. IMHO many startups are successful because they have a technical person at the top. Who is capable to understand, evaluate and positively reward technical work within their organization.
bob1029 4y ago
> What is it about tinkerers being put into these positions as a natural progression? Probably ambition. At a certain level of experience, you realize you cannot create your technological dreamscape by yourself. Becoming lord of a technology company and directing its resources is like programming the ultimate computer.
_gabe_ 4y ago
> Even John Carmack - an ultimate prototype for a nerd - has evolved into a CTO I didn't get the impression that he's your typical CTO and completely hands off with code and development. If anything, it sounded like he's still very much in the trenches but has learned how to delegate work well and pick which problems are worth the time investment. When it came to his work on Oculus and the work he's about to do in the field of AGI it sounded like he'll definitely be making direct contributions. It's entirely possible that I misunderstood his stance throughout the interview though and he's a more hands off guy now.
larve 4y ago
I feel that turning to leadership (tech leadership, but also pure engineering management) can be a natural "tinkerer" progression. If you want bigger impact, you turn to more "abstraction", and use tooling and patterns to fill out the details. In this case, you can consider that your role is to give broad directions. They are very important directions, because they shape the "design space" of the problem you are trying to solve. After that, you have a set of "programming tools" (I don't want it to sound mechanistic, because it is the opposite of that), which are your team and reports, and they will be able to fill in the details (and the details here can be significant pieces of design and architecture by themselves). And your role is to choose the right tools, and allow them to work to their full potential. This means clearing obstacles, clear communication, technical help at times, mentorship, aligning expectations and giving them clear paths for growth. All these things can be considered engineering at a larger scale. You want to get a really big system shipped and productive? This is the work, these are the skills you need.
edmundsauto 4y ago
It's not they are tinkerers necessarily - it's that they want their impact to scale larger than what they themselves can create. Solo coder -> tech lead -> CTO is like going from single CPU -> multiple CPU -> distributed systems.
robertlagrant 4y ago
My impression is lawyers and doctors also get promoted for core skills rather than management skills.
hcarvalhoalves 4y ago
> It always seemed to me that people with more of a managerial background would be better managers The distinction dates back to the industrial revolution, where you had manufacturing line workers and line managers. A manager would usually be an owner's relative or someone they trusted – more loyal to the company than the unions. This distinction perpetuated well into current age, just notice how much implicit bias there is about "programmers don't have people skills" to keep workers accepting a career ceiling. Most managers aren't skilled either, and not respected by the workers due to it, but companies won't keep them from managing because they need someone to be responsible for plans, estimates, OKRs, etc. I guess in software it's "more" common due to survivorship bias – the business of software is so messed up and people have so little idea how to manage it, that companies without experienced leaders have a smaller chance. Strong companies and teams in the field have leaders with enough hands-on experience to have natural authority.
zenlot 4y ago
Who's programming 12 hours a day. He never ever said in that interview, that he's going away from programming. He's been "CTO" in the early days too. You misunderstood the whole conversation if you think he's turning into an "architect" only.
ebcode 4y ago
> is software/system development the only field where masters of their craft ultimately become directors? Theatre and Cinema are two other fields where folks have been known to work their way up to director/producer/megalomaniac... I think Weinberg in "The Psychology of Computer Programming" talks about the performative nature of writing code.
baby 4y ago
Carmack did end up moving from Meta VR (although he still works there one day a week). I presume it is because he was not doing enough engineering work. I used to read all of his posts internally and he really did seem to be an amazing CTO.

sarchertech 4y ago

I’m currently a principal engineer, but I’ve had architect titles in the past. One worry I’ve had with taking jobs with that title is that the salary range seems significantly lower on average. Even if the specific job paid well, I worry about pigeonholing myself into those roles.

OnionBlender 4y ago

Does anyone know of any good books for getting better at System Design that isn't web focused? Every time I look for "System Design" the books are about creating a highly distributed web app with SQL or NoSQL database, load balancers, and other web centric stuff. I've spent most of my career fixing bugs or improving existing apps/systems in C++. Every time I interview somewhere they want me to design a (usually offline) program or service and I don't know how best to do that. At this rate I'll be stuck at Senior Engineer for the rest of my career. (I'm already 14 years in).

larve 4y ago
I learned a lot by writing "prototypes." If you look at system design and architecture from afar, they're all very simple patterns that can be written up in a self contained python or javascript program: * RPC = function call * API design = function signature * data schema = simple structs * message queue = list * load balancer = simple facade object * database = hash table / in memory SQLite Once you have a rough sketch in place, you can run your prototype and test different behaviours. It is a bit "tricky" to tackle asynchronous and concurrency, which you should do from the start. I prefer to do everything based on callbacks, and run my simulation from a static input list of events, ticking every component with a tick() method. This way I can generate different scenarios, save them to JSON, replay them. Here's some examples: - https://github.com/wesen/summer-pasture-netflix https://github.com/wesen/summer-pasture-netflix cell.py is a kubernetes prototype (blackboard system), while app.py is the start of the netflix event notification system. I am thinking of writing a course / book about the subject, and have a bunch of (wild) notes here: https://publish.obsidian.md/manuel/ZK/Structure+Notes/2+-+Software/2i0+-+Prototypes/SK+-+2i0+-++2022-08-13+-+Building+prototypes+for+software+architecture https://publish.obsidian.md/manuel/ZK/Structure+Notes/2+-+So...
- pramodbiligiri 4y ago
  This sounds like a great idea. If you can help simulate non-determinism and faults that will take it to the next level.
  - 4y ago

CSMastermind 4y ago

> no matter how many smart people you hire, and how many nice frameworks are used - if the organization and processes are not in order, you will keep losing time and money. I disagree. In my experience hiring the right people limits the need for organization and process. Obviously you can't eliminate those completely but if you hire the right people your org will generally succeed in spite of a lack of organization and process. I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team.

tetek 4y ago
You are not wrong, my experience comes from what I think might be a niche. A startup doing both hardware and software, and does so with an offshore team. For 'simple' systems, Systems Engineer is not needed. Typically Software Engineers share architectural responsibilities. I recommend listening to this https://youtu.be/pSfZutP9H-U?t=386 https://youtu.be/pSfZutP9H-U?t=386 from 6:30 to 10min. I find it very valuable. > I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team. I agree 100%.
Cwizard 4y ago
Yes but if you hire a lot of smart people and then have a process where they cannot make any decisions but instead someone from sales decides every little thing you will still not get anywhere regardless of how many smart people you have. But I agree if you have a lot of smart people you can go for a lean process, and let everyone mostly guide themselves. But you do have to provide that freedom.
spaetzleesser 4y ago
Exactly. There is a difference between "right people" and "smart people". The right people are usually smart but smart people aren't always right for the team. If you have the right people you don't much process or organization imposed from top. They will organize themselves.

angarg12 4y ago

> I can see, that I’m indeed a Systems Architect, and not a Software Engineer anymore. I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do? I prefer how my company career ladder, where there are no architect titles. Each level is 'fractal', containing almost the same responsibilities but at different scales and scope. So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service, while as a Staff Engineer you will oversee the architecture of a system spanning several teams. This way architecture is everyone's responsibility, and people have a natural path to grow their skills.

tetek 4y ago
Of course I'm not just architecting everything by myself, and making decisions all day long. Every Software Engineer still makes a lot of architectural decisions on different levels - exactly how you described. There's just a lot of other activities that Systems Engineer does (and still consults them with the team, but is at the end responsible for doing them). (all the bullet points in "The realization")
cryptozeus 4y ago
No, architects also code. Google “lean architect”
humbleMouse 4y ago
Cwizard 4y ago
Did you read the article? The description the OP gives does not sound like the situation you describe
pjmlp 4y ago
We never left those days on the enterprise space.
swatcoder 4y ago
It might be harmful if it was universal, but it's really just another facet of the normal, perpetual tension between generalization and specialization. Large organizations working on grand projects benefit from slicing up roles so that experienced specialists can anticipate and prepare solutions for the uncommon challenges that arise in those projects (and so that the bureaucracy of large orgs can be managed at all). Meanwhile, smaller organizations, or groups working on more modestly scoped projects, benefit from generalists who can hold much of the project in their head and develop highly cohesive solutions that can be delivered efficiently. A skyscraper requires seasoned architects or its going to crumble under its own weight, but even an extensive residential home renovation can be planned by a general contractor and done way cheaper than if many specialists were brought in. It sounds like there are some organizations/projects you wouldn't want to be involved with, but that doesn't mean that those organizations/projects are doing something harmful or regressive. They just don't suit the way you want to approach work.

balentio 4y ago

Most folks define their title by what they are paid to do. It follows then that if you are not paid to do anything, then you don't have the title, right? Well, if that's not quite right, then you haven't really become anything but rather you are someone who is mucking about with systems and happen to be being paid for it.

spaetzleesser 4y ago

Whenever I hear the title "Architect" I have to think about certain segments of the IT department at my company and also the participants at my scrum master training. Both were full of "senior architects" that were good at talking but were so out of touch with reality that they couldn't contribute anything real. It's pretty easy to architect a system if you ignore the little, annoying details that make an implementation difficult. And to know what's hard you need hands-on implementation experience. These guys get away with their nonsense because often the implementation is outsourced to developers who just implement what they are told to do without asking questions. Never mind that the bad design decisions make the system much slower than it has to be and takes way more effort to implement.

hu3 4y ago
Gotta love those over-engineered projects where to add a text field to an existing form requires editing more than 15 files. So may layers of indirection that debugging is a nightmare and callstacks look like the full lorem ipsum.
- oaiey 4y ago
  That is overengineering and bad technical design. Can happen completely independent of the architecture profession.
  - contingencies 4y ago
    The problem with object-oriented languages is they've got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. - Joe Armstrong ... via https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
oaiey 4y ago
I am an architect. Proud of my profession. I code only for fun. I do solution architecture in a scope with 100s of impacted developers. I highly suspect that dozens of them are describing us exactly like you do. However, once they challenge us and we elaborate the problem to them, their solution usually falls apart and they align with our solution. Why: because in that scale of complexity we operate in not every engineer can maintain the necessary knowledge. I highly respect the approach of architecture in the team/agile environment and seniority in engineering. But at a given scale, someone is there to orchestrate all of it and does the overall plan. That person is a solution architect.

Melatonic 4y ago

In my opinion "architect" should probably really be the most senior role of people that are doing infrastructure. It can be applied to more specific software engineering roles but it just seems more appropriate considering the infrastructure is what everyone is using as a tool to run their various projects.

agluszak 4y ago

Greetings from "the legendary MIM faculty" ;)

joeamroo 4y ago

I prevent cross-site scripting, I monitor for DDoS attacks, emergency database rollbacks, and faulty transaction handlings. The Internet heard of it? Transfers half a petabyte of data every minute. Do you have any idea how that happens? All those YouPorn ones and zeroes streaming directly to your shitty, little smart phone day after day? Every dipshit who shits his pants if he can't get the new dubstep Skrillex remix in under 12 seconds? It's not magic, it's talent and sweat. People like me, ensuring your packets get delivered, un-sniffed. So what do I do? I make sure that one bad config on one key component doesn't bankrupt the entire fucking company. That's what the fuck I do.

nonethewiser 4y ago
- Michael Scott, The Office.