8 ms·

Internal RFCs saved us months of wasted work

102 points by romannikolaev 11mo ago

pstuart 11mo ago
This seems to be a worthwhile exercise to explore -- formal but lightweight, and a lodestone during development. That said, a ticketing system should ostensibly offer this same effect, but in my experience they're often populated with brief titles and maybe a short paragraph on expectations.
- romannikolaev 11mo ago
  We usually use this before tickets are even created. In many cases, an RFC can lead to an epic-level ticket that includes multiple user stories. In other cases, it can be very tactical. I think you are right, probably it can be expressed directly in the form of a ticket. The discussion well happen in the comments to the ticket in this case.
  - pstuart 11mo ago
    A benefit of the RFC approach is if it lives in the repo itself the documentation is along side the code. Anything that will prompt stakeholders to engage and clarify expectations is a win. It's hard to make this happen even when everybody is philosophically aligned on the need to do so, so reframing it as an RFC could be quite the useful "One Weird Trick" ;-).
- observationist 11mo ago
  Ticketing systems inevitably get Goodharted. Everyone starts in good faith, then the manager starts using the number of tickets closed, touched, etc as a proxy for work being done, then the agents replace number of tickets with things actually accomplished.
- swiftcoder 11mo ago
  > That said, a ticketing system should ostensibly offer this same effect If the ticketing system were private to the engineers, it probably would serve the same purpose. But inevitably ticketing systems have wider visibility, and become a mechanism for signalling progress to management/product/sales... at which point engineers are actively incentivised not to put actual data in the ticketing system
- baud147258 11mo ago
  > a short paragraph on expectations. or worse, multiple bulleted paragraphs fulls of expectations that have little to do with how the software actually work and is used by the actual users.
cjs_ac 11mo ago
> The RFC approach has several advantages over verbal alignment. First of all, it is more precise. The need to write forces the author to clearly structure their thoughts into a coherent logical narrative. While writing, the author has time to examine their proposed solution from different angles and clearly see pros and cons of it. > Another advantage of the document over verbal explanation is that a well-written RFC leaves little room for misinterpretation. It can include diagrams, examples, or calculations to illustrate and support the idea. > Finally, we can return and reread the RFC later. Human memory is unreliable; already after a day, details that were crystal clear in one’s mind start to get blurry. When these details are written down, it is easy to review them at any time. ‘You have to write things down, because spoken words disappear into the air,’ was one of the first bits of feedback I received in my teacher training. > The most common objection is that writing proposals is “a waste of time” compared to writing code. The extra time spent writing is actually spent thinking.
- pjc50 11mo ago
  >> The most common objection is that writing proposals is “a waste of time” compared to writing code. > The extra time spent writing is actually spent thinking. Common theme for decades is "we can save a few days of planning with just a few weeks of programming". But then there's the darker realization that sometimes the people you are working for are incapable of reasoning about planning artefacts or understanding how the system will look or operate simply from a document. So you need to present the system in small iterative chunks and repeatedly re-align expectations with reality: Agile. And sometimes you genuinely need to do exploratory work which doesn't fit into a planning framework - actual research!
  - physicles 11mo ago
    > sometimes the people you are working for are incapable of reasoning about planning artefacts or understanding how the system will look or operate simply from a document I’m wrestling with this now. Over my career I’ve seen a strong correlation between good writers and good software engineers, but not everyone fits this mold. Shorter cycles and more chances for communication and feedback are helpful here.
sneak 11mo ago
Write a spec, or the developers will write one for you, in a much less clear language.
- afandian 11mo ago
  It's not a foregone conclusion. I've had great success writing BDD-style literate tests (not using a framework, just comments that follow a particular grammar). They've allowed verbally negotiating the approximate expected specification, and dealing with the realities of what comes up once you start writing code. But you end up with a spec, expressed as a number of BDD scenarios. Obviously this does't work for all software. But it suits systems with end-to-end behaviour and sequences of actions. So it's possible to leave it to developers, but expect a high quality spec along with the code.
StackBPoppin 11mo ago
I've tried suggesting this for my team since there are constant complaints of lack of communication. However, the response to this is "we have Teams/Jira/Confluence", but 99% of Jira tickets have no comments for clarification, Confluence has articles that are out of date by 5 years and Teams is never used for clarifying requirements.
- zihotki 11mo ago
  It's not the lack of communication. IMO, it's the lack of team culture. Keeping documentation up to date is something only the team could do. And it can't be solved by using Confluence/Wiki/mailing lists tools.
  - YouAreWRONGtoo 11mo ago
    [dead]
- koliber 11mo ago
  That's like me trying to get my son to wash his hair and him responding by saying "We have shampoo in the shower." I am right and my son is right, but his hair is still not washed. I became a more effective manager and a better father when I learned how to talk to him better.
  - pdimitar 11mo ago
    Would you share how? Your comment leaves with a cliff-hanger. I also don't get your son's response at all. How is he contradicting you at all and how does that lead to unwashed hair?
    - consz 11mo ago
      He's saying that there's shampoo in the shower but he didn't use it (implied) -- however, the question wasn't about the presence of shampoo in the shower.
jojobas 11mo ago
This guy invented a spec?
- YouAreWRONGtoo 11mo ago
  [dead]
- nobleach 11mo ago
  In our org, an RFC precedes a tech spec. The RFC literally is the "let's formally talk about this before we nail down a specification". For smaller specs, annotated comments can serve this purpose. Before this process, what we had found was no one was paying attention to tech discussions in our eng slack channels. Having an RFC gave us an inflection point where we could point back and say, "an official discussion happened, we decided to move forward with a spec".
deleted 11mo ago
[deleted]
aranw 11mo ago
Been trying to decide whether adopting a traditional RFC process or Oxide's RFD (https://rfd.shared.oxide.computer/rfd/0001 https://rfd.shared.oxide.computer/rfd/0001) would better suit my team. We're using ADRs at the moment but we've ended up mixing a discussion like process into it and review process and using ADRs more like RFCs/RFDs
koliber 11mo ago
Love the title. Reminds me of one of my favorite quotes: "The single biggest problem in communication is the illusion that it has taken place." This is what user stories were supposed to accomplish in a more lightweight way. The whole scrum DoR (definition of ready) status means that something is clear and ready for development. Stories are written and are sent to the engineering team for clarification. This is where the comments are supposed to come in. There is a clear step for clarification of stories, before the story is ready for development. It gets marked as DoR when that clarification is done. It does not matter if you use RFCs, user stories, or hallway conversations as your process of clarifying work. If it does not work, it does not work. Any way you can get your teams to communicate more clearly is great.
- physicles 11mo ago
  > "The single biggest problem in communication is the illusion that it has taken place." Love this! Corollary: when you have too many meetings, that’s easy to notice. When you don’t have enough meetings, that’s harder to notice. I’m in the process of carefully adding meetings and process to our small team of 6 (we had a PM from a large company drop in a few years ago and haphazardly add a bunch of process, and it didn’t really help). We’re fully remote and have a daily huddle and, on average, 1 hour of meetings a week. It turns out this isn’t enough. So far, each bit of communication we’ve added has resulted in better outcomes and higher morale because we feel more like a team.
monkeydust 11mo ago
We use them internally. Works well but its not a replacement for a business requirement specification, however, it can help in the creation of a (better) one with greater buy-in from all stakeholders
pdimitar 11mo ago
Linear should take notes. In my previous consulting and termed engagements the lack of a good standardized spec templates has been a problem. Of course we ultimately made those but it took much more in-fighting than it should have. If the platform already offers a few baked templates then discussions are much quicker resolved. Same as auto-formatting of code; metric tons of tiresome bikeshedding disappear almost overnight.
phpnode 11mo ago
Not a panacea for broken culture. I've just left a company that used to have an internal RFC process and it was a very significant barrier to progress that stifled innovation, led to breakdown of working relationships and caused the most productive engineers to run for the exits. RFC is a request for comments, and it turns out you have to be really careful about the kinds of comments you solicit and who from. As soon as you ask people to comment you are setting an expectation that you will take their feedback onboard and address their points, but there’s a real asymmetry here - it is much easier to leave a critical comment, or to ask a question, than it is to address the concerns or answer the question. This asymmetrical nature puts much more work on the shoulders of the RFC author. Similarly, the people writing the RFC have typically thought much harder and longer and deeper about the problem than the people giving feedback on it. This leads to authors having to re-explain their thinking in detail, covering points that they’d omitted for brevity or because they are obvious to those with a good understanding of the problem. Suddenly, the task changes from shipping the feature or making the change to achieving consensus on the RFC. I have seen that process take months - far longer, and being far more expensive than just doing the work would be. The worst part is - no one is in the wrong! The authors write the RFC in good faith, the commenters review the RFC in good faith, and they rightly expect that any problems they identify are addressed. But the whole process can be soul crushingly slow, painful and full of professional conflict. I’m not saying that RFCs are bad overall, but you have to have a culture of accountability, pragmatism and getting things done to actually make this process work.
- Wilder7977 11mo ago
  In my organization we have RFCs, PRDs, ADRs etc. and I would say that the process is fairly broken. That said, I think what you mention is an important but not the only failure mode of a proposals process. In some cases I have seen, people use RFCs to steamroll decisions when they are the only stakeholders. Here the waste comes from the fact that the proposal becomes just a bureaucratic step or a way to diffuse responsibility. In the case you mention (which I have seen many times) I would say the general issue is that the goals and the constraints are not qualified sufficiently. If that's the case, then there are only 2 cases: there is an objective way to measure if an objection or comment makes sense or not, or it is subjective. If it's objective, then consensus is easy to reach, if it's subjective, it needs to be acknowledged and usually the decisions falls on those who are responsible for the outcome (e.g., the team who needs to maintain or build the thing). Of course, the debate can move to constraints, goals or even ways to measure, but these are generally more straightforward.
artembugara 11mo ago
We started doing quarterly RFC at Newscatcher, and it was a big game-changer. We're entirely remote. I got this idea from Netflix's founder's book "No Rules Rules" (highly recommend it) Overall, I think the main idea is: context is what matters, and RFC helps you get your (mine, I'm the founder) vision into people's heads a bit more. Therefore, people can be more autonomous and move on faster.
raw_anon_1111 11mo ago
I’m the first person to say Amazon (AWS) was a shitty place to work. But I did learn a lot while there working in Professional Services. I still use modified versions of the PRFAQ when starting a consulting project now that I lead projects at a third party consulting company. https://productstrategy.co/working-backwards-the-amazon-prfaq-for-product-innovation/ https://productstrategy.co/working-backwards-the-amazon-prfa...
Scubabear68 11mo ago
My interpretation is that this had little to do with specifics of RFCs, and everything to do with the verbal culture described. I don’t care what format you use in terms of both process and also literal format. Just write shit down.
pityJuke 11mo ago
My current struggle is getting people to engage with my RFCs: that is, leaving comments. I certainly know they're not perfect, and I'd much rather know that before going into a meeting where we discuss the RFC. Especially so I can do any asynchronous research, or just spend some damned time thinking about it. That being said, this is improving in my team. And I think the stakeholders above me appreciate it.
deleted 11mo ago
[deleted]
tedchs 11mo ago
My favorite section in an RFC is "alternatives considered". I've seen many times that an option that had been initially discarded became the solution after review and discussion. It's also a great way to answer later questions about "why didn't you do X instead".

102 points by romannikolaev 11mo ago

This seems to be a worthwhile exercise to explore -- formal but lightweight, and a lodestone during development. That said, a ticketing system should ostensibly offer this same effect, but in my experience they're often populated with brief titles and maybe a short paragraph on expectations.

romannikolaev 11mo ago
We usually use this before tickets are even created. In many cases, an RFC can lead to an epic-level ticket that includes multiple user stories. In other cases, it can be very tactical. I think you are right, probably it can be expressed directly in the form of a ticket. The discussion well happen in the comments to the ticket in this case.
- pstuart 11mo ago
  A benefit of the RFC approach is if it lives in the repo itself the documentation is along side the code. Anything that will prompt stakeholders to engage and clarify expectations is a win. It's hard to make this happen even when everybody is philosophically aligned on the need to do so, so reframing it as an RFC could be quite the useful "One Weird Trick" ;-).
observationist 11mo ago
Ticketing systems inevitably get Goodharted. Everyone starts in good faith, then the manager starts using the number of tickets closed, touched, etc as a proxy for work being done, then the agents replace number of tickets with things actually accomplished.
swiftcoder 11mo ago
> That said, a ticketing system should ostensibly offer this same effect If the ticketing system were private to the engineers, it probably would serve the same purpose. But inevitably ticketing systems have wider visibility, and become a mechanism for signalling progress to management/product/sales... at which point engineers are actively incentivised not to put actual data in the ticketing system
baud147258 11mo ago
> a short paragraph on expectations. or worse, multiple bulleted paragraphs fulls of expectations that have little to do with how the software actually work and is used by the actual users.

cjs_ac 11mo ago

> The RFC approach has several advantages over verbal alignment. First of all, it is more precise. The need to write forces the author to clearly structure their thoughts into a coherent logical narrative. While writing, the author has time to examine their proposed solution from different angles and clearly see pros and cons of it. > Another advantage of the document over verbal explanation is that a well-written RFC leaves little room for misinterpretation. It can include diagrams, examples, or calculations to illustrate and support the idea. > Finally, we can return and reread the RFC later. Human memory is unreliable; already after a day, details that were crystal clear in one’s mind start to get blurry. When these details are written down, it is easy to review them at any time. ‘You have to write things down, because spoken words disappear into the air,’ was one of the first bits of feedback I received in my teacher training. > The most common objection is that writing proposals is “a waste of time” compared to writing code. The extra time spent writing is actually spent thinking.

pjc50 11mo ago
>> The most common objection is that writing proposals is “a waste of time” compared to writing code. > The extra time spent writing is actually spent thinking. Common theme for decades is "we can save a few days of planning with just a few weeks of programming". But then there's the darker realization that sometimes the people you are working for are incapable of reasoning about planning artefacts or understanding how the system will look or operate simply from a document. So you need to present the system in small iterative chunks and repeatedly re-align expectations with reality: Agile. And sometimes you genuinely need to do exploratory work which doesn't fit into a planning framework - actual research!
- physicles 11mo ago
  > sometimes the people you are working for are incapable of reasoning about planning artefacts or understanding how the system will look or operate simply from a document I’m wrestling with this now. Over my career I’ve seen a strong correlation between good writers and good software engineers, but not everyone fits this mold. Shorter cycles and more chances for communication and feedback are helpful here.

sneak 11mo ago

Write a spec, or the developers will write one for you, in a much less clear language.

afandian 11mo ago
It's not a foregone conclusion. I've had great success writing BDD-style literate tests (not using a framework, just comments that follow a particular grammar). They've allowed verbally negotiating the approximate expected specification, and dealing with the realities of what comes up once you start writing code. But you end up with a spec, expressed as a number of BDD scenarios. Obviously this does't work for all software. But it suits systems with end-to-end behaviour and sequences of actions. So it's possible to leave it to developers, but expect a high quality spec along with the code.

StackBPoppin 11mo ago

I've tried suggesting this for my team since there are constant complaints of lack of communication. However, the response to this is "we have Teams/Jira/Confluence", but 99% of Jira tickets have no comments for clarification, Confluence has articles that are out of date by 5 years and Teams is never used for clarifying requirements.

zihotki 11mo ago
It's not the lack of communication. IMO, it's the lack of team culture. Keeping documentation up to date is something only the team could do. And it can't be solved by using Confluence/Wiki/mailing lists tools.
- YouAreWRONGtoo 11mo ago
  [dead]
koliber 11mo ago
That's like me trying to get my son to wash his hair and him responding by saying "We have shampoo in the shower." I am right and my son is right, but his hair is still not washed. I became a more effective manager and a better father when I learned how to talk to him better.
- pdimitar 11mo ago
  Would you share how? Your comment leaves with a cliff-hanger. I also don't get your son's response at all. How is he contradicting you at all and how does that lead to unwashed hair?
  - consz 11mo ago
    He's saying that there's shampoo in the shower but he didn't use it (implied) -- however, the question wasn't about the presence of shampoo in the shower.

jojobas 11mo ago

This guy invented a spec?

YouAreWRONGtoo 11mo ago
[dead]
nobleach 11mo ago
In our org, an RFC precedes a tech spec. The RFC literally is the "let's formally talk about this before we nail down a specification". For smaller specs, annotated comments can serve this purpose. Before this process, what we had found was no one was paying attention to tech discussions in our eng slack channels. Having an RFC gave us an inflection point where we could point back and say, "an official discussion happened, we decided to move forward with a spec".

deleted 11mo ago

[deleted]

aranw 11mo ago

Been trying to decide whether adopting a traditional RFC process or Oxide's RFD (https://rfd.shared.oxide.computer/rfd/0001 https://rfd.shared.oxide.computer/rfd/0001) would better suit my team. We're using ADRs at the moment but we've ended up mixing a discussion like process into it and review process and using ADRs more like RFCs/RFDs

koliber 11mo ago

Love the title. Reminds me of one of my favorite quotes: "The single biggest problem in communication is the illusion that it has taken place." This is what user stories were supposed to accomplish in a more lightweight way. The whole scrum DoR (definition of ready) status means that something is clear and ready for development. Stories are written and are sent to the engineering team for clarification. This is where the comments are supposed to come in. There is a clear step for clarification of stories, before the story is ready for development. It gets marked as DoR when that clarification is done. It does not matter if you use RFCs, user stories, or hallway conversations as your process of clarifying work. If it does not work, it does not work. Any way you can get your teams to communicate more clearly is great.

physicles 11mo ago
> "The single biggest problem in communication is the illusion that it has taken place." Love this! Corollary: when you have too many meetings, that’s easy to notice. When you don’t have enough meetings, that’s harder to notice. I’m in the process of carefully adding meetings and process to our small team of 6 (we had a PM from a large company drop in a few years ago and haphazardly add a bunch of process, and it didn’t really help). We’re fully remote and have a daily huddle and, on average, 1 hour of meetings a week. It turns out this isn’t enough. So far, each bit of communication we’ve added has resulted in better outcomes and higher morale because we feel more like a team.

monkeydust 11mo ago

We use them internally. Works well but its not a replacement for a business requirement specification, however, it can help in the creation of a (better) one with greater buy-in from all stakeholders

pdimitar 11mo ago

Linear should take notes. In my previous consulting and termed engagements the lack of a good standardized spec templates has been a problem. Of course we ultimately made those but it took much more in-fighting than it should have. If the platform already offers a few baked templates then discussions are much quicker resolved. Same as auto-formatting of code; metric tons of tiresome bikeshedding disappear almost overnight.

phpnode 11mo ago

Not a panacea for broken culture. I've just left a company that used to have an internal RFC process and it was a very significant barrier to progress that stifled innovation, led to breakdown of working relationships and caused the most productive engineers to run for the exits. RFC is a request for comments, and it turns out you have to be really careful about the kinds of comments you solicit and who from. As soon as you ask people to comment you are setting an expectation that you will take their feedback onboard and address their points, but there’s a real asymmetry here - it is much easier to leave a critical comment, or to ask a question, than it is to address the concerns or answer the question. This asymmetrical nature puts much more work on the shoulders of the RFC author. Similarly, the people writing the RFC have typically thought much harder and longer and deeper about the problem than the people giving feedback on it. This leads to authors having to re-explain their thinking in detail, covering points that they’d omitted for brevity or because they are obvious to those with a good understanding of the problem. Suddenly, the task changes from shipping the feature or making the change to achieving consensus on the RFC. I have seen that process take months - far longer, and being far more expensive than just doing the work would be. The worst part is - no one is in the wrong! The authors write the RFC in good faith, the commenters review the RFC in good faith, and they rightly expect that any problems they identify are addressed. But the whole process can be soul crushingly slow, painful and full of professional conflict. I’m not saying that RFCs are bad overall, but you have to have a culture of accountability, pragmatism and getting things done to actually make this process work.

Wilder7977 11mo ago
In my organization we have RFCs, PRDs, ADRs etc. and I would say that the process is fairly broken. That said, I think what you mention is an important but not the only failure mode of a proposals process. In some cases I have seen, people use RFCs to steamroll decisions when they are the only stakeholders. Here the waste comes from the fact that the proposal becomes just a bureaucratic step or a way to diffuse responsibility. In the case you mention (which I have seen many times) I would say the general issue is that the goals and the constraints are not qualified sufficiently. If that's the case, then there are only 2 cases: there is an objective way to measure if an objection or comment makes sense or not, or it is subjective. If it's objective, then consensus is easy to reach, if it's subjective, it needs to be acknowledged and usually the decisions falls on those who are responsible for the outcome (e.g., the team who needs to maintain or build the thing). Of course, the debate can move to constraints, goals or even ways to measure, but these are generally more straightforward.

artembugara 11mo ago

We started doing quarterly RFC at Newscatcher, and it was a big game-changer. We're entirely remote. I got this idea from Netflix's founder's book "No Rules Rules" (highly recommend it) Overall, I think the main idea is: context is what matters, and RFC helps you get your (mine, I'm the founder) vision into people's heads a bit more. Therefore, people can be more autonomous and move on faster.

raw_anon_1111 11mo ago

I’m the first person to say Amazon (AWS) was a shitty place to work. But I did learn a lot while there working in Professional Services. I still use modified versions of the PRFAQ when starting a consulting project now that I lead projects at a third party consulting company. https://productstrategy.co/working-backwards-the-amazon-prfaq-for-product-innovation/ https://productstrategy.co/working-backwards-the-amazon-prfa...

Scubabear68 11mo ago

My interpretation is that this had little to do with specifics of RFCs, and everything to do with the verbal culture described. I don’t care what format you use in terms of both process and also literal format. Just write shit down.

pityJuke 11mo ago

My current struggle is getting people to engage with my RFCs: that is, leaving comments. I certainly know they're not perfect, and I'd much rather know that before going into a meeting where we discuss the RFC. Especially so I can do any asynchronous research, or just spend some damned time thinking about it. That being said, this is improving in my team. And I think the stakeholders above me appreciate it.

deleted 11mo ago

[deleted]

tedchs 11mo ago

My favorite section in an RFC is "alternatives considered". I've seen many times that an option that had been initially discarded became the solution after review and discussion. It's also a great way to answer later questions about "why didn't you do X instead".