6 ms·

Something people forget about the rise of agile is how many of the practices have become mainstream. Most shops I encountered 25 years ago: - had a manual b

by michael1999 3y ago

Something people forget about the rise of agile is how many of the practices have become mainstream. Most shops I encountered 25 years ago: - had a manual build - deployed by hand - did all testing by hand - never touched code except for new features - might not even use source control But in the modern environment, we assume: - comprehensive source control, or even total BOM control - automated builds - automated tests - continuous integration - continuous refactoring - incremental design Congratulations, you are already doing half of XP! Those were all considered radical back in the day. What's left as purely "Agile" is the branded process stuff. E.g standardized meetings and titles like Scrum Master, SAF, etc. Given how prescriptive the business culture around it has become, it is a kind of anti-agile. So death to Agile, long live agile. Personally, I'm still an XP fiend at heart. Give me: a single list of prioritized work; a test suite I can trust that runs in < 15 minutes; and a dedicated team with good comms; and we'll tackle any problem you give us.

bluGill 3y ago
Most of those were considered normal best practice in watarfall shops I worked at. They were common, but we knew many not doing them and laughed at those fools.
subharmonicon 3y ago
Not my experience at all. - We happily adopted a CI system from another team in 1998. - We had a decent unit testing framework and a fair bit of automation for integration testing (although yes, some aspects of UI were hand tested periodically). - We continuously refactored and improved existing code, especially if we touched something in it or around it. The only practice I think was poor then was mixing the refactoring and new work into a single commit, which I would never do today. - I was using source control even in school in the late 80s, and I’ve never come across a team that didn’t, although I’m sure they did exist at that time. - As far as deployment, everything I’ve worked on has been shipped on a very episodic basis (anywhere from a month between shipping to a few years), so most of the automation I’ve seen there from early on was about deploying internally for testing, and that’s always been more or less automated. Although I did work at one place that tried Scrum for a little while around 2003-2005, the only practice I’ve generally seen adopted from that in the remaining years has been having periodic check-ins on status, e.g. monthly demo days to show off accomplishments from the last month. These are relatively low-prep although not impromptu, and that practice isn’t uniform. With all that said, some things I have seen change, which aren’t mentioned by parent: - Much more emphasis on developers writing tests. In the 90’s test teams were sometimes as large as development teams, and usually way behind the development team in progress. There was a belief that developers couldn’t possibly write good tests, or simply wouldn’t. - I never worked anywhere where it was “docs first”, at least not to the extent that people mean that when they talk about older development practices. We did sometimes write the equivalent of a 1-pager to get everyone aligned on what particular work was about, and had a meeting to discuss and determine if everyone was in fact aligned. I think that was very useful, and sadly what I see more often than not now is “code first” meaning that you write code and then try to use code reviews to deal with everything, which means that what could have been an hour of writing something up and an hour meeting turns into someone spending days coding something that has to be thrown away because it’s so clear during code reviews that the approach is the wrong one. So now this “agile” approach means a throwaway work sometimes because people aren’t communicating in any meaningful way before writing code. [edited formatting]
- michael1999 3y ago
  XP never claimed to invent any of those practices. But they were unevenly distributed. I note that 1998 was ~25 years ago, so I'm not sure we disagree here. You just described CI as a new-ish practice you adopted from another team. I'm happy you had your experience. But I will testify I've seen lots of places that would slap your hand if you touched code to "clean it up". When "testing" was two people running a week-long manual script, narrowing the scope of changes was a required practice! As for developer testing: a core idea of XP is that we developers should take responsibility for the quality of the code we ship. Test-first was a practice to support that goal. Automated testing wasn't completely unknown, but many shops had zero automated tests. Or if they had automated tests, they were clicky-clicky tests run by a ui-automation tool, and were written by the testing team. Developer written tests were rare back then. I remember interning at MS in the 90s and how proud they were that their target tester:developer ratio was >1! The creation of JUnit was an important historical event for us. As for the lack of communication and wasted work: I completely agree. Turning a story/use-case problem statement into a concrete design has to happen sometime. We used to do a whole-team tasking meeting for each story we opened, and group estimate of our tasks on the whiteboard. Combined with pairing, whole team rotation, and continuous integration, there was very little opportunity for someone to get lost.
  - subharmonicon 3y ago
    I don’t doubt anything you’re saying, either. I was just providing another perspective. I think one thing I have seen over the years is people making extreme assumptions based on experiences they hear via the grapevine, and seeing more perspectives seems like one way to counter that. For example I have seen the claim here that CI systems were somehow new in the same timeframe that XP and Scrum were first written about and being adopted, and that testing was always an afterthought back then, etc. If anything I think we were often building more reliable and higher quality systems in that timeframe, if only because there was more alignment that testing and fixing bugs mattered. Today I get responses like “quality isn’t just about bugs” if I am critical about the quality of a particular product. As if what users want is 2N as many features, none of which quite work rather than N features that are all solid.
rapind 3y ago
Source control was definitely a thing, but I think the popularity of git and what grew from it deserves a lot of the credit for current build pipelines. IMO a lot more than agile or xp.
- michael1999 3y ago
  That's not how it happened. Git wasn't even invented until 2005, and Thoughtworks released CruiseControl (the first packaged CI server) in 2001. Hudson came along later (2005?) But that whole scene started on CVS and jumped to Subversion. CI was well established when git showed up.
fzeindl 3y ago
> Personally, I'm still an XP fiend at heart. Give me: a single list of prioritized work; a test suite I can trust … For me the essence of XP was always programming in pairs in front of a computer. That notion was somehow lost in the article and you also don‘t mention it.
- michael1999 3y ago
  Pairing was the practice people talked about the most, because it caused the most friction. I'm not sure that I would say it was the essence of XP -- I think XP was about taking collective responsibility, sustainable pace, and feedback. Pairing was a practice to support those values. I miss pairing. It is powerful on its own, and is shockingly effective when combined with the whole XP thing. But you can still get a ton of value from the rest to XP even with async comms.
  - fzeindl 3y ago
    I miss it too, I did almost every CS class at university paired and we generally scored very well.
oxfordmale 3y ago
None of these has anything to do with Agile nor XP. This are CI/CD technologies, and perfectly fit into a waterfall deployment too.
- deleted 3y ago
  [deleted]
- michael1999 3y ago
  So why had nobody released a CI server until Thoughtworks published CruiseControl in 2001?
  - oxfordmale 3y ago
    Companies like Microsoft already had internal build servers for Windows. From Wiki: "In 1994, Grady Booch used the phrase continuous integration in Object-Oriented Analysis and Design with Applications (2nd edition)[8] to explain how, when developing using micro processes." Cruisecontrol and XP were building on an already known concept. By 2001 hardware also became cheap enough to have a dedicated build servers. The XP community sometimes give a distorted view of waterfall. It present waterfall as a dogmatic approach run by incompetents. In reality waterfall was a lot more advanced, with testing, build tools and a phased delivery approach.
PH95VuimJjqBqy 3y ago
source control is not XP. One of the reasons people dislike staunch agilistas is because they try to lay claim to anything that turns out to be a good idea and deny everything that turns out to be a bad idea. The Joel test was published in 2000, the very first question on that test is "do you use source control". It was known back in the 90's that using source control was a good idea. https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/ https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
- deleted 3y ago
  [deleted]
- michael1999 3y ago
  Sure. XP didn’t invent anything. But Joel published that because those practices were not widely known/practiced. That was his contribution/reaction to that industry conversation - single backlog, continuous integration, tests, etc. They really were uncommon and controversial in many shops. From Joel: “The truth is that most software organizations are running with a score of 2 or 3, and they need serious help”
  - PH95VuimJjqBqy 3y ago
    the value of source control was understood before XP even existed, it just hadn't been widely disseminated, which is _why_ Joel created his test in the first place. XP and agile had absolutely nothing to do with the use, or lack thereof, of source control. At best you could argue there might be a correlation due to the sophistication of the shop itself.
    - michael1999 3y ago
      Completely agree. SCCS goes back to 1973. I meant my comment about the lack of source control as a remark on how many shops had no hygiene at all, not that XP was the inventor of it. Joel wasn't a fan of the agile hype, and published his test in opposition to it. I agree with his assertion that much of the value was in dev hygiene: builds, tests, automation, and continuous delivery. And those could be adopted separately from the rest of "Agile". But agile had the buzz. Was the buzz part cause of broader adoption? I believe yes. Agile consultancies inflicted build hygiene on reluctant teams in the same way that Rational Rose salespeople inflicted RUP on them. And I'd argue those early Agile consulting engagements were better value-for-money than those RUP licenses. I feel more strongly that continuous integration and continuous testing became widespread as part of the early agile movement. Again - none of those practices were invented there. Booch published his "Booch Method" in 1991, and Microsoft had monster overnight automated testing when I was there in 1992. But Beck wrote SUnit in 1995, and then JUnit when the Smalltalk diaspora fled to Java. XP was about evangelizing practices the authors had found to work and turning them to 11. CruiseControl came out of Thoughtworks in 2001, and the XP book literally coined the term "continuous integration".

bluGill 3y ago

Most of those were considered normal best practice in watarfall shops I worked at. They were common, but we knew many not doing them and laughed at those fools.

subharmonicon 3y ago

Not my experience at all. - We happily adopted a CI system from another team in 1998. - We had a decent unit testing framework and a fair bit of automation for integration testing (although yes, some aspects of UI were hand tested periodically). - We continuously refactored and improved existing code, especially if we touched something in it or around it. The only practice I think was poor then was mixing the refactoring and new work into a single commit, which I would never do today. - I was using source control even in school in the late 80s, and I’ve never come across a team that didn’t, although I’m sure they did exist at that time. - As far as deployment, everything I’ve worked on has been shipped on a very episodic basis (anywhere from a month between shipping to a few years), so most of the automation I’ve seen there from early on was about deploying internally for testing, and that’s always been more or less automated. Although I did work at one place that tried Scrum for a little while around 2003-2005, the only practice I’ve generally seen adopted from that in the remaining years has been having periodic check-ins on status, e.g. monthly demo days to show off accomplishments from the last month. These are relatively low-prep although not impromptu, and that practice isn’t uniform. With all that said, some things I have seen change, which aren’t mentioned by parent: - Much more emphasis on developers writing tests. In the 90’s test teams were sometimes as large as development teams, and usually way behind the development team in progress. There was a belief that developers couldn’t possibly write good tests, or simply wouldn’t. - I never worked anywhere where it was “docs first”, at least not to the extent that people mean that when they talk about older development practices. We did sometimes write the equivalent of a 1-pager to get everyone aligned on what particular work was about, and had a meeting to discuss and determine if everyone was in fact aligned. I think that was very useful, and sadly what I see more often than not now is “code first” meaning that you write code and then try to use code reviews to deal with everything, which means that what could have been an hour of writing something up and an hour meeting turns into someone spending days coding something that has to be thrown away because it’s so clear during code reviews that the approach is the wrong one. So now this “agile” approach means a throwaway work sometimes because people aren’t communicating in any meaningful way before writing code. [edited formatting]

michael1999 3y ago
XP never claimed to invent any of those practices. But they were unevenly distributed. I note that 1998 was ~25 years ago, so I'm not sure we disagree here. You just described CI as a new-ish practice you adopted from another team. I'm happy you had your experience. But I will testify I've seen lots of places that would slap your hand if you touched code to "clean it up". When "testing" was two people running a week-long manual script, narrowing the scope of changes was a required practice! As for developer testing: a core idea of XP is that we developers should take responsibility for the quality of the code we ship. Test-first was a practice to support that goal. Automated testing wasn't completely unknown, but many shops had zero automated tests. Or if they had automated tests, they were clicky-clicky tests run by a ui-automation tool, and were written by the testing team. Developer written tests were rare back then. I remember interning at MS in the 90s and how proud they were that their target tester:developer ratio was >1! The creation of JUnit was an important historical event for us. As for the lack of communication and wasted work: I completely agree. Turning a story/use-case problem statement into a concrete design has to happen sometime. We used to do a whole-team tasking meeting for each story we opened, and group estimate of our tasks on the whiteboard. Combined with pairing, whole team rotation, and continuous integration, there was very little opportunity for someone to get lost.
- subharmonicon 3y ago
  I don’t doubt anything you’re saying, either. I was just providing another perspective. I think one thing I have seen over the years is people making extreme assumptions based on experiences they hear via the grapevine, and seeing more perspectives seems like one way to counter that. For example I have seen the claim here that CI systems were somehow new in the same timeframe that XP and Scrum were first written about and being adopted, and that testing was always an afterthought back then, etc. If anything I think we were often building more reliable and higher quality systems in that timeframe, if only because there was more alignment that testing and fixing bugs mattered. Today I get responses like “quality isn’t just about bugs” if I am critical about the quality of a particular product. As if what users want is 2N as many features, none of which quite work rather than N features that are all solid.

rapind 3y ago

Source control was definitely a thing, but I think the popularity of git and what grew from it deserves a lot of the credit for current build pipelines. IMO a lot more than agile or xp.

michael1999 3y ago
That's not how it happened. Git wasn't even invented until 2005, and Thoughtworks released CruiseControl (the first packaged CI server) in 2001. Hudson came along later (2005?) But that whole scene started on CVS and jumped to Subversion. CI was well established when git showed up.

fzeindl 3y ago

> Personally, I'm still an XP fiend at heart. Give me: a single list of prioritized work; a test suite I can trust … For me the essence of XP was always programming in pairs in front of a computer. That notion was somehow lost in the article and you also don‘t mention it.

michael1999 3y ago
Pairing was the practice people talked about the most, because it caused the most friction. I'm not sure that I would say it was the essence of XP -- I think XP was about taking collective responsibility, sustainable pace, and feedback. Pairing was a practice to support those values. I miss pairing. It is powerful on its own, and is shockingly effective when combined with the whole XP thing. But you can still get a ton of value from the rest to XP even with async comms.
- fzeindl 3y ago
  I miss it too, I did almost every CS class at university paired and we generally scored very well.

oxfordmale 3y ago

None of these has anything to do with Agile nor XP. This are CI/CD technologies, and perfectly fit into a waterfall deployment too.

deleted 3y ago
[deleted]
michael1999 3y ago
So why had nobody released a CI server until Thoughtworks published CruiseControl in 2001?
- oxfordmale 3y ago
  Companies like Microsoft already had internal build servers for Windows. From Wiki: "In 1994, Grady Booch used the phrase continuous integration in Object-Oriented Analysis and Design with Applications (2nd edition)[8] to explain how, when developing using micro processes." Cruisecontrol and XP were building on an already known concept. By 2001 hardware also became cheap enough to have a dedicated build servers. The XP community sometimes give a distorted view of waterfall. It present waterfall as a dogmatic approach run by incompetents. In reality waterfall was a lot more advanced, with testing, build tools and a phased delivery approach.

PH95VuimJjqBqy 3y ago

source control is not XP. One of the reasons people dislike staunch agilistas is because they try to lay claim to anything that turns out to be a good idea and deny everything that turns out to be a bad idea. The Joel test was published in 2000, the very first question on that test is "do you use source control". It was known back in the 90's that using source control was a good idea. https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-steps-to-better-code/ https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...

deleted 3y ago
[deleted]
michael1999 3y ago
Sure. XP didn’t invent anything. But Joel published that because those practices were not widely known/practiced. That was his contribution/reaction to that industry conversation - single backlog, continuous integration, tests, etc. They really were uncommon and controversial in many shops. From Joel: “The truth is that most software organizations are running with a score of 2 or 3, and they need serious help”
- PH95VuimJjqBqy 3y ago
  the value of source control was understood before XP even existed, it just hadn't been widely disseminated, which is _why_ Joel created his test in the first place. XP and agile had absolutely nothing to do with the use, or lack thereof, of source control. At best you could argue there might be a correlation due to the sophistication of the shop itself.
  - michael1999 3y ago
    Completely agree. SCCS goes back to 1973. I meant my comment about the lack of source control as a remark on how many shops had no hygiene at all, not that XP was the inventor of it. Joel wasn't a fan of the agile hype, and published his test in opposition to it. I agree with his assertion that much of the value was in dev hygiene: builds, tests, automation, and continuous delivery. And those could be adopted separately from the rest of "Agile". But agile had the buzz. Was the buzz part cause of broader adoption? I believe yes. Agile consultancies inflicted build hygiene on reluctant teams in the same way that Rational Rose salespeople inflicted RUP on them. And I'd argue those early Agile consulting engagements were better value-for-money than those RUP licenses. I feel more strongly that continuous integration and continuous testing became widespread as part of the early agile movement. Again - none of those practices were invented there. Booch published his "Booch Method" in 1991, and Microsoft had monster overnight automated testing when I was there in 1992. But Beck wrote SUnit in 1995, and then JUnit when the Smalltalk diaspora fled to Java. XP was about evangelizing practices the authors had found to work and turning them to 11. CruiseControl came out of Thoughtworks in 2001, and the XP book literally coined the term "continuous integration".