9 ms·

Former OS X developer here. I'd say the biggest change in the development methodology happened when Bertrand Serlet was replaced with Craig Federighi. With Be

Former OS X developer here. I'd say the biggest change in the development methodology happened when Bertrand Serlet was replaced with Craig Federighi. With Bertrand, we would move in giant monolithic releases where every group would just dump in whatever they had ready and the whole thing would get released with nightly builds. With SnowLeopard in particular, I remember three dozen releases in a row where Xcode was unusable due to obj-c garbage collection issues. Random stuff you didn't expect like CoreGraphics would have showstopper issues and then we'd report it and it would get fixed by the next week. This resulted in extremely late releases that had a ton of bugs that we piled patches onto as time went on. Craig moved the organization onto a sprint system, where we would develop new features for 2 weeks and then spend a week fixing bugs. After 10 or 12 or 16 of these cycles, we would deem it ready and ship it out. I felt this produced more stable but more conservative software. It seemed like giant rewrites and massive features would be very difficult to introduce and if they did get done, wouldn't happen until two thirds or so into the release cycle. On the other hand, Craig has consistently been able to release on time with most of the features promised. I was only there up to the release of Lion (the first Craig release), so I don't know how updates and patches worked from then on. Maybe they're worse now. But I've been using OS X all this time, and honestly I don't think it's any worse than before. What has changed is that releases and features happen more often. Tiger and Leopard had a good 2 years to mature and get patches while their delayed successors missed target dates. In the meantime they stagnated with ancient unix tools, safari build, QuickTime frameworks, graphics drivers etc. They felt stable because they were just old, sort of like Debian stable. Meanwhile, the development versions of Leopard and Snow Leopard (the two I spent most of my career at Apple developing) were downright horrible and unreleasable. Each of those releases went gold and had an almost immediate .1 release to fix glaring issues. It's just that you remember them better because they had a longer history as a stable legacy OS than the modern versions.

toyg 12y ago
you remember them better because they had a longer history as a stable legacy OS than the modern versions. Does this tell us anything about the state of modern commercial OS development? Users like stability and predictability in their software. Sure, they like new whizz-bang as well, but that excitement lasts for a week after you buy a new machine, and then they want stability every day for years. That conflicts with commercial pressures to sell Shiny New Things every few months. Flat icons? Let's make them skeuomorphic. Skeuomorphic icons? Let's make them flat! Tabs are stupid, let's make them all windows. Windows are stupid, let's make them all tabs! Multitasking is great, let's have more dashboards. Single-tasking is great, let's have more fullscreen! And so on and so forth, rinse and repeat. I think hardware waves used to hide most of this game, since every year new hardware was so much more powerful than before, you could justify this constant churning as "adding more stuff". This has not really been true on the desktop for almost a decade now, so the long-term patterns emerge and they are quite ugly to see.
- rdtsc 12y ago
  > Flat icons? Let's make them skeuomorphic. Skeuomorphic icons? Let's make them flat! Tabs are stupid, let's make them all windows. ... My guess the other problem is simply this -- they hired so many developers, designers, managers. They produced a great stable OS, worked hard, then it was making billions of dollars. Ok, what next? Institute 4 day weeks? (maybe not a bad idea, but I am being hyperbolic here a bit). So there is perhaps some pressure from the inside to develop new features. Designers say "flat is best now" so they get to work and feel busy. Developers want to build the next internal restructuring. Managers probably want to be responsible for some new awesome feature and seal their legacy.
  - zrm 12y ago
    > They produced a great stable OS, worked hard, then it was making billions of dollars. Ok, what next? Institute 4 day weeks? (maybe not a bad idea, but I am being hyperbolic here a bit). It's really a business problem. Once you have a successful high margin product, investors expect you to continue making those numbers. If you sit on your duff then competitors will catch up and you find yourself in a low margin commodity market. But is it isn't always obvious what to develop, especially once the low hanging fruit is gone. Sometimes there isn't a right answer and the market is simply mature enough that it's destined to become a commodity market. And if you can't find something good to do, people have an aversion to doing nothing. So instead they do something bad.
  - gurkendoktor 12y ago
    I agree, and that was the feeling I got while watching the 2014 WWDC. Heaps and heaps of unrelated announcements (and the feeling that nothing of exceptional quality would come out of it). I wish Apple had instead focused, for the whole year, on getting iCloud Photos 100% right. Unlike "flat" redesigns, this could even have turned into passive income for Apple. It hasn't even shipped on OS X and I've already given up on it. If they'd just frozen OS X at 10.9, everyone could still work on cloud functionality for years before they'd be "finished".
  - toyg 12y ago
    > So there is perhaps some pressure from the inside to develop new features. Oh yeah, these pressures are both internal and external. There is a reason concepts like NIH, bike-shedding and "reinventing the wheel" are endemic problems in the software world. I guess we're just seeing that Apple is not immune from them, after all.
throwawayosx 12y ago
current apple engineer... the sprint (milestone) development system is still in place... it's not the problem though, it's the problem is the focus on new useless [imo] features at the expense of core functionality and quality hope marco, geoff and others keep writing these articles so that eventually tim or someone sees one and shakes things up. pressure from the bottom has not worked so far
- 23david 12y ago
  would be interesting to hear what the distinction is between useless and core features. Maybe I'm just not hitting core features with OSX 10.10, but the features I'm using seem fine. And not seeing stability issues with third-party software.
  - SkyMarshal 12y ago
    Fwiw I can crash the entire OS by abusing tabs in Chrome. Too many tabs open => System freeze, hard reboot required. OS X Mavericks on MBPr 15". That shouldn't be possible. Crash the browser yes, not the whole OS.
    - lobster_johnson 12y ago
      Sounds like you have a hardware problem, actually. Faulty GPU or RAM, quite likely.
      - aragot 12y ago
        No, you can't single him out like this: Since Mavericks, OS crashes are very often reported, and Chrome tabs seem to be often reported as a cause. I myself have never had crashes with Snow Leopard and I see them often in Yosemite. That's exactly why we see articles reporting the Apple has lost the Quality ground.
      - rdl 12y ago
        I have exactly the same problem on multiple machines. Plus, the wifi fiasco, over which more than one person should have been fired. I am barely sticking with Apple for now, mostly because in startups it is the default, but I don't plan to replace my apple equipment with more apple products.
visarga 12y ago
Yosemite has regressed, in my opinion. All Screen Sharing sessions cause my MacBook Pro to freeze for 5 minutes after they are closed. I had to reboot the laptop many times.
- DigitalJack 12y ago
  You should consider showing this behavior at the genius bar if that is feasible for you. It's not normal for Yosemite.
- esaruoho 12y ago
  Have you reported this to http://bugreport.apple.com/ http://bugreport.apple.com/ ? I know most of my Yosemite / Mavericks bugs were tagged as duplicates, but there might be some sort of a "wow, so many duplicates, let's fix this" type of ranking re: bugs reported to the Radar. I, for one, would wish Mr. Federighi would push out a Snow Yosemite. No new features, simply bugfixing. I have taken to simply posting screenshots on my blog about various AppleQA issues. I wish there was a way to really make a difference, god know I'm fanatical enough about wanting Apple stuff to just work.
pi-err 12y ago
> But I've been using OS X all this time, and honestly I don't think it's any worse than before. Agreed. Yosemite has been flat out the best OS X version I used. Zero bug except a few Airplay issues. I'm even using beta builds for production, this thing is that stable. I'm suspecting that Marco's point shows how a maturing part of the developer/consumer now takes a larger mindshare. Not a bad thing, it's just that it doesn't tell much on the product. edit: Marco says this gets him 7.8m/s traffic, so there has to be something for _many_ here.
bane 12y ago
> Craig moved the organization onto a sprint system, where we would develop new features for 2 weeks and then spend a week fixing bugs. After 10 or 12 or 16 of these cycles, we would deem it ready and ship it out. I felt this produced more stable but more conservative software. It seemed like giant rewrites and massive features would be very difficult to introduce and if they did get done, wouldn't happen until two thirds or so into the release cycle. I've noticed this problem too in Agile shops. Small iterations happen fast on well scheduled. But big new efforts tend to be like swimming up-river. Whatever comes after Agile is probably going to formalize a long-track parallel cadence for a dedicated team to grind away on. Then once that's released it goes into the normal sprint cadence. It seems to usually be "solved" right now by lots of weeping, halting production and blowing up of anybody else's schedule.
- dragonwriter 12y ago
  > I've noticed this problem too in Agile shops. Small iterations happen fast on well scheduled. But big new efforts tend to be like swimming up-river. Whatever comes after Agile is probably going to formalize a long-track parallel cadence for a dedicated team to grind away on. That seems to be a not-that-uncommon practice in Agile/Lean environments with the resources to implement it already (you don't see a lot of coverage of it in books about "Agile" software methods specifically, though I've seen considerable coverage of it in "Lean" software development books -- which tend to address the kind of pragmatic metamethodology the Agile manifesto calls for, while "Agile" books, ironically, tend to focus on more on narrow prescriptive methodologies, particularly Scrum and close variants.) The immediate and longer-term (or multiple potential solutions with different risk profiles without a strict "this one is for now, this one is the long term objective solution" division) in parallel thing that Google has done in lots of areas for a long time (and people outside often question with "why is Google doing both X and Y when they have overlapping use cases".)
exAppleToo 12y ago
The good thing about the nightly builds was you didn't have to use them, and people could respond quickly to showstoppers rather than wait for a sprint. There was a quicklook team to catch bugs which stopped nightly builds from release to general dev, and unless you really needed that build ( to test a new API) you didn't install until quality is restored. With fortnightly builds, if that is now what is happening, there will be massive instability every 2 weeks, until the final round of bug fixing cycles. Which can't finish because the OS has to be release at an event.
exAppleToo 12y ago
The good thing about the nightly builds was you didn't have to use them, and people could respond quickly to showstoppers rather than wait for a sprint. There was a quicklook team to catch bugs which stopped nightly builds from release to general dev, and unless you really needed that build ( to test a new API) you didn't install until quality was restored. The xCode bug was therefore unusual, as they could have fixed it the next day ( and with enough heat they would have). With fortnightly builds which are only then released to Engineering, if that is now what is happening, there will be massive instability every 2 weeks, until the final round of bug fixing cycle stops all features being added. Which can't finish in time because the OS has to be released at an Apple event. Except for 10.0 when the date was known months in advance the old OS cycles were released when ready ( when show stoppers were 0). Of course that did lead to some wrangling about what a show stopper was, but consider if iOS 8 was not released with the HealthKit bugs, and other major issues, and instead we got what was in iOs 8.1, or 10.1 instead of 10.0. Still some issues but mostly robust. Apple needs to decouple software from hardware releases.

toyg 12y ago

you remember them better because they had a longer history as a stable legacy OS than the modern versions. Does this tell us anything about the state of modern commercial OS development? Users like stability and predictability in their software. Sure, they like new whizz-bang as well, but that excitement lasts for a week after you buy a new machine, and then they want stability every day for years. That conflicts with commercial pressures to sell Shiny New Things every few months. Flat icons? Let's make them skeuomorphic. Skeuomorphic icons? Let's make them flat! Tabs are stupid, let's make them all windows. Windows are stupid, let's make them all tabs! Multitasking is great, let's have more dashboards. Single-tasking is great, let's have more fullscreen! And so on and so forth, rinse and repeat. I think hardware waves used to hide most of this game, since every year new hardware was so much more powerful than before, you could justify this constant churning as "adding more stuff". This has not really been true on the desktop for almost a decade now, so the long-term patterns emerge and they are quite ugly to see.

rdtsc 12y ago
> Flat icons? Let's make them skeuomorphic. Skeuomorphic icons? Let's make them flat! Tabs are stupid, let's make them all windows. ... My guess the other problem is simply this -- they hired so many developers, designers, managers. They produced a great stable OS, worked hard, then it was making billions of dollars. Ok, what next? Institute 4 day weeks? (maybe not a bad idea, but I am being hyperbolic here a bit). So there is perhaps some pressure from the inside to develop new features. Designers say "flat is best now" so they get to work and feel busy. Developers want to build the next internal restructuring. Managers probably want to be responsible for some new awesome feature and seal their legacy.
- zrm 12y ago
  > They produced a great stable OS, worked hard, then it was making billions of dollars. Ok, what next? Institute 4 day weeks? (maybe not a bad idea, but I am being hyperbolic here a bit). It's really a business problem. Once you have a successful high margin product, investors expect you to continue making those numbers. If you sit on your duff then competitors will catch up and you find yourself in a low margin commodity market. But is it isn't always obvious what to develop, especially once the low hanging fruit is gone. Sometimes there isn't a right answer and the market is simply mature enough that it's destined to become a commodity market. And if you can't find something good to do, people have an aversion to doing nothing. So instead they do something bad.
- gurkendoktor 12y ago
  I agree, and that was the feeling I got while watching the 2014 WWDC. Heaps and heaps of unrelated announcements (and the feeling that nothing of exceptional quality would come out of it). I wish Apple had instead focused, for the whole year, on getting iCloud Photos 100% right. Unlike "flat" redesigns, this could even have turned into passive income for Apple. It hasn't even shipped on OS X and I've already given up on it. If they'd just frozen OS X at 10.9, everyone could still work on cloud functionality for years before they'd be "finished".
- toyg 12y ago
  > So there is perhaps some pressure from the inside to develop new features. Oh yeah, these pressures are both internal and external. There is a reason concepts like NIH, bike-shedding and "reinventing the wheel" are endemic problems in the software world. I guess we're just seeing that Apple is not immune from them, after all.

throwawayosx 12y ago

current apple engineer... the sprint (milestone) development system is still in place... it's not the problem though, it's the problem is the focus on new useless [imo] features at the expense of core functionality and quality hope marco, geoff and others keep writing these articles so that eventually tim or someone sees one and shakes things up. pressure from the bottom has not worked so far

23david 12y ago
would be interesting to hear what the distinction is between useless and core features. Maybe I'm just not hitting core features with OSX 10.10, but the features I'm using seem fine. And not seeing stability issues with third-party software.
- SkyMarshal 12y ago
  Fwiw I can crash the entire OS by abusing tabs in Chrome. Too many tabs open => System freeze, hard reboot required. OS X Mavericks on MBPr 15". That shouldn't be possible. Crash the browser yes, not the whole OS.
  - lobster_johnson 12y ago
    Sounds like you have a hardware problem, actually. Faulty GPU or RAM, quite likely.
    - aragot 12y ago
      No, you can't single him out like this: Since Mavericks, OS crashes are very often reported, and Chrome tabs seem to be often reported as a cause. I myself have never had crashes with Snow Leopard and I see them often in Yosemite. That's exactly why we see articles reporting the Apple has lost the Quality ground.
    - rdl 12y ago
      I have exactly the same problem on multiple machines. Plus, the wifi fiasco, over which more than one person should have been fired. I am barely sticking with Apple for now, mostly because in startups it is the default, but I don't plan to replace my apple equipment with more apple products.

visarga 12y ago

Yosemite has regressed, in my opinion. All Screen Sharing sessions cause my MacBook Pro to freeze for 5 minutes after they are closed. I had to reboot the laptop many times.

DigitalJack 12y ago
You should consider showing this behavior at the genius bar if that is feasible for you. It's not normal for Yosemite.
esaruoho 12y ago
Have you reported this to http://bugreport.apple.com/ http://bugreport.apple.com/ ? I know most of my Yosemite / Mavericks bugs were tagged as duplicates, but there might be some sort of a "wow, so many duplicates, let's fix this" type of ranking re: bugs reported to the Radar. I, for one, would wish Mr. Federighi would push out a Snow Yosemite. No new features, simply bugfixing. I have taken to simply posting screenshots on my blog about various AppleQA issues. I wish there was a way to really make a difference, god know I'm fanatical enough about wanting Apple stuff to just work.

pi-err 12y ago

> But I've been using OS X all this time, and honestly I don't think it's any worse than before. Agreed. Yosemite has been flat out the best OS X version I used. Zero bug except a few Airplay issues. I'm even using beta builds for production, this thing is that stable. I'm suspecting that Marco's point shows how a maturing part of the developer/consumer now takes a larger mindshare. Not a bad thing, it's just that it doesn't tell much on the product. edit: Marco says this gets him 7.8m/s traffic, so there has to be something for _many_ here.

bane 12y ago

> Craig moved the organization onto a sprint system, where we would develop new features for 2 weeks and then spend a week fixing bugs. After 10 or 12 or 16 of these cycles, we would deem it ready and ship it out. I felt this produced more stable but more conservative software. It seemed like giant rewrites and massive features would be very difficult to introduce and if they did get done, wouldn't happen until two thirds or so into the release cycle. I've noticed this problem too in Agile shops. Small iterations happen fast on well scheduled. But big new efforts tend to be like swimming up-river. Whatever comes after Agile is probably going to formalize a long-track parallel cadence for a dedicated team to grind away on. Then once that's released it goes into the normal sprint cadence. It seems to usually be "solved" right now by lots of weeping, halting production and blowing up of anybody else's schedule.

dragonwriter 12y ago
> I've noticed this problem too in Agile shops. Small iterations happen fast on well scheduled. But big new efforts tend to be like swimming up-river. Whatever comes after Agile is probably going to formalize a long-track parallel cadence for a dedicated team to grind away on. That seems to be a not-that-uncommon practice in Agile/Lean environments with the resources to implement it already (you don't see a lot of coverage of it in books about "Agile" software methods specifically, though I've seen considerable coverage of it in "Lean" software development books -- which tend to address the kind of pragmatic metamethodology the Agile manifesto calls for, while "Agile" books, ironically, tend to focus on more on narrow prescriptive methodologies, particularly Scrum and close variants.) The immediate and longer-term (or multiple potential solutions with different risk profiles without a strict "this one is for now, this one is the long term objective solution" division) in parallel thing that Google has done in lots of areas for a long time (and people outside often question with "why is Google doing both X and Y when they have overlapping use cases".)

exAppleToo 12y ago

The good thing about the nightly builds was you didn't have to use them, and people could respond quickly to showstoppers rather than wait for a sprint. There was a quicklook team to catch bugs which stopped nightly builds from release to general dev, and unless you really needed that build ( to test a new API) you didn't install until quality was restored. The xCode bug was therefore unusual, as they could have fixed it the next day ( and with enough heat they would have). With fortnightly builds which are only then released to Engineering, if that is now what is happening, there will be massive instability every 2 weeks, until the final round of bug fixing cycle stops all features being added. Which can't finish in time because the OS has to be released at an Apple event. Except for 10.0 when the date was known months in advance the old OS cycles were released when ready ( when show stoppers were 0). Of course that did lead to some wrangling about what a show stopper was, but consider if iOS 8 was not released with the HealthKit bugs, and other major issues, and instead we got what was in iOs 8.1, or 10.1 instead of 10.0. Still some issues but mostly robust. Apple needs to decouple software from hardware releases.