6 ms·

Apple Developer Center is Fully Restored

46 points by nimeshneema 13y ago

k-mcgrady 13y ago
It took over 3 weeks to restore everything. I hope Apple does some sort of post-mortem (at least something basic). When a security breach takes you down for this long it was either a very serious breach or your security and infrastructure was shit and your customers deserve to know how you've remedied that.
X-Cubed 13y ago
I was wondering why I was unable to access any of the services that were supposedly back online. Now that the Member Center is back, it turns out there were new Terms & Conditions I needed to accept first... argh.
seivan 13y ago
My account is still subject to expiring even though I did a renewal before things went down - should have eleven months at least. Anyone else got this issue?
fredley 13y ago
So the whole place went down a few days before we were ready to launch. We need to upload an iPhone app to iTunes Connect. We have been prevented from doing this because we needed to accept some new T&Cs on the member centre. Despite iTunes Connect being up, we've been stuck. Apple support were unhelpful - we've just had to wait it out. Now finally I've been able to get on the member centre and accept the T&Cs, but there's clearly some delay in iTunes Connect realising this. I still can't upload my app. Many more profanities are being hurled at Apple this morning.
- harrytuttle 13y ago
  Sorry but when you base your businesses on walled gardens you are forever at the mercy of others. There is nothing to complain about here; you've just got to suck it up.
  - fredley 13y ago
    The business is not based around mobile, but if you're going to have any kind of mobile apps - however incidental to your core product - you have to put up with it.
AlexanderDhoore 13y ago
I wonder what Apple's server infrastructure looks like? They aren't programming it all in WebObjects [1] are they? Do they use linux and/or freeBSD? Of all the things Apple does, their "cloud" seems the weakest product. Apple is the only big software company that doesn't develop server-side technologies. If I was the boss of Apple I'd standardize on FreeBSD and Golang today! But, I'm not... obviously :) [1] http://en.wikipedia.org/wiki/WebObjects http://en.wikipedia.org/wiki/WebObjects
- nwh 13y ago
  I've brought this up before, but it's a mix of everything, and all pretty bad. > If you've ever poked around with the way that Apple's website works, you can see that the entire place is a huge mess. There's old servers running ancient (pre-2004) perl scripts alongside the brand new iCloud gear. I can't imagine how the authentication for AppleID is working as login details still work on the ancient pages (think pinstripes and glassy buttons). Depending what URL you hit, the webserver is using php3, php4, perl, python or maybe WebObjects (java). > At one point I wrote a scraper that was targeting one of their product pages, and kept getting random, unexplainable results. It turned out that one of their product areas was behind a round-robin load balancer, with three completely different apache versions on each server. The page was dying on one but not the other two. In the end I just had to repetitively scrape until I hit a good response. https://news.ycombinator.com/item?id=6078854 https://news.ycombinator.com/item?id=6078854
  - AlexanderDhoore 13y ago
    Wow, that's exactly what I was afraid of... You can tell it's absolutely not their priority. It's more of an afterthought. "Oh, yeah, btw, we need some servers as well, but first do all the other stuff."
- Terretta 13y ago
  > Apple is the only big software company that doesn't develop server-side technologies. iCloud storage, iTunes Match, iMessage, to name just a few. If you stop to think how these all just work across devices and desktops, and at what volumes, you'll realize they've got some server chops.
dave1010uk 13y ago
Not not often one to praise Apple, but I'm really impressed that they spent the time they did to fix this. Apple are known for their strong brand and taking down the dev centre for so long must have been an awkward decision between the marketing side and the technical one. I don't know the extent of the problem but taking down an online service for weeks is very uncommon. I guess (speculating a bit) that Apple could have applied a quick fix to the problem within a day to save their (short term, at least) brand appearance. I think "just getting it secure enough" is the most many people would do if an important service was down. It appears that Apple took the time required to deploy a proper fix, prioritising security over shot term wins. It would be interesting to see what someone like Microsoft or Amazon would have done in this situation, or what Apple would have done if it was all of iTunes instead of the dev centre.
- dave1010uk 13y ago
  Does anyone know of a centralised database of security breaches? It would be great if there were a standard (like CVEs), which included how well the breach was dealt with (eg informing public, mitigation, prevention, changes made). If there isn't one and there's interest here then maybe I'll start one.
- thezilch 13y ago
  You're impressed they spent time keeping their cash cow alive? No offense to the developers, but there has been very little to no communication. I'm magically waiting for the guy with the best F5 time on the dashboard to get my news? Again, no offense to the developers, but there's no indication that this was not a "quick fix" by industry standards (of security) and not just "secure enough." As usual, a dashboard of green lights does not identify uptime, as can be established by the issues people are still having with the platform.
  - deminature 13y ago
    >No offense to the developers, but there has been very little to no communication. There's been weekly or slightly shorter than weekly updates and a 24-hour status page. Also, most key portal functionality was available not that long after the breach. It's a little unfair to say "very little to no communication".
pknerd 13y ago
It is not for me. When I try to Join SAfari developer Program it is redirected to this URL: https://developer.apple.com/iphone/urlRedirect.action?mode=enroll&successURL=https://developer.apple.com/ios/enroll/individualEnrollmentIndex.action&cancelURL=https://developer.apple.com/programs https://developer.apple.com/iphone/urlRedirect.action?mode=e...
- nwh 13y ago
  From the action URL, this seems to be the page you want — https://developer.apple.com/ios/enroll/individualEnrollmentIndex.action https://developer.apple.com/ios/enroll/individualEnrollmentI...
mickeyckm 13y ago
Hmmmm, after I accepted the new terms & conditions, the iTunesconnect still ask me to accept it in Member Center. I didn't expect to have some sort of delay between the changes in Member Center and iTunesconnect. Hope it takes effect soon. Sigh.
msie 13y ago
Argh, it's silly that it takes some time for a membership renewal to take effect.
- msie 13y ago
  Why isn't this valid criticism? Buy a digital membership anywhere else and the effect is immediate. Buy something on iTunes and you get the product immediately.
jmomo 13y ago
If Apple had a security breach, and it seems like a massive one given the circumstances, are they not required by California law to report it to their customers?
- pilif 13y ago
  They did report that there was a breach, that some personal data about developers (but no credit card info or passwords) was stolen and that they are restoring the developer center. That's more than enough reporting. If you want a full post-mortem, that's likely not something you'd get from a public company as big as Apple
  - robryan 13y ago
    AWS always gives full and detailed postmortems. I wouldn't say it is a trait of big companies specially not to disclose to customers, but it is something I would expect Apple to probably do.
  - tazjin 13y ago
    You wouldn't get a full post-mortem from a company like Apple, because they are wilfully intransparent. My employer (Spotify) for example posts somewhat detailed post-mortems[1] after big outages - doing this always comes with a spike in job applications so it's positive in several ways. Another comment mentioned that AWS post-mortems are also detailed and public, they don't really have another choice because their customers have their infrastructure running on AWS - so they want to know what happened and not be left in the dark. [1] http://labs.spotify.com/2013/06/04/incident-management-at-spotify/ http://labs.spotify.com/2013/06/04/incident-management-at-sp...
marshray 13y ago
Let's pour a cider on the ground for all the fallen evenings and weekends that were surely sacrificed by our industry comrades to accomplish this restoration.
- harrytuttle 13y ago
  For fucks sake. If they'd done a half decent job to start with, they wouldn't be there. They fucked up, now they're paying for it.
  - marshray 13y ago
    Surely you don't believe that everyone who busted their ass to fix the problem was also responsible for causing it?
jng 13y ago
Great Saturday morning news, I've finally able to run my new iOS app on my iPhone! I had to accept new T&C before getting certificates/provisioning-profiles, and this has been impossible for weeks.
- marknutter 13y ago
  The profiles center has been a online for quite a while now.
  - myko 13y ago
    > I had to accept new T&C before getting certificates/provisioning-profiles, and this has been impossible for weeks. If you can't accept the new T&C you can't get into the profile center.
  - eddieroger 13y ago
    But the agreements portion wasn't. My company's enterprise license was supposed to run out a week ago. Luckily they extended it, but the hub of everything - the agreements server - was down.
    - marknutter 13y ago
      ugh, what a mess.
kyriakos 13y ago
..if microsoft's or google's services were down for this long hell would freeze... just saying that apple seems to be getting treated better for no reason
- marknutter 13y ago
  You mean, if Microsoft's or Google's developer services were down for this long. I honestly don't think people would raise any more stink than they did with Apple.
- corresation 13y ago
  On the flip side, Apple took a security issue very, very seriously, and for that they should be commended. I mean, obviously better if it were found beforehand, but clearly someone outside of the people in charge of these web properties had a WTF moment and life became unpleasant for some people for a while. Most firms would have duct-taped the specific section exploited.
  - myko 13y ago
    > On the flip side, Apple took a security issue very, very seriously, and for that they should be commended. Though they were warned before the breach occurred, right? They didn't take it seriously until well after they knew they needed to fix it.

46 points by nimeshneema 13y ago

It took over 3 weeks to restore everything. I hope Apple does some sort of post-mortem (at least something basic). When a security breach takes you down for this long it was either a very serious breach or your security and infrastructure was shit and your customers deserve to know how you've remedied that.

X-Cubed 13y ago

I was wondering why I was unable to access any of the services that were supposedly back online. Now that the Member Center is back, it turns out there were new Terms & Conditions I needed to accept first... argh.

seivan 13y ago

My account is still subject to expiring even though I did a renewal before things went down - should have eleven months at least. Anyone else got this issue?

fredley 13y ago

So the whole place went down a few days before we were ready to launch. We need to upload an iPhone app to iTunes Connect. We have been prevented from doing this because we needed to accept some new T&Cs on the member centre. Despite iTunes Connect being up, we've been stuck. Apple support were unhelpful - we've just had to wait it out. Now finally I've been able to get on the member centre and accept the T&Cs, but there's clearly some delay in iTunes Connect realising this. I still can't upload my app. Many more profanities are being hurled at Apple this morning.

harrytuttle 13y ago
Sorry but when you base your businesses on walled gardens you are forever at the mercy of others. There is nothing to complain about here; you've just got to suck it up.
- fredley 13y ago
  The business is not based around mobile, but if you're going to have any kind of mobile apps - however incidental to your core product - you have to put up with it.

AlexanderDhoore 13y ago

I wonder what Apple's server infrastructure looks like? They aren't programming it all in WebObjects [1] are they? Do they use linux and/or freeBSD? Of all the things Apple does, their "cloud" seems the weakest product. Apple is the only big software company that doesn't develop server-side technologies. If I was the boss of Apple I'd standardize on FreeBSD and Golang today! But, I'm not... obviously :) [1] http://en.wikipedia.org/wiki/WebObjects http://en.wikipedia.org/wiki/WebObjects

nwh 13y ago
I've brought this up before, but it's a mix of everything, and all pretty bad. > If you've ever poked around with the way that Apple's website works, you can see that the entire place is a huge mess. There's old servers running ancient (pre-2004) perl scripts alongside the brand new iCloud gear. I can't imagine how the authentication for AppleID is working as login details still work on the ancient pages (think pinstripes and glassy buttons). Depending what URL you hit, the webserver is using php3, php4, perl, python or maybe WebObjects (java). > At one point I wrote a scraper that was targeting one of their product pages, and kept getting random, unexplainable results. It turned out that one of their product areas was behind a round-robin load balancer, with three completely different apache versions on each server. The page was dying on one but not the other two. In the end I just had to repetitively scrape until I hit a good response. https://news.ycombinator.com/item?id=6078854 https://news.ycombinator.com/item?id=6078854
- AlexanderDhoore 13y ago
  Wow, that's exactly what I was afraid of... You can tell it's absolutely not their priority. It's more of an afterthought. "Oh, yeah, btw, we need some servers as well, but first do all the other stuff."
Terretta 13y ago
> Apple is the only big software company that doesn't develop server-side technologies. iCloud storage, iTunes Match, iMessage, to name just a few. If you stop to think how these all just work across devices and desktops, and at what volumes, you'll realize they've got some server chops.

dave1010uk 13y ago

Not not often one to praise Apple, but I'm really impressed that they spent the time they did to fix this. Apple are known for their strong brand and taking down the dev centre for so long must have been an awkward decision between the marketing side and the technical one. I don't know the extent of the problem but taking down an online service for weeks is very uncommon. I guess (speculating a bit) that Apple could have applied a quick fix to the problem within a day to save their (short term, at least) brand appearance. I think "just getting it secure enough" is the most many people would do if an important service was down. It appears that Apple took the time required to deploy a proper fix, prioritising security over shot term wins. It would be interesting to see what someone like Microsoft or Amazon would have done in this situation, or what Apple would have done if it was all of iTunes instead of the dev centre.

dave1010uk 13y ago
Does anyone know of a centralised database of security breaches? It would be great if there were a standard (like CVEs), which included how well the breach was dealt with (eg informing public, mitigation, prevention, changes made). If there isn't one and there's interest here then maybe I'll start one.
thezilch 13y ago
You're impressed they spent time keeping their cash cow alive? No offense to the developers, but there has been very little to no communication. I'm magically waiting for the guy with the best F5 time on the dashboard to get my news? Again, no offense to the developers, but there's no indication that this was not a "quick fix" by industry standards (of security) and not just "secure enough." As usual, a dashboard of green lights does not identify uptime, as can be established by the issues people are still having with the platform.
- deminature 13y ago
  >No offense to the developers, but there has been very little to no communication. There's been weekly or slightly shorter than weekly updates and a 24-hour status page. Also, most key portal functionality was available not that long after the breach. It's a little unfair to say "very little to no communication".

pknerd 13y ago

It is not for me. When I try to Join SAfari developer Program it is redirected to this URL: https://developer.apple.com/iphone/urlRedirect.action?mode=enroll&successURL=https://developer.apple.com/ios/enroll/individualEnrollmentIndex.action&cancelURL=https://developer.apple.com/programs https://developer.apple.com/iphone/urlRedirect.action?mode=e...

nwh 13y ago
From the action URL, this seems to be the page you want — https://developer.apple.com/ios/enroll/individualEnrollmentIndex.action https://developer.apple.com/ios/enroll/individualEnrollmentI...

mickeyckm 13y ago

Hmmmm, after I accepted the new terms & conditions, the iTunesconnect still ask me to accept it in Member Center. I didn't expect to have some sort of delay between the changes in Member Center and iTunesconnect. Hope it takes effect soon. Sigh.

msie 13y ago

Argh, it's silly that it takes some time for a membership renewal to take effect.

msie 13y ago
Why isn't this valid criticism? Buy a digital membership anywhere else and the effect is immediate. Buy something on iTunes and you get the product immediately.

jmomo 13y ago

If Apple had a security breach, and it seems like a massive one given the circumstances, are they not required by California law to report it to their customers?

pilif 13y ago
They did report that there was a breach, that some personal data about developers (but no credit card info or passwords) was stolen and that they are restoring the developer center. That's more than enough reporting. If you want a full post-mortem, that's likely not something you'd get from a public company as big as Apple
- robryan 13y ago
  AWS always gives full and detailed postmortems. I wouldn't say it is a trait of big companies specially not to disclose to customers, but it is something I would expect Apple to probably do.
- tazjin 13y ago
  You wouldn't get a full post-mortem from a company like Apple, because they are wilfully intransparent. My employer (Spotify) for example posts somewhat detailed post-mortems[1] after big outages - doing this always comes with a spike in job applications so it's positive in several ways. Another comment mentioned that AWS post-mortems are also detailed and public, they don't really have another choice because their customers have their infrastructure running on AWS - so they want to know what happened and not be left in the dark. [1] http://labs.spotify.com/2013/06/04/incident-management-at-spotify/ http://labs.spotify.com/2013/06/04/incident-management-at-sp...

marshray 13y ago

Let's pour a cider on the ground for all the fallen evenings and weekends that were surely sacrificed by our industry comrades to accomplish this restoration.

harrytuttle 13y ago
For fucks sake. If they'd done a half decent job to start with, they wouldn't be there. They fucked up, now they're paying for it.
- marshray 13y ago
  Surely you don't believe that everyone who busted their ass to fix the problem was also responsible for causing it?

jng 13y ago

Great Saturday morning news, I've finally able to run my new iOS app on my iPhone! I had to accept new T&C before getting certificates/provisioning-profiles, and this has been impossible for weeks.

marknutter 13y ago
The profiles center has been a online for quite a while now.
- myko 13y ago
  > I had to accept new T&C before getting certificates/provisioning-profiles, and this has been impossible for weeks. If you can't accept the new T&C you can't get into the profile center.
- eddieroger 13y ago
  But the agreements portion wasn't. My company's enterprise license was supposed to run out a week ago. Luckily they extended it, but the hub of everything - the agreements server - was down.
  - marknutter 13y ago
    ugh, what a mess.

kyriakos 13y ago

..if microsoft's or google's services were down for this long hell would freeze... just saying that apple seems to be getting treated better for no reason

marknutter 13y ago
You mean, if Microsoft's or Google's developer services were down for this long. I honestly don't think people would raise any more stink than they did with Apple.
corresation 13y ago
On the flip side, Apple took a security issue very, very seriously, and for that they should be commended. I mean, obviously better if it were found beforehand, but clearly someone outside of the people in charge of these web properties had a WTF moment and life became unpleasant for some people for a while. Most firms would have duct-taped the specific section exploited.
- myko 13y ago
  > On the flip side, Apple took a security issue very, very seriously, and for that they should be commended. Though they were warned before the breach occurred, right? They didn't take it seriously until well after they knew they needed to fix it.