11 ms·

Apple Developer Center still down

209 points by vinhnx 13y ago

coldtea 13y ago
Gee, thanks Marco, we would have never known ourselves...
- nicholassmith 13y ago
  That sounds like "I just read the title so I could post a snarky remark". He does put some suggestions forward of what he thinks could be the root cause.
  - coldtea 13y ago
    I read the whole article. The suggestions cover all the cases one would that thought himself. He is being Captain Obvious. Including the gem to "set some time aside" as developers, because we might have some work to do after the site comes back up. Everything in the comment is what men wiser than us called "idle speculation".
    - TwistedWeasel 13y ago
      If you don't like reading idle speculation then you should probably avoid 95% of tech blogs.
      - FrankBlack 13y ago
        I think that comment is Quickmeme-worthy. http://www.quickmeme.com/Captain-Hindsight/ http://www.quickmeme.com/Captain-Hindsight/
- saurik 13y ago
  Marco's audience includes a bunch of people who wouldn't have known this, or might have assumed it's back up: this draws attention back to it for people who like discussing this sort of thing (although, past that point, I largely agree with you, as I'm not certain what really there is to discuss).
nwh 13y ago
My money is personally on massive, unrecoverable data loss. If you've ever poked around with the way that Apple's website works, you can see that the entire place is a huge mess. There's old servers running ancient (pre-2004) perl scripts alongside the brand new iCloud gear. I can't imagine how the authentication for AppleID is working as login details still work on the ancient pages (think pinstripes and glassy buttons). Depending what URL you hit, the webserver is using php3, php4, perl, python or maybe WebObjects (java). At one point I wrote a scraper that was targeting one of their product pages, and kept getting random, unexplainable results. It turned out that one of their product areas was behind a round-robin load balancer, with three completely different apache versions on each server. The page was dying on one but not the other two. In the end I just had to repetitively scrape until I hit a good response. Even the domain for the "maintenance" page for the developer section is telling. It's just a broken template system regurgitating a bit of the homepage. http://devimages.apple.com/maintenance/ http://devimages.apple.com/maintenance/ http://devimages.apple.com/ http://devimages.apple.com/ Truly a hacked together system. Some engineers at Apple must be having a truly awful weekend, no matter the cause and solution.
- nicholassmith 13y ago
  I've spent more time using the developer resources recently, and I completely agree. Clicking one link will take you an entirely different feeling architecture, like parts have been built on top without thinking about what the impacts could be. It's probably been a house of cards for a long and it's now fallen down.
- saurik 13y ago
  Interesting; the http://devimages.apple.com/ http://devimages.apple.com/ URL seems to return the underlying contents of the file, bypassing the SSI (I'm so happy they use SSI... seriously: I love SSI ;P). You can then see the raw  directives, and pull the individual parts. (It isn't quite then fair to say that it is a "broken bit of their template system"; it is more that it is a poor way to setup a static large-file caching endpoint, and may itself lead to a security vulnerability. To be clear: that's probably worse ;P.)
NelsonMinar 13y ago
Could you imagine if the Linux Developer Center went down? It'd be nearly impossible to write Linux software, or download Linux development tools, or generate a Linux binary that a device was allowed to run. It'd be awful.
- coldtea 13y ago
  Nice attempt at snark, but something being fault-proof in this regard doesn't magically make it worthy in others. Like desktop use, for one. People will not adopt Linux for their desktop/laptop even when offered for free. Where by people I mean more than 1% of them.
  - ekianjo 13y ago
    Yeah well its not like computers ship without OS so your point does not really make sense. And its not like Linux has marketing unlike all other OSes out there. When you want to compare stuff you got to have similar factors in place otherwise it does not make sense.
  - pavanky 13y ago
    That is not totally true. The reason most people stay with Windows is because the hardware ships with it. In my experience Linux usage has about 50% retention rate among those I installed the OS for. If you'd install OSX or Windows on a non techie who grew up using just Linux, I'd think the retention rates would be much lower.
    - briandear 13y ago
      a 50% retention rate? That's not very good. That means that after you've installed Linux, half of the people switch back.
      - jlgreco 13y ago
        No, that is actually stunningly good, considering the alleged state of the Linux desktop and considering the audience that the Linux desktop has traditionally been targeting. It is so good, that I suspect he has a strong selection bias (he is probably only installing Linux for users he thinks may be able to use it fulltime. This selection is probably unconscious.) (And because it always seems to be necessary to add this disclaimer: I've been using Linux fulltime for over a decade; I'm not a critic, just a realist.)
peterkelly 13y ago
I would be very interested to see what the implications would be if this had happened to iCloud. I'm sure Apple are as unhappy about whatever has happened as the rest of us (likely much more so), but I think at least some communication from them about it would be in order.
ttflee 13y ago
> or device provisioning and certificates (potentially very profitable) Well, an `App'(, the name of which I would not like to identify, ) which was installed using Safari exploits and whose intended use is to help users search and install apps free of charge from AppStore, is still up and running. I guess there exist various exploits up and down the App Store chains, till now.
trackztar 13y ago
If it weren't Sunday, I wouldn't speculate, but it is. My 2 cents: iOS7 updates galore. Just imagine everything that could be overhauled. Even the old gray textured background we see is not very much like iOS 7. iOS is in a big transition here. You can't even update apps at all unless you now include 'widescreen' support, for example. If you don't, iTunes reports an Invalid Binary (no default 586 image, etc). So I would guess a huge overhaul, and typical Apple, is taking care of that vs. arguing or commenting on theories.
- tylermac1 13y ago
  Wouldn't they offer a warning that the dev center would be down if it's just a design overhaul?
  - deleted 13y ago
    [deleted]
- grey-area 13y ago
  So I would guess a huge overhaul, and typical Apple, is taking care of that vs. arguing or commenting on theories. A design overhaul wouldn't require the dev centre to go down at all - they could just prepare all the assets etc on testing servers and switch them over when they are ready. Given the normal warning given on any maintenance, and the obvious negative consequences for Apple's business of any extended outage (extended in this case being over a day or so), this is unintended, and most likely caused by a security breach. I think we can safely rule out a design overhaul unless Apple are incredibly incompetent. NB itunesconnect is still up (the bit which deals with app upload, itunes store metadata etc), only the dev center - dealing with certs/device registration/distribution etc - is down.
  - dkuntz2 13y ago
    Perhaps they're also trying to pull together all the disparate things running on the dev servers? nwh's comment ( https://news.ycombinator.com/item?id=6078854 https://news.ycombinator.com/item?id=6078854 ) says they've got a lot of different systems running, maybe they're trying to unify them to be on one system, making upgrading/changing easier in the future?
Tloewald 13y ago
Amazon bas been down for as much as a half day in recent memory — and not dev stuff but their customer-facing money-making site, and mid-week too.
- mullingitover 13y ago
  Apple's customer-facing store was down last night for several hours.
  - Terretta 13y ago
    Doesn't matter. People aren't going to decide, gee, I actually wanted a Toshiba.
    - MAGZine 13y ago
      people also aren't going to decide 'gee, i'm going to go to x store and pay more for something' anymore then they'll go to BB to buy their apple product of choice.
- briandear 13y ago
  How many times has AWS gone down taking with it half the web? That to me is a bigger issue than not being able to download iOS beta 3 for a few days.
qnk 13y ago
What's even worse is that we, the developers, their customers, will never get to know what happened. Because that's the Apple way. I'd really love that Apple proves me wrong on this one and comes clean on the problem, the cause and prevention measures being put in place so this won't happen again, whatever it is.
- general_failure 13y ago
  Nope they won't say a thing and they will get away with it. Oh the things apple devs have to pout upon with. In another world HN would be up in arms about how to manage downtime, how this is u acceptable and how they are going to switch to somebody else immediately. But not for apple because they have no choice...
- k-mcgrady 13y ago
  They'll have to come clean on this. People pay for this service AND many businesses rely on it. If I'm correct enterprise businesses won't be able to add new devices to the system for example. Apple will need a good explanation for this and if it is a security problem they'll probably be issuing password reset requests and maybe invalidating certificates - you can't just do those things without an explanation.
kailuowang 13y ago
I have received 6 "How to reset your Apple ID password" Email from Apple during the last couple of days, none of which was triggered by me. Could this be related?
- swalsh 13y ago
  That's funny, a few days ago i received several too.
- sarreph 13y ago
  Definitely. http://www.zdnet.com/amid-extended-apple-developer-site-downtime-users-report-unauthorized-password-reset-emails-7000018333/ http://www.zdnet.com/amid-extended-apple-developer-site-down...
- FPSDavid 13y ago
  Same here.
- tsenkov 13y ago
  This makes our best guess to be "security breach".
  - rimantas 13y ago
    I receive similar emails from Gmail now and then. Does not me think that Gmail had been compromised.
    - MAGZine 13y ago
      right, but we don't have multiple people all confirming GMail password resets while GMail just so conveniently happens to be down.
- mrpizzadelivery 13y ago
  Was anyone else prompted to "update their expired password" when attempting to login to iTunesConnect? I did have the same password for about three years so it didn't seem too odd. Of course I double-checked the url and it looked good. Also I waited for a few days in the hopes that someone else would notice if this was a scam of some sort. Unfortunately after updating my password I forgot the new one. At least I think I forgot as my new password (as best I can remember it) doesn't work.
slowdown 13y ago
Why is a mere unfounded speculation not backed up by any facts written to garner pageviews featured on the frontpage?
- freehunter 13y ago
  Because there are no facts, Apple hasn't said a word. The alternative is that we ignore this. At least with this article (which no one has to read), there's a place for HN Apple devs to talk amongst each other.
  - slowdown 13y ago
    >a place for HN Apple devs to talk amongst each other So, you're telling me that I need to go read his article, earn him pageviews and ad money and come back here to discuss about the issue? I think a Ask HN post would have done the job, much better, no? Just saw your edit: >(which no one has to read) wut?
    - freehunter 13y ago
      Edit? No, I wrote that in one try. No one has to read the article, and it seems like a lot of people don't read the article before commenting. The comment box doesn't give you a quiz on the article before letting you comment.
  - hrktb 13y ago
    It goes very deep in meta territory, but hey, it's not as if there were any fact intersting to discuss. I remember a lot of discussions a year or two ago when marco was very vocal about the reasons he didn't have comments on his blog, and would prefer to have all of these on HN for e.g. slowdown's comment goes the other way round, on why HN should care about commenting on marco's blog. There is another thread [1] with 138 comments and more quality opinions and speculations than anyone should need, Marco's speculations are also represented in the top comments. Agree or not with he's stance, it's a valid point IMO. I think it's fun to see this kind of reaction on why some contents should be commented or not, and the motivations to push a site or another. [1] https://news.ycombinator.com/item?id=6071233 https://news.ycombinator.com/item?id=6071233
Jgrubb 13y ago
This reminds me, yesterday i got an obviously spoofed phishing email from "apple" telling me to reset my passwords and reenter my CC info. Anybody else get that? http://imgur.com/hyta4bC http://imgur.com/hyta4bC
- tazjin 13y ago
  Can you have a look at the mail headers and check if it was just a standard phishing mail?
  - Jgrubb 13y ago
    The "update now" link was to some completely other site, so without having checked out the headers yet I'd say it's a safe bet.
    - Achshar 13y ago
      maybe they were using some parked or addon domain? I know the chances are that it was not, but the entire thread talks about the poorly designed apple dev backend, which includes links like that.
- rossjudson 13y ago
  It never ceases to amaze me that such emails are carefully constructed in the graphical sense, but miserable failures in the grammatical.
  - ams6110 13y ago
    The British spelling of "apologise" is also a giveaway.
zackmorris 13y ago
Apple's websites are among the worst corporate sites I've ever used. I don't say that just to be critical or cynical (ahem), I'm merely pointing out that they have tremendous room for improvement. As opposed to say php.net or stackoverflow.com which still give me that fresh air "why can't all sites be like this?" feeling every time I visit them. Most of the time when I visit Apple's sites now, I just assume that what I need to find will either be buried in a convoluted maze, or simply won't exist, and I'll find myself on a "this has been deprecated" 404-style page which takes me someplace only loosely related to what I was looking for. So I end up back at google to try to find a copy of the information either cached somewhere or offsite. Simply being an Apple developer is a chore. Keeping up with yearly certificate expirations is taxing when you are contracting for several clients. And they never really worked out an easy way to allow several developers to share certs. I just assume now that the other developers will invalidate whatever shared cert I made. The situation is bad enough, and exacerbated by Apple stubbornly refusing to see the flaws, that I wish a startup would encapsulate the friction and just take care of all the minutia for me. I should never have to personally deal with provisioning. Anything short of a one click submission to iTunes Connect is reminiscent of all the TCP/IP details that we used to have to put in our modems in the dialup days, when all that should have been required was a phone number and passcode. I can't gently forgive them for it. So I think this downtime could be a wakeup call for them that the inefficiencies in their system are even costing them now.
- briandear 13y ago
  Have you spent any time on the Verizon or Sirius XM sites? How about the average multinational insurance company? "Among the worst corporate sites" is a huge leap, considering the profound amount of crap. Spending 5 minutes using the average electronic medical records (EMR) web application would be enough to make your hair fall out. Forget about interoperability of EMR formats. It's a mess. I'm not defending Apple at all, I've spent many nights throwing iPhones over the provisioning and certificate process, but to me, the outcome of surviving that process is that I'm making money from the apps, so it's not ideal, but at least my iOS users are actually paying me money, as opposed to Android users that, as a group tend to expect something for nothing. We ought to also look at the inefficiencies of the Android system while we're here. 35% of users are still on Gingerbread! You can't really expect to earn money as a developer if you ignore 35% of your target market, with iOS we have over 90% running iOS 6 and only about 6% running Android 4.2, which means that developers can't take advantage of a new Android feature without leaving behind the majority of their market, or creating multiple versions and then on top of that, having to test on a myriad of different hardware configurations. The fact also remains that Android users are generally cheap -- they don't like to pay for apps. So you have a highly fragmented OS environment, coupled with a user base that, in generally spends much less that iOS users and, on top of that, you have a royal mess in the copy-protection scheme used in Android -- which had to be disabled in Jelly Bean due to it breaking apps. Of course, the security in Android is top notch! Great job on that Google. If we want to talk inefficiencies with Apple, we certainly can, but to ignore the Google mess is disingenuous. As far as the effect of Apple's "inefficiencies," is it really affecting them? Are people still buying apps and computers? My last point is that the assertion of Apple "stubbornly refusing to see the flaws.." That's interesting, because unless one works for Apple at a level high enough to be involved in the conversation, that suggestion is merely conjecture. That's right up there with the pundits being disappointed that iWatch has been delayed, despite having no proof or any acknowledgment from Apple that even such a product exists. There's an easy solution to not having to deal with Apple's "inefficiencies" -- don't deal with Apple and go make your money with all of the people paying money for Android apps. Or, respond to one of the hundreds of job listings at Apple and do something about it.
undoware 13y ago
Let us compare: World's premier closed-source shop: (presumably) gets hacked; goes down; stays down. Github, Rubygems, Linux Kernel, etc. get hacked; restore from SHA256SUM'ed backups; keep moving. Turns out what used to be called "hobby" projects matter, because code made without love has a smell, and no one does a "hobby" for anything but. (Remember the 'ama' in 'amateur') (ok ok so kernel.org was down for a while. But remember all the heavy lifting done by git to keep those commits clean. It was just the server, not the data.)
- angersock 13y ago
  You probably have better developers pushing live code over at those other places, I'd bet.
tsenkov 13y ago
Instead of fighting about which platform is the best, I hope all of you will agree that Apple should not leave us in the dark for 15min, let alone 3 days, or not even "post mortem" to answer the question "what happened?".
tater 13y ago
Forstall has the backups.
allwein 13y ago
Apple is set to report their earnings on Tuesday, July 23rd. If this isn't back up by Tuesday evening, it's going to be a very interesting earnings conference call.
xixora1 13y ago
So… Did everyone get the email saying they had an intrusion and they're rebuilding the system?
- vinhnx 13y ago
  yep, I got it a few hours ago.
esalman 13y ago
Just received following email- confirmed breach: Last Thursday, an intruder attempted to secure personal information of our registered developers from our developer website. Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed. In the spirit of transparency, we want to inform you of the issue. We took the site down immediately on Thursday and have been working around the clock since then. In order to prevent a security threat like this from happening again, we’re completely overhauling our developer systems, updating our server software, and rebuilding our entire database. We apologize for the significant inconvenience that our downtime has caused you and we expect to have the developer website up again soon.
navs 13y ago
Apple just released an update (developers should be getting it in their inbox) but still no ETA on when it'll be up. Very unfortunate for those of us waiting to release apps before August.
deleted 13y ago
[deleted]
atgm 13y ago
I just went to check on the dev center and got this e-mail: Last Thursday, an intruder attempted to secure personal information of our registered developers from our developer website. Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed. In the spirit of transparency, we want to inform you of the issue. We took the site down immediately on Thursday and have been working around the clock since then. In order to prevent a security threat like this from happening again, we’re completely overhauling our developer systems, updating our server software, and rebuilding our entire database. We apologize for the significant inconvenience that our downtime has caused you and we expect to have the developer website up again soon.

209 points by vinhnx 13y ago

coldtea 13y ago

Gee, thanks Marco, we would have never known ourselves...

nicholassmith 13y ago
That sounds like "I just read the title so I could post a snarky remark". He does put some suggestions forward of what he thinks could be the root cause.
- coldtea 13y ago
  I read the whole article. The suggestions cover all the cases one would that thought himself. He is being Captain Obvious. Including the gem to "set some time aside" as developers, because we might have some work to do after the site comes back up. Everything in the comment is what men wiser than us called "idle speculation".
  - TwistedWeasel 13y ago
    If you don't like reading idle speculation then you should probably avoid 95% of tech blogs.
    - FrankBlack 13y ago
      I think that comment is Quickmeme-worthy. http://www.quickmeme.com/Captain-Hindsight/ http://www.quickmeme.com/Captain-Hindsight/
saurik 13y ago
Marco's audience includes a bunch of people who wouldn't have known this, or might have assumed it's back up: this draws attention back to it for people who like discussing this sort of thing (although, past that point, I largely agree with you, as I'm not certain what really there is to discuss).

nwh 13y ago

My money is personally on massive, unrecoverable data loss. If you've ever poked around with the way that Apple's website works, you can see that the entire place is a huge mess. There's old servers running ancient (pre-2004) perl scripts alongside the brand new iCloud gear. I can't imagine how the authentication for AppleID is working as login details still work on the ancient pages (think pinstripes and glassy buttons). Depending what URL you hit, the webserver is using php3, php4, perl, python or maybe WebObjects (java). At one point I wrote a scraper that was targeting one of their product pages, and kept getting random, unexplainable results. It turned out that one of their product areas was behind a round-robin load balancer, with three completely different apache versions on each server. The page was dying on one but not the other two. In the end I just had to repetitively scrape until I hit a good response. Even the domain for the "maintenance" page for the developer section is telling. It's just a broken template system regurgitating a bit of the homepage. http://devimages.apple.com/maintenance/ http://devimages.apple.com/maintenance/ http://devimages.apple.com/ http://devimages.apple.com/ Truly a hacked together system. Some engineers at Apple must be having a truly awful weekend, no matter the cause and solution.

nicholassmith 13y ago
I've spent more time using the developer resources recently, and I completely agree. Clicking one link will take you an entirely different feeling architecture, like parts have been built on top without thinking about what the impacts could be. It's probably been a house of cards for a long and it's now fallen down.
saurik 13y ago
Interesting; the http://devimages.apple.com/ http://devimages.apple.com/ URL seems to return the underlying contents of the file, bypassing the SSI (I'm so happy they use SSI... seriously: I love SSI ;P). You can then see the raw  directives, and pull the individual parts. (It isn't quite then fair to say that it is a "broken bit of their template system"; it is more that it is a poor way to setup a static large-file caching endpoint, and may itself lead to a security vulnerability. To be clear: that's probably worse ;P.)

NelsonMinar 13y ago

Could you imagine if the Linux Developer Center went down? It'd be nearly impossible to write Linux software, or download Linux development tools, or generate a Linux binary that a device was allowed to run. It'd be awful.

coldtea 13y ago
Nice attempt at snark, but something being fault-proof in this regard doesn't magically make it worthy in others. Like desktop use, for one. People will not adopt Linux for their desktop/laptop even when offered for free. Where by people I mean more than 1% of them.
- ekianjo 13y ago
  Yeah well its not like computers ship without OS so your point does not really make sense. And its not like Linux has marketing unlike all other OSes out there. When you want to compare stuff you got to have similar factors in place otherwise it does not make sense.
- pavanky 13y ago
  That is not totally true. The reason most people stay with Windows is because the hardware ships with it. In my experience Linux usage has about 50% retention rate among those I installed the OS for. If you'd install OSX or Windows on a non techie who grew up using just Linux, I'd think the retention rates would be much lower.
  - briandear 13y ago
    a 50% retention rate? That's not very good. That means that after you've installed Linux, half of the people switch back.
    - jlgreco 13y ago
      No, that is actually stunningly good, considering the alleged state of the Linux desktop and considering the audience that the Linux desktop has traditionally been targeting. It is so good, that I suspect he has a strong selection bias (he is probably only installing Linux for users he thinks may be able to use it fulltime. This selection is probably unconscious.) (And because it always seems to be necessary to add this disclaimer: I've been using Linux fulltime for over a decade; I'm not a critic, just a realist.)

peterkelly 13y ago

I would be very interested to see what the implications would be if this had happened to iCloud. I'm sure Apple are as unhappy about whatever has happened as the rest of us (likely much more so), but I think at least some communication from them about it would be in order.

ttflee 13y ago

> or device provisioning and certificates (potentially very profitable) Well, an `App'(, the name of which I would not like to identify, ) which was installed using Safari exploits and whose intended use is to help users search and install apps free of charge from AppStore, is still up and running. I guess there exist various exploits up and down the App Store chains, till now.

trackztar 13y ago

If it weren't Sunday, I wouldn't speculate, but it is. My 2 cents: iOS7 updates galore. Just imagine everything that could be overhauled. Even the old gray textured background we see is not very much like iOS 7. iOS is in a big transition here. You can't even update apps at all unless you now include 'widescreen' support, for example. If you don't, iTunes reports an Invalid Binary (no default 586 image, etc). So I would guess a huge overhaul, and typical Apple, is taking care of that vs. arguing or commenting on theories.

tylermac1 13y ago
Wouldn't they offer a warning that the dev center would be down if it's just a design overhaul?
- deleted 13y ago
  [deleted]
grey-area 13y ago
So I would guess a huge overhaul, and typical Apple, is taking care of that vs. arguing or commenting on theories. A design overhaul wouldn't require the dev centre to go down at all - they could just prepare all the assets etc on testing servers and switch them over when they are ready. Given the normal warning given on any maintenance, and the obvious negative consequences for Apple's business of any extended outage (extended in this case being over a day or so), this is unintended, and most likely caused by a security breach. I think we can safely rule out a design overhaul unless Apple are incredibly incompetent. NB itunesconnect is still up (the bit which deals with app upload, itunes store metadata etc), only the dev center - dealing with certs/device registration/distribution etc - is down.
- dkuntz2 13y ago
  Perhaps they're also trying to pull together all the disparate things running on the dev servers? nwh's comment ( https://news.ycombinator.com/item?id=6078854 https://news.ycombinator.com/item?id=6078854 ) says they've got a lot of different systems running, maybe they're trying to unify them to be on one system, making upgrading/changing easier in the future?

Tloewald 13y ago

Amazon bas been down for as much as a half day in recent memory — and not dev stuff but their customer-facing money-making site, and mid-week too.

mullingitover 13y ago
Apple's customer-facing store was down last night for several hours.
- Terretta 13y ago
  Doesn't matter. People aren't going to decide, gee, I actually wanted a Toshiba.
  - MAGZine 13y ago
    people also aren't going to decide 'gee, i'm going to go to x store and pay more for something' anymore then they'll go to BB to buy their apple product of choice.
briandear 13y ago
How many times has AWS gone down taking with it half the web? That to me is a bigger issue than not being able to download iOS beta 3 for a few days.

qnk 13y ago

What's even worse is that we, the developers, their customers, will never get to know what happened. Because that's the Apple way. I'd really love that Apple proves me wrong on this one and comes clean on the problem, the cause and prevention measures being put in place so this won't happen again, whatever it is.

general_failure 13y ago
Nope they won't say a thing and they will get away with it. Oh the things apple devs have to pout upon with. In another world HN would be up in arms about how to manage downtime, how this is u acceptable and how they are going to switch to somebody else immediately. But not for apple because they have no choice...
k-mcgrady 13y ago
They'll have to come clean on this. People pay for this service AND many businesses rely on it. If I'm correct enterprise businesses won't be able to add new devices to the system for example. Apple will need a good explanation for this and if it is a security problem they'll probably be issuing password reset requests and maybe invalidating certificates - you can't just do those things without an explanation.

kailuowang 13y ago

I have received 6 "How to reset your Apple ID password" Email from Apple during the last couple of days, none of which was triggered by me. Could this be related?

swalsh 13y ago
That's funny, a few days ago i received several too.
sarreph 13y ago
Definitely. http://www.zdnet.com/amid-extended-apple-developer-site-downtime-users-report-unauthorized-password-reset-emails-7000018333/ http://www.zdnet.com/amid-extended-apple-developer-site-down...
FPSDavid 13y ago
Same here.
tsenkov 13y ago
This makes our best guess to be "security breach".
- rimantas 13y ago
  I receive similar emails from Gmail now and then. Does not me think that Gmail had been compromised.
  - MAGZine 13y ago
    right, but we don't have multiple people all confirming GMail password resets while GMail just so conveniently happens to be down.
mrpizzadelivery 13y ago
Was anyone else prompted to "update their expired password" when attempting to login to iTunesConnect? I did have the same password for about three years so it didn't seem too odd. Of course I double-checked the url and it looked good. Also I waited for a few days in the hopes that someone else would notice if this was a scam of some sort. Unfortunately after updating my password I forgot the new one. At least I think I forgot as my new password (as best I can remember it) doesn't work.

slowdown 13y ago

Why is a mere unfounded speculation not backed up by any facts written to garner pageviews featured on the frontpage?

freehunter 13y ago
Because there are no facts, Apple hasn't said a word. The alternative is that we ignore this. At least with this article (which no one has to read), there's a place for HN Apple devs to talk amongst each other.
- slowdown 13y ago
  >a place for HN Apple devs to talk amongst each other So, you're telling me that I need to go read his article, earn him pageviews and ad money and come back here to discuss about the issue? I think a Ask HN post would have done the job, much better, no? Just saw your edit: >(which no one has to read) wut?
  - freehunter 13y ago
    Edit? No, I wrote that in one try. No one has to read the article, and it seems like a lot of people don't read the article before commenting. The comment box doesn't give you a quiz on the article before letting you comment.
- hrktb 13y ago
  It goes very deep in meta territory, but hey, it's not as if there were any fact intersting to discuss. I remember a lot of discussions a year or two ago when marco was very vocal about the reasons he didn't have comments on his blog, and would prefer to have all of these on HN for e.g. slowdown's comment goes the other way round, on why HN should care about commenting on marco's blog. There is another thread [1] with 138 comments and more quality opinions and speculations than anyone should need, Marco's speculations are also represented in the top comments. Agree or not with he's stance, it's a valid point IMO. I think it's fun to see this kind of reaction on why some contents should be commented or not, and the motivations to push a site or another. [1] https://news.ycombinator.com/item?id=6071233 https://news.ycombinator.com/item?id=6071233

Jgrubb 13y ago

This reminds me, yesterday i got an obviously spoofed phishing email from "apple" telling me to reset my passwords and reenter my CC info. Anybody else get that? http://imgur.com/hyta4bC http://imgur.com/hyta4bC

tazjin 13y ago
Can you have a look at the mail headers and check if it was just a standard phishing mail?
- Jgrubb 13y ago
  The "update now" link was to some completely other site, so without having checked out the headers yet I'd say it's a safe bet.
  - Achshar 13y ago
    maybe they were using some parked or addon domain? I know the chances are that it was not, but the entire thread talks about the poorly designed apple dev backend, which includes links like that.
rossjudson 13y ago
It never ceases to amaze me that such emails are carefully constructed in the graphical sense, but miserable failures in the grammatical.
- ams6110 13y ago
  The British spelling of "apologise" is also a giveaway.

zackmorris 13y ago

Apple's websites are among the worst corporate sites I've ever used. I don't say that just to be critical or cynical (ahem), I'm merely pointing out that they have tremendous room for improvement. As opposed to say php.net or stackoverflow.com which still give me that fresh air "why can't all sites be like this?" feeling every time I visit them. Most of the time when I visit Apple's sites now, I just assume that what I need to find will either be buried in a convoluted maze, or simply won't exist, and I'll find myself on a "this has been deprecated" 404-style page which takes me someplace only loosely related to what I was looking for. So I end up back at google to try to find a copy of the information either cached somewhere or offsite. Simply being an Apple developer is a chore. Keeping up with yearly certificate expirations is taxing when you are contracting for several clients. And they never really worked out an easy way to allow several developers to share certs. I just assume now that the other developers will invalidate whatever shared cert I made. The situation is bad enough, and exacerbated by Apple stubbornly refusing to see the flaws, that I wish a startup would encapsulate the friction and just take care of all the minutia for me. I should never have to personally deal with provisioning. Anything short of a one click submission to iTunes Connect is reminiscent of all the TCP/IP details that we used to have to put in our modems in the dialup days, when all that should have been required was a phone number and passcode. I can't gently forgive them for it. So I think this downtime could be a wakeup call for them that the inefficiencies in their system are even costing them now.

briandear 13y ago
Have you spent any time on the Verizon or Sirius XM sites? How about the average multinational insurance company? "Among the worst corporate sites" is a huge leap, considering the profound amount of crap. Spending 5 minutes using the average electronic medical records (EMR) web application would be enough to make your hair fall out. Forget about interoperability of EMR formats. It's a mess. I'm not defending Apple at all, I've spent many nights throwing iPhones over the provisioning and certificate process, but to me, the outcome of surviving that process is that I'm making money from the apps, so it's not ideal, but at least my iOS users are actually paying me money, as opposed to Android users that, as a group tend to expect something for nothing. We ought to also look at the inefficiencies of the Android system while we're here. 35% of users are still on Gingerbread! You can't really expect to earn money as a developer if you ignore 35% of your target market, with iOS we have over 90% running iOS 6 and only about 6% running Android 4.2, which means that developers can't take advantage of a new Android feature without leaving behind the majority of their market, or creating multiple versions and then on top of that, having to test on a myriad of different hardware configurations. The fact also remains that Android users are generally cheap -- they don't like to pay for apps. So you have a highly fragmented OS environment, coupled with a user base that, in generally spends much less that iOS users and, on top of that, you have a royal mess in the copy-protection scheme used in Android -- which had to be disabled in Jelly Bean due to it breaking apps. Of course, the security in Android is top notch! Great job on that Google. If we want to talk inefficiencies with Apple, we certainly can, but to ignore the Google mess is disingenuous. As far as the effect of Apple's "inefficiencies," is it really affecting them? Are people still buying apps and computers? My last point is that the assertion of Apple "stubbornly refusing to see the flaws.." That's interesting, because unless one works for Apple at a level high enough to be involved in the conversation, that suggestion is merely conjecture. That's right up there with the pundits being disappointed that iWatch has been delayed, despite having no proof or any acknowledgment from Apple that even such a product exists. There's an easy solution to not having to deal with Apple's "inefficiencies" -- don't deal with Apple and go make your money with all of the people paying money for Android apps. Or, respond to one of the hundreds of job listings at Apple and do something about it.

undoware 13y ago

Let us compare: World's premier closed-source shop: (presumably) gets hacked; goes down; stays down. Github, Rubygems, Linux Kernel, etc. get hacked; restore from SHA256SUM'ed backups; keep moving. Turns out what used to be called "hobby" projects matter, because code made without love has a smell, and no one does a "hobby" for anything but. (Remember the 'ama' in 'amateur') (ok ok so kernel.org was down for a while. But remember all the heavy lifting done by git to keep those commits clean. It was just the server, not the data.)

angersock 13y ago
You probably have better developers pushing live code over at those other places, I'd bet.

tsenkov 13y ago

Instead of fighting about which platform is the best, I hope all of you will agree that Apple should not leave us in the dark for 15min, let alone 3 days, or not even "post mortem" to answer the question "what happened?".

tater 13y ago

Forstall has the backups.

allwein 13y ago

Apple is set to report their earnings on Tuesday, July 23rd. If this isn't back up by Tuesday evening, it's going to be a very interesting earnings conference call.

xixora1 13y ago

So… Did everyone get the email saying they had an intrusion and they're rebuilding the system?

vinhnx 13y ago
yep, I got it a few hours ago.

esalman 13y ago

Just received following email- confirmed breach: Last Thursday, an intruder attempted to secure personal information of our registered developers from our developer website. Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed. In the spirit of transparency, we want to inform you of the issue. We took the site down immediately on Thursday and have been working around the clock since then. In order to prevent a security threat like this from happening again, we’re completely overhauling our developer systems, updating our server software, and rebuilding our entire database. We apologize for the significant inconvenience that our downtime has caused you and we expect to have the developer website up again soon.

navs 13y ago

Apple just released an update (developers should be getting it in their inbox) but still no ETA on when it'll be up. Very unfortunate for those of us waiting to release apps before August.

deleted 13y ago

[deleted]

atgm 13y ago

I just went to check on the dev center and got this e-mail: Last Thursday, an intruder attempted to secure personal information of our registered developers from our developer website. Sensitive personal information was encrypted and cannot be accessed, however, we have not been able to rule out the possibility that some developers’ names, mailing addresses, and/or email addresses may have been accessed. In the spirit of transparency, we want to inform you of the issue. We took the site down immediately on Thursday and have been working around the clock since then. In order to prevent a security threat like this from happening again, we’re completely overhauling our developer systems, updating our server software, and rebuilding our entire database. We apologize for the significant inconvenience that our downtime has caused you and we expect to have the developer website up again soon.