7 ms·

Spotify is down due to this, which is, uh, pretty hilarious https://news.spotify.com/us/2016/02/23/announcing-spotify-infrastructures-googley-future/ https://ne

Spotify is down due to this, which is, uh, pretty hilarious https://news.spotify.com/us/2016/02/23/announcing-spotify-infrastructures-googley-future/ https://news.spotify.com/us/2016/02/23/announcing-spotify-in...

outside1234 11y ago
I bet they and Google will have a positive post written up about it tomorrow though. :)
draw_down 11y ago
I remember people speculating that Spotify most likely received a big discount for doing this. Guess you get what you pay for ;)
- trhway 11y ago
  Google provides 99.95% - https://cloud.google.com/appengine/sla https://cloud.google.com/appengine/sla That is like 17 of such 15 minutes breaks per year, i.e. an allowance for one small (or a large fixed quickly) screwdup/month :)
  - ec109685 11y ago
    This allows unlimited small outages: "Downtime Period" means, for an Application, a period of five consecutive minutes of Downtime. Intermittent Downtime for a period of less than five minutes will not be counted towards any Downtime Periods.
  - jpatokal 11y ago
    99.95% is correct, but the Compute Engine SLA is actually here: https://cloud.google.com/compute/sla https://cloud.google.com/compute/sla Today's incident did not impact App Engine at all. (Disclaimer: I work in Google Cloud Support.)
r0fls 11y ago
It seems to be working fine, I just tested.
- BinaryIdiot 11y ago
  Outage lasted about 16 minutes
Mafana0 11y ago
I see your point. Based on @SpotifyStatus [0], it wasn't uncommon for Spotify to have service disruptions before they did the move though. [0]: https://twitter.com/SpotifyStatus https://twitter.com/SpotifyStatus
eva1984 11y ago
To be fair, AWS has some significant down time in the second half of last year.
- cavisne 11y ago
  Not across multiple regions though. It's not trivial to make an application cross region but at least there is a way to engineer around an outage, unlike this outage
jasonjei 11y ago
Was Google services (search, email, drives, apps) impacted at all?
- asadlionpk 11y ago
  No they don't use gcloud to host their own apps. Yes that's ridiculous.
  - morgante 11y ago
    Which is exactly why I won't use GCE. If Google isn't confident enough to use it for themselves, neither will I. The fact that Amazon dogfoods AWS is a major advantage for them.
    - brown9-2 11y ago
      Even if Amazon.com uses AWS (the extent of which seems it may be a mixture of marketing hype and urban legend), there are many ways AWS could fail that affects customers but leaves Amazon.com unaffected.
- tempestn 11y ago
  Google Custom Search was down for a similar time period. (Outage lasted longer than GCE, but seems likely related.)
- kahwooi 11y ago
  It would be hilarious if they use to host their services on AWS or Azure.
chris_wot 11y ago
And they won't be down again in a long, long time.
ryankupyn 11y ago
If Spotify wanted to be really sneaky, some amount of downtime might be good for them financially. The bulk of their revenue comes from customers who subscribe on a per-month basis, while they pay out royalties on a per-song-played basis. This outage is reducing the amount they have to pay, and if the outage-elasticity-of-demand is low enough they could (hypothetically) come out ahead!
- flashman 11y ago
  > while they pay out royalties on a per-song-played basis I believe this is inaccurate. They pay out royalties on a share-of-all-plays basis, don't they?[0] So an outage wouldn't reduce the payout amount, it would just slightly alter the balance of payments for individual rightsholders. [0]http://www.spotifyartists.com/spotify-explained/#royalties-in-detail http://www.spotifyartists.com/spotify-explained/#royalties-i...: "That 70% is split amongst the rights holders in accordance with the popularity of their music on the service. The label or publisher then divides these royalties and accounts to each artist depending on their individual deals... Spotify does not calculate royalties based upon a fixed “per play” rate."
  - itsnotvalid 11y ago
    Rather, as they don't have SLA to end-users, they get credits from Google, so they still earn money.
    - kahwooi 11y ago
      But the reputation is damage.
trevor-e 11y ago
I was getting really frustrated at the gym when the Spotify app wouldn't work. Didn't expect to find the answer here.
sundvor 11y ago
Happy to say I didn't notice; I am using a 517 song offlined playlist ("EVE Online" by Michael Andrew) for my programming work.
- bonestamp2 11y ago
  That sounds awesome, care to share?
  - sundvor 10y ago
    Sorry late - link is https://open.spotify.com/user/1231239981/playlist/3ka1SYnv2bm3HgMi5x248d https://open.spotify.com/user/1231239981/playlist/3ka1SYnv2b... . Hope you see this post and enjoy it. I see from the down votes that my reply must have been seen as kind of off topic to the GCE issue, however since Spotify came up as a "victim", I did feel it prudent to mention that Spotify Premium has offline playlists to allow users to weather network issues of any kind. Also for me personally, big playlists of quality music like this one is fantastic for my work.
NetStrikeForce 11y ago
I think it's a great idea to diversify not just regions, but providers. The future is for per-application virtual networks that are agnostic to the underlying hosting provider. These networks work as an overlay, which means that your applications can be moved through providers without changing their architecture at all. You could even shut it down in provider A and start it in provider B without any changes at all. At Wormhole[1] we have identified this problem and solved it. [1]: https://wormhole.network https://wormhole.network

outside1234 11y ago

I bet they and Google will have a positive post written up about it tomorrow though. :)

draw_down 11y ago

I remember people speculating that Spotify most likely received a big discount for doing this. Guess you get what you pay for ;)

trhway 11y ago
Google provides 99.95% - https://cloud.google.com/appengine/sla https://cloud.google.com/appengine/sla That is like 17 of such 15 minutes breaks per year, i.e. an allowance for one small (or a large fixed quickly) screwdup/month :)
- ec109685 11y ago
  This allows unlimited small outages: "Downtime Period" means, for an Application, a period of five consecutive minutes of Downtime. Intermittent Downtime for a period of less than five minutes will not be counted towards any Downtime Periods.
- jpatokal 11y ago
  99.95% is correct, but the Compute Engine SLA is actually here: https://cloud.google.com/compute/sla https://cloud.google.com/compute/sla Today's incident did not impact App Engine at all. (Disclaimer: I work in Google Cloud Support.)

r0fls 11y ago

It seems to be working fine, I just tested.

BinaryIdiot 11y ago
Outage lasted about 16 minutes

Mafana0 11y ago

I see your point. Based on @SpotifyStatus [0], it wasn't uncommon for Spotify to have service disruptions before they did the move though. [0]: https://twitter.com/SpotifyStatus https://twitter.com/SpotifyStatus

eva1984 11y ago

To be fair, AWS has some significant down time in the second half of last year.

cavisne 11y ago
Not across multiple regions though. It's not trivial to make an application cross region but at least there is a way to engineer around an outage, unlike this outage

jasonjei 11y ago

Was Google services (search, email, drives, apps) impacted at all?

asadlionpk 11y ago
No they don't use gcloud to host their own apps. Yes that's ridiculous.
- morgante 11y ago
  Which is exactly why I won't use GCE. If Google isn't confident enough to use it for themselves, neither will I. The fact that Amazon dogfoods AWS is a major advantage for them.
  - brown9-2 11y ago
    Even if Amazon.com uses AWS (the extent of which seems it may be a mixture of marketing hype and urban legend), there are many ways AWS could fail that affects customers but leaves Amazon.com unaffected.
tempestn 11y ago
Google Custom Search was down for a similar time period. (Outage lasted longer than GCE, but seems likely related.)
kahwooi 11y ago
It would be hilarious if they use to host their services on AWS or Azure.

chris_wot 11y ago

And they won't be down again in a long, long time.

ryankupyn 11y ago

If Spotify wanted to be really sneaky, some amount of downtime might be good for them financially. The bulk of their revenue comes from customers who subscribe on a per-month basis, while they pay out royalties on a per-song-played basis. This outage is reducing the amount they have to pay, and if the outage-elasticity-of-demand is low enough they could (hypothetically) come out ahead!

flashman 11y ago
> while they pay out royalties on a per-song-played basis I believe this is inaccurate. They pay out royalties on a share-of-all-plays basis, don't they?[0] So an outage wouldn't reduce the payout amount, it would just slightly alter the balance of payments for individual rightsholders. [0]http://www.spotifyartists.com/spotify-explained/#royalties-in-detail http://www.spotifyartists.com/spotify-explained/#royalties-i...: "That 70% is split amongst the rights holders in accordance with the popularity of their music on the service. The label or publisher then divides these royalties and accounts to each artist depending on their individual deals... Spotify does not calculate royalties based upon a fixed “per play” rate."
- itsnotvalid 11y ago
  Rather, as they don't have SLA to end-users, they get credits from Google, so they still earn money.
  - kahwooi 11y ago
    But the reputation is damage.

trevor-e 11y ago

I was getting really frustrated at the gym when the Spotify app wouldn't work. Didn't expect to find the answer here.

sundvor 11y ago

Happy to say I didn't notice; I am using a 517 song offlined playlist ("EVE Online" by Michael Andrew) for my programming work.

bonestamp2 11y ago
That sounds awesome, care to share?
- sundvor 10y ago
  Sorry late - link is https://open.spotify.com/user/1231239981/playlist/3ka1SYnv2bm3HgMi5x248d https://open.spotify.com/user/1231239981/playlist/3ka1SYnv2b... . Hope you see this post and enjoy it. I see from the down votes that my reply must have been seen as kind of off topic to the GCE issue, however since Spotify came up as a "victim", I did feel it prudent to mention that Spotify Premium has offline playlists to allow users to weather network issues of any kind. Also for me personally, big playlists of quality music like this one is fantastic for my work.

NetStrikeForce 11y ago

I think it's a great idea to diversify not just regions, but providers. The future is for per-application virtual networks that are agnostic to the underlying hosting provider. These networks work as an overlay, which means that your applications can be moved through providers without changing their architecture at all. You could even shut it down in provider A and start it in provider B without any changes at all. At Wormhole[1] we have identified this problem and solved it. [1]: https://wormhole.network https://wormhole.network