4 ms·

I didn't know so many sites were depending on Fastly. Stack Overflow, GitHub, reddit, .... Even pip is unavailable. My development workflow is completely janked

by barosl 5y ago

I didn't know so many sites were depending on Fastly. Stack Overflow, GitHub, reddit, .... Even pip is unavailable. My development workflow is completely janked up. It is a bit scary that we are putting too many eggs in one basket.

liveoneggs 5y ago
fastly gives free service to things like pip. It's actually very nice.
- JulianWasTaken 5y ago
  Bit pedantic, but it's PyPI that Fastly gives services to, not pip (and PyPI that's down, not pip). The two are only loosely related – pip is a piece of software.
joshenders 5y ago
Blame site operators that are single homing and not loadbalancing CDNs
- notyourday 5y ago
  For sites of any complexity with any dynamic content having CDN redundancy is akin to being multi-cloud -- it is not worth the effort. A lot of dynamic sites use Fastly for its programmatic edge control and a near immediate ( ~1s-4s, typically around 2 ) global cache invalidation for any tagged objects with a single call to the tag. That feature alone simplifies backend logic significantly. To make this feature portable to CDNs that do not support it and provide only regular cache invalidation requires a complicated workflow setup which significantly increases the cache bust time, which in turn removes all the advantages of the treat dynamic content as static and cache bust on write approach.
  - joshenders 5y ago
    >> For sites of any complexity with any dynamic content having CDN redundancy is akin to being multi-cloud — it is not worth the effort. I proposed and lead our multi-CDN project at Pinterest for both static and dynamic content and I can tell you, many many times over, it has been well worth the effort. Everybody should do this if not only for contract negotiating leverage. Cache invalidation is fast enough on all CDNs now for most use cases (yes, including Akamai). But realistically, most sites (Pinterest included) are not using clever cache invalidation for dynamic content because it’s not worth the integration effort (and it’s very difficult to abstract for large 1k+ engineering teams). Most customers are just using DSAs for the L4/L5 benefits (both security and perf). In that case, it’s not complicated to implement multi-cdn.
Zyansheep 5y ago
Must... decentralize... internet...
samhh 5y ago
Hackage (Haskell) is down as well: http://hackage.haskell.org http://hackage.haskell.org
- fgaz 5y ago
  The mirrors still work though, and cabal will just fall back to those
kypro 5y ago
You would think sites like Github and key government sites would at least have a fall back at the ready. It reasonable to use a CDN like Fastly, but having a single point of failure seems silly if you're the BBC or Gov UK. Although, it does seem BBC managed to get back up and running pretty quick so perhaps they were prepared for this.
- danpalmer 5y ago
  Gov.UK is back up too. They have a mandate from government to be able to provide emergency communications so I expect they did have a backup and have managed to switch over, but just took 30 mins to do so. Gov.UK is supposed to be a bit like BBC 1 or Radio 1 – in a national emergency they can be taken over to disseminate critical information, like if there was a nuclear attack launched on the UK.
  - ekzy 5y ago
    no, it's just that the incident has just been fixed by fastly https://status.fastly.com/incidents/vpk0ssybt3bj https://status.fastly.com/incidents/vpk0ssybt3bj

by barosl 5y ago

liveoneggs 5y ago

fastly gives free service to things like pip. It's actually very nice.

JulianWasTaken 5y ago
Bit pedantic, but it's PyPI that Fastly gives services to, not pip (and PyPI that's down, not pip). The two are only loosely related – pip is a piece of software.

joshenders 5y ago

Blame site operators that are single homing and not loadbalancing CDNs

notyourday 5y ago
For sites of any complexity with any dynamic content having CDN redundancy is akin to being multi-cloud -- it is not worth the effort. A lot of dynamic sites use Fastly for its programmatic edge control and a near immediate ( ~1s-4s, typically around 2 ) global cache invalidation for any tagged objects with a single call to the tag. That feature alone simplifies backend logic significantly. To make this feature portable to CDNs that do not support it and provide only regular cache invalidation requires a complicated workflow setup which significantly increases the cache bust time, which in turn removes all the advantages of the treat dynamic content as static and cache bust on write approach.
- joshenders 5y ago
  >> For sites of any complexity with any dynamic content having CDN redundancy is akin to being multi-cloud — it is not worth the effort. I proposed and lead our multi-CDN project at Pinterest for both static and dynamic content and I can tell you, many many times over, it has been well worth the effort. Everybody should do this if not only for contract negotiating leverage. Cache invalidation is fast enough on all CDNs now for most use cases (yes, including Akamai). But realistically, most sites (Pinterest included) are not using clever cache invalidation for dynamic content because it’s not worth the integration effort (and it’s very difficult to abstract for large 1k+ engineering teams). Most customers are just using DSAs for the L4/L5 benefits (both security and perf). In that case, it’s not complicated to implement multi-cdn.

Zyansheep 5y ago

Must... decentralize... internet...

samhh 5y ago

Hackage (Haskell) is down as well: http://hackage.haskell.org http://hackage.haskell.org

fgaz 5y ago
The mirrors still work though, and cabal will just fall back to those

kypro 5y ago

You would think sites like Github and key government sites would at least have a fall back at the ready. It reasonable to use a CDN like Fastly, but having a single point of failure seems silly if you're the BBC or Gov UK. Although, it does seem BBC managed to get back up and running pretty quick so perhaps they were prepared for this.

danpalmer 5y ago
Gov.UK is back up too. They have a mandate from government to be able to provide emergency communications so I expect they did have a backup and have managed to switch over, but just took 30 mins to do so. Gov.UK is supposed to be a bit like BBC 1 or Radio 1 – in a national emergency they can be taken over to disseminate critical information, like if there was a nuclear attack launched on the UK.
- ekzy 5y ago
  no, it's just that the incident has just been fixed by fastly https://status.fastly.com/incidents/vpk0ssybt3bj https://status.fastly.com/incidents/vpk0ssybt3bj