5 ms·

Why AWS? It'd doubtlessly be cheaper to throw Flickr onto dedicated servers (by a ton). It would also probably help if they freed the source code of Flickr, so

by kick 7y ago

Why AWS? It'd doubtlessly be cheaper to throw Flickr onto dedicated servers (by a ton). It would also probably help if they freed the source code of Flickr, so others could contribute and make it lighter to run. A site like Flickr is exactly what you want to keep as far away from AWS as possible.

giancarlostoro 7y ago
I agreed. You dont even need crazy SSDs for all the data. You can have spin disks which have gotten cheaper for TB of data for the older images and SSDs for the cached ones (most recently accessed) and voila.
- basch 7y ago
  For write once archiving, bluray is an option too. Although who knows how long the discs last.
kibibyte 7y ago
SmugMug seems to be hosted on AWS (based on doing a DNS lookup and seeing who owns that IP). I wouldn't be surprised if moving it all to AWS was the path of least resistance to an engineering team that understands AWS more than managing a dedicated hardware fleet.
- paranoidrobot 7y ago
  The SmugMug team were the first people I remember seeing use S3 for more than a 'hello world' project. This blog entry from 2006[1] is only 13 years old, but feels like it should be older. Back then, putting that much data for a publicly accessible site was considered pretty insane. [1] https://don.blogs.smugmug.com/2006/08/12/amazon-s3-the-holy-grail/ https://don.blogs.smugmug.com/2006/08/12/amazon-s3-the-holy-...
  - lonelappde 7y ago
    Older how? S3 was invented in 2006. What was S3 used for if not for hosting website data?
    - sct202 7y ago
      At the place I work we transfer files with one of our vendors via s3.
    - paranoidrobot 7y ago
      > Older how? > S3 was invented in 2006. Because looking back at that, it feels odd that it's only 13 years ago, and how much the industry has changed in that time. Back then S3 was this weird thing that Amazon came out with. A file hosting service that didn't let you do anything except store or fetch files. Sure, it's low cost... but I can't process them without downloading them again and paying for the data access (twice!). Why the hell would I want that? SmugMug's use case was the first I saw that actually made sense. Back then, if you weren't able to afford buying a server and putting it in a colo facility, you'd go rent a server on a monthly/yearly basis from the likes of ThePlanet, and host those files yourself on your server. If you needed redundancy, you'd have to figure out syncing between the servers yourself. Now? Spin up an AWS instance or trigger a function, serve a link to the file in S3 or via a cloudfront distribution. Even if you need to process the data before serving it - stream it straight out of S3, through your application and to the user.
sansnomme 7y ago
Stackexchange is probably one of the only companies that's doing hardware efficiently. https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... Everyone else is more than happy to give more money to cloud hosting companies.
- kick 7y ago
  Somewhere on codinghorror.com, Jeff Atwood has a really good post on this.
  - vladd 7y ago
    Are you talking about "The Cloud is Just Someone Else's Computer"? If so, it's available at https://blog.codinghorror.com/the-cloud-is-just-someone-elses-computer/ https://blog.codinghorror.com/the-cloud-is-just-someone-else...
- mey 7y ago
  Backblaze is doing storage cheaper with their own gear.
- basch 7y ago
  Dropbox, ServiceNow? ServiceNow goes so far as to give customers non-shared bare metal.
fragsworth 7y ago
You're forgetting that engineers are expensive, and dedicated servers means spending more engineering time on things. Almost everyone considers hosting their own hardware, and it turns out that for most companies it's just not economical. I really doubt they switched to AWS because it's "cool". Also, isn't CDN bandwidth and data transfer going to be the lion's share of their costs? That's not a problem you can just throw dedicated servers at.
- kick 7y ago
  I'm not forgetting that at all: I don't see how a dedicated server takes any more time than AWS. AWS adds extra things you have to do and manage. Also, data transfer and bandwidth is far cheaper with dedicated and colos than AWS.
  - anyfoo 7y ago
    Physical space, building and upgrading the servers, monitoring and replacing components of the servers and their vast storage arrays when they are broken, upgrading the hardware, making sure all the software is up to date with latest security patches, reaching either end of life of their version and having to perform major upgrades with potential compatibility problems, cable management, all this at remote locations... Vs. managing an Amazon account from a desk, any desk really, you don’t see the appeal?
    - latch 7y ago
      Imagine you're looking for a new place to live. You go to a realtor and he starts showing you McMansions and telling you how great they are. You say: "But those are really expensive, require a ton of maintenance, and hard to sell later on" Your agent looks at you and says: "Yes but do you really went to find land, get the permits, get it developed, deal with all those contractors and deal with the inevitable problems of a new constructions? A McMansion is obviously better." The agent has apparently never heard of apartments, condos, semi-detached and so on (yet somehow holds a very strong opinion). This is a problem; you find a new agent.
    - 7y ago
haecceity 7y ago
What makes Flickr more suited for dedicated hosting rather than awe?
- ryanlol 7y ago
  Money?
- y4mi 7y ago
  Bandwidth is expensive on aws. Image and video hosting is usually significantly cheaper on bare metal where you just get unmetered bandwidth from housing providers
  - lonelappde 7y ago
    You don't know what bulk discounts large customers negotiate. High egress prices are a way to get a few extra bucks off small-potato hypermilers
    - y4mi 7y ago
      You're right, I don't know their discounts. It's still more expensive unless they get paid for using the bandwidth though

by kick 7y ago

giancarlostoro 7y ago

I agreed. You dont even need crazy SSDs for all the data. You can have spin disks which have gotten cheaper for TB of data for the older images and SSDs for the cached ones (most recently accessed) and voila.

basch 7y ago
For write once archiving, bluray is an option too. Although who knows how long the discs last.

kibibyte 7y ago

SmugMug seems to be hosted on AWS (based on doing a DNS lookup and seeing who owns that IP). I wouldn't be surprised if moving it all to AWS was the path of least resistance to an engineering team that understands AWS more than managing a dedicated hardware fleet.

paranoidrobot 7y ago
The SmugMug team were the first people I remember seeing use S3 for more than a 'hello world' project. This blog entry from 2006[1] is only 13 years old, but feels like it should be older. Back then, putting that much data for a publicly accessible site was considered pretty insane. [1] https://don.blogs.smugmug.com/2006/08/12/amazon-s3-the-holy-grail/ https://don.blogs.smugmug.com/2006/08/12/amazon-s3-the-holy-...
- lonelappde 7y ago
  Older how? S3 was invented in 2006. What was S3 used for if not for hosting website data?
  - sct202 7y ago
    At the place I work we transfer files with one of our vendors via s3.
  - paranoidrobot 7y ago
    > Older how? > S3 was invented in 2006. Because looking back at that, it feels odd that it's only 13 years ago, and how much the industry has changed in that time. Back then S3 was this weird thing that Amazon came out with. A file hosting service that didn't let you do anything except store or fetch files. Sure, it's low cost... but I can't process them without downloading them again and paying for the data access (twice!). Why the hell would I want that? SmugMug's use case was the first I saw that actually made sense. Back then, if you weren't able to afford buying a server and putting it in a colo facility, you'd go rent a server on a monthly/yearly basis from the likes of ThePlanet, and host those files yourself on your server. If you needed redundancy, you'd have to figure out syncing between the servers yourself. Now? Spin up an AWS instance or trigger a function, serve a link to the file in S3 or via a cloudfront distribution. Even if you need to process the data before serving it - stream it straight out of S3, through your application and to the user.

sansnomme 7y ago

Stackexchange is probably one of the only companies that's doing hardware efficiently. https://nickcraver.com/blog/2016/02/17/stack-overflow-the-architecture-2016-edition/ https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... Everyone else is more than happy to give more money to cloud hosting companies.

kick 7y ago
Somewhere on codinghorror.com, Jeff Atwood has a really good post on this.
- vladd 7y ago
  Are you talking about "The Cloud is Just Someone Else's Computer"? If so, it's available at https://blog.codinghorror.com/the-cloud-is-just-someone-elses-computer/ https://blog.codinghorror.com/the-cloud-is-just-someone-else...
mey 7y ago
Backblaze is doing storage cheaper with their own gear.
basch 7y ago
Dropbox, ServiceNow? ServiceNow goes so far as to give customers non-shared bare metal.

fragsworth 7y ago

You're forgetting that engineers are expensive, and dedicated servers means spending more engineering time on things. Almost everyone considers hosting their own hardware, and it turns out that for most companies it's just not economical. I really doubt they switched to AWS because it's "cool". Also, isn't CDN bandwidth and data transfer going to be the lion's share of their costs? That's not a problem you can just throw dedicated servers at.

kick 7y ago
I'm not forgetting that at all: I don't see how a dedicated server takes any more time than AWS. AWS adds extra things you have to do and manage. Also, data transfer and bandwidth is far cheaper with dedicated and colos than AWS.
- anyfoo 7y ago
  Physical space, building and upgrading the servers, monitoring and replacing components of the servers and their vast storage arrays when they are broken, upgrading the hardware, making sure all the software is up to date with latest security patches, reaching either end of life of their version and having to perform major upgrades with potential compatibility problems, cable management, all this at remote locations... Vs. managing an Amazon account from a desk, any desk really, you don’t see the appeal?
  - latch 7y ago
    Imagine you're looking for a new place to live. You go to a realtor and he starts showing you McMansions and telling you how great they are. You say: "But those are really expensive, require a ton of maintenance, and hard to sell later on" Your agent looks at you and says: "Yes but do you really went to find land, get the permits, get it developed, deal with all those contractors and deal with the inevitable problems of a new constructions? A McMansion is obviously better." The agent has apparently never heard of apartments, condos, semi-detached and so on (yet somehow holds a very strong opinion). This is a problem; you find a new agent.
  - 7y ago

haecceity 7y ago

What makes Flickr more suited for dedicated hosting rather than awe?

ryanlol 7y ago
Money?
y4mi 7y ago
Bandwidth is expensive on aws. Image and video hosting is usually significantly cheaper on bare metal where you just get unmetered bandwidth from housing providers
- lonelappde 7y ago
  You don't know what bulk discounts large customers negotiate. High egress prices are a way to get a few extra bucks off small-potato hypermilers
  - y4mi 7y ago
    You're right, I don't know their discounts. It's still more expensive unless they get paid for using the bandwidth though