6 ms·

I am so tired of the "someone else's server" trope as if there is no value add. S3 is a fantastic piece of architecture and service and the fact that the commo

I am so tired of the "someone else's server" trope as if there is no value add. S3 is a fantastic piece of architecture and service and the fact that the common dev has such tools and service available instantly from their personal laptop or even phone is mind boggling. We are very fortunate that AWS led more services being made available to common folk than what had been done in the past. Its not all about cost or hardware ownership--the value these near infinitely scalable services is easy to ignore but astounding at the grass roots level.

anonym29 3y ago
As someone who constantly scrapes and indexes every bit in every public S3 bucket I can find, for purposes of selling query access to my indexes, I couldn't agree more - FANTASTIC value add. Of course, I never have, don't now, and never would run a business on someone else's computer though... all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s
- cebert 3y ago
  Do you feel like that practice is unethical at all?
  - johnnyworker 3y ago
    Probing for open buckets and selling the results for people likely looking to do nasty things with the is unethical IMO, but it's still more ethical to do it and then bring it up to warn others of it, than to just do it. I mean, how do you know if the people talking about this fantastic value add at the fingertips of anyone are not doing it? When it comes to doing basic napkin math and publish the results, that I find highly ethical, actually.
    - chii 3y ago
      > it's still more ethical to do it and then bring it up to warn others of it, than to just do it. ala, i was gonna murder you, but at least you were warned!
      - johnnyworker 3y ago
        It took me 3 secs to find this: https://buckets.grayhatwarfare.com/ https://buckets.grayhatwarfare.com/ So I think the practical difference is mostly between praising S3 (as synonym for "the cloud" when Amazon wasn't even mentioned, no less) and warning at least to use it diligently, which when it comes to costs might often be not using it at all.
        leetrout 3y ago
        The context of the conversation is on tape drives which is about mass storage. The comment I replied to said: > someone else's server in Virgina While everyone may operate there now it was exceedingly common in mu experience to be referring to AWS when referencing Dulles / Ashburn / NoVA or just Virginia since AWS started there first. So I figure S3 is the correct comparison to not having a 50 TB tape drive under your desk when "the cloud" is being discussed.
  - anonym29 3y ago
    I feel like companies collecting sensitive information and slapping it on someone else's computer without the slightest clue how to protect it, or running a cloud hyperscaler and leaving storage open and indexed by default is far more unethical than anything I'm doing. I'm not the unethical collector, I'm not the one who decided to host that collected info on the cloud, and I'm not the CSP who gave those unethical collectors shitty security defaults. I'm just the messenger notifying others that this is happening. US case law is abundantly clear that web scraping is 100% legal. Further, I have never signed any agreement with AWS (I don't even have an AWS account) regarding AUP or anything like that, so I have no contractual restrictions on this either.
- LoganDark 3y ago
  > all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s The benefit is that you don't have to buy the entire server in order to use a portion of the resources on it. If you're going to use the entire server, buy the entire server — if you're only going to use a portion of a server, why buy an entire server?
  - chii 3y ago
    > if you're only going to use a portion of a server only if your data usage decreases over time. There's a break even point, and that break even point is closer than you think. But by then, you're locked in. It is also the same reason why they are able to charge you more money than a self-setup storage system, since once you're locked in, the cost of switching is almost always just slightly higher than staying.
  - anonym29 3y ago
    > if you're only going to use a portion of a server, why buy an entire server? Unless your compute needs fall below those of an Rpi Zero, it is almost always cheaper to buy physical hardware once that doesn't have a monthly bill attached. Sure, scalability is a real value proposition from the hyperscalers that isn't the same with physical hardware, but is scalability worth paying 10x+ in OpEx for what could've been 1x in CapEx? For most businesses, I have a hard time making financial sense of that one.
    - leetrout 3y ago
      Do you sign long term lease agreements for colo space, power and bandwidth for the hardware or are you still paying that annually/monthly? There are no free lunches. I do not disagree buying hardware is cheaper but operational overhead and opportunity cost are real.
      - LoganDark 3y ago
        Eh, buying hardware isn't necessarily cheaper if you factor in operational costs like power, cooling, and especially internet (which gets very expensive very quickly for servers). Datacenters get to spread this cost out over many servers, small installations don't.

anonym29 3y ago

As someone who constantly scrapes and indexes every bit in every public S3 bucket I can find, for purposes of selling query access to my indexes, I couldn't agree more - FANTASTIC value add. Of course, I never have, don't now, and never would run a business on someone else's computer though... all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s

cebert 3y ago
Do you feel like that practice is unethical at all?
- johnnyworker 3y ago
  Probing for open buckets and selling the results for people likely looking to do nasty things with the is unethical IMO, but it's still more ethical to do it and then bring it up to warn others of it, than to just do it. I mean, how do you know if the people talking about this fantastic value add at the fingertips of anyone are not doing it? When it comes to doing basic napkin math and publish the results, that I find highly ethical, actually.
  - chii 3y ago
    > it's still more ethical to do it and then bring it up to warn others of it, than to just do it. ala, i was gonna murder you, but at least you were warned!
    - johnnyworker 3y ago
      It took me 3 secs to find this: https://buckets.grayhatwarfare.com/ https://buckets.grayhatwarfare.com/ So I think the practical difference is mostly between praising S3 (as synonym for "the cloud" when Amazon wasn't even mentioned, no less) and warning at least to use it diligently, which when it comes to costs might often be not using it at all.
      - leetrout 3y ago
        The context of the conversation is on tape drives which is about mass storage. The comment I replied to said: > someone else's server in Virgina While everyone may operate there now it was exceedingly common in mu experience to be referring to AWS when referencing Dulles / Ashburn / NoVA or just Virginia since AWS started there first. So I figure S3 is the correct comparison to not having a 50 TB tape drive under your desk when "the cloud" is being discussed.
- anonym29 3y ago
  I feel like companies collecting sensitive information and slapping it on someone else's computer without the slightest clue how to protect it, or running a cloud hyperscaler and leaving storage open and indexed by default is far more unethical than anything I'm doing. I'm not the unethical collector, I'm not the one who decided to host that collected info on the cloud, and I'm not the CSP who gave those unethical collectors shitty security defaults. I'm just the messenger notifying others that this is happening. US case law is abundantly clear that web scraping is 100% legal. Further, I have never signed any agreement with AWS (I don't even have an AWS account) regarding AUP or anything like that, so I have no contractual restrictions on this either.
LoganDark 3y ago
> all the hyperscalers will happily charge you more for 3 months worth of compute than the cost of the hardware the compute is being done on. Would I rather build a server for $5k or pay $1600/mo to rent that exact same server? Such a difficult decision! /s The benefit is that you don't have to buy the entire server in order to use a portion of the resources on it. If you're going to use the entire server, buy the entire server — if you're only going to use a portion of a server, why buy an entire server?
- chii 3y ago
  > if you're only going to use a portion of a server only if your data usage decreases over time. There's a break even point, and that break even point is closer than you think. But by then, you're locked in. It is also the same reason why they are able to charge you more money than a self-setup storage system, since once you're locked in, the cost of switching is almost always just slightly higher than staying.
- anonym29 3y ago
  > if you're only going to use a portion of a server, why buy an entire server? Unless your compute needs fall below those of an Rpi Zero, it is almost always cheaper to buy physical hardware once that doesn't have a monthly bill attached. Sure, scalability is a real value proposition from the hyperscalers that isn't the same with physical hardware, but is scalability worth paying 10x+ in OpEx for what could've been 1x in CapEx? For most businesses, I have a hard time making financial sense of that one.
  - leetrout 3y ago
    Do you sign long term lease agreements for colo space, power and bandwidth for the hardware or are you still paying that annually/monthly? There are no free lunches. I do not disagree buying hardware is cheaper but operational overhead and opportunity cost are real.
    - LoganDark 3y ago
      Eh, buying hardware isn't necessarily cheaper if you factor in operational costs like power, cooling, and especially internet (which gets very expensive very quickly for servers). Datacenters get to spread this cost out over many servers, small installations don't.