6 ms·

And the first terabyte of analysis for free in BigQuery!! (Work at g)

by vgt 9y ago

diggan 9y ago
That's all fine and dandy, but I assume I could also pull down the entire dataset for free and run unlimited analysis locally for free?
- IAmEveryone 9y ago
  Yes, although data transfer fees may apply if you go over the free quota.
  - deleted 9y ago
    [deleted]
  - diggan 9y ago
    Ah, I see. These "public" datasets are not actually public as they are only available if you login (and I assume input your payment details before). Shame on me to think "Public Datasets" meant that I could just download them via rsync/http. If you're interested in truly public datasets, see http://academictorrents.com/ http://academictorrents.com/ which hosts datasets as torrents.
    - komali2 9y ago
      This seems critical - as Google is a corporation, I wonder where this sense comes from? Obviously we have certain expectations for a couple who's tagline reads "don't be evil," but does that extend to criticizing them for not providing terabytes of free transfers of terabytes of free data, which they pay to host and serve?
      - diggan 9y ago
        > criticizing them for not providing terabytes of free transfers of terabytes of free data My criticism is not towards them not providing free transfers of data. My criticism is that they say "these datasets are freely hosted and accessible" and public, while only being available while logged in to Google, which I don't think classifies as public. I would not have any problem if it's just called "Google-hosted Datasets" or "Google-only Public Datasets", I just think the current naming is misleading. Edit: compare this to the AWS Public Datasets which are actually available without a AWS account, just go to https://landsat-pds.s3.amazonaws.com/c1/L8/139/045/LC08_L1TP_139045_20170304_20170316_01_T1/index.html https://landsat-pds.s3.amazonaws.com/c1/L8/139/045/LC08_L1TP... for example, which is the Landsat Dataset
        kalcode 9y ago
        A public library still requires a library card... the term public means its available to the public. Signing up google, library card, doesn't negate that fact that any member of the public can access the data with a reasonable and insignificant barrier.
        diggan 9y ago
        Sure, the difference between getting a library card and signing up to a Google account is that usually the first has a user agreement which is about 10 lines long with human language and the second one has a user agreement which is 10+ pages with lawyer speak and probably includes that they are allowed to sell your data. How is it reasonable for a normal person to understand those kind of user agreements? And if that's your definition of public, isn't everything public, if you have the right amount of money, knows the right people and can get access to the right place?
        kalcode 9y ago
        You only pay a price because you are using someone else system. Which cost money. Public library are public, you also pay taxes on them. Because using it as a service still has a cost. Google give you 1TB on data to query each month. That is a lot of data, if you need more for free collect the data over time. But you still need a library card and if the facility gets overused they ask for a small tax to cover the operating costs. I fail to see how this isn't considered public. Public literally just means accessible to the general public. That all. A public event can still charge a fee for entry. As long as the fee doesn't create a barrier for most the public, you know accessible to the general public. Also if a library sold their data would it no longer be considered public? I believe it would still be considered public. User data isn't sold typically by name or email, byt activity. If a library compiled a list of books checked out and their frequency, the amount of people entering everyday etc that be the equivalent of most user data being sold. Very rare for a company to sell your actual personal data, when they do they disassociate your personal information with it. So if the above doesn't disqualify a library from being public, then neither would this dataset that is public. If you really disagree with that then you are just trying to be pedantic at that point.

And the first terabyte of analysis for free in BigQuery!! (Work at g)

by vgt 9y ago

And the first terabyte of analysis for free in BigQuery!! (Work at g)

That's all fine and dandy, but I assume I could also pull down the entire dataset for free and run unlimited analysis locally for free?

IAmEveryone 9y ago
Yes, although data transfer fees may apply if you go over the free quota.
- deleted 9y ago
  [deleted]
- diggan 9y ago
  Ah, I see. These "public" datasets are not actually public as they are only available if you login (and I assume input your payment details before). Shame on me to think "Public Datasets" meant that I could just download them via rsync/http. If you're interested in truly public datasets, see http://academictorrents.com/ http://academictorrents.com/ which hosts datasets as torrents.
  - komali2 9y ago
    This seems critical - as Google is a corporation, I wonder where this sense comes from? Obviously we have certain expectations for a couple who's tagline reads "don't be evil," but does that extend to criticizing them for not providing terabytes of free transfers of terabytes of free data, which they pay to host and serve?
    - diggan 9y ago
      > criticizing them for not providing terabytes of free transfers of terabytes of free data My criticism is not towards them not providing free transfers of data. My criticism is that they say "these datasets are freely hosted and accessible" and public, while only being available while logged in to Google, which I don't think classifies as public. I would not have any problem if it's just called "Google-hosted Datasets" or "Google-only Public Datasets", I just think the current naming is misleading. Edit: compare this to the AWS Public Datasets which are actually available without a AWS account, just go to https://landsat-pds.s3.amazonaws.com/c1/L8/139/045/LC08_L1TP_139045_20170304_20170316_01_T1/index.html https://landsat-pds.s3.amazonaws.com/c1/L8/139/045/LC08_L1TP... for example, which is the Landsat Dataset
      - kalcode 9y ago
        A public library still requires a library card... the term public means its available to the public. Signing up google, library card, doesn't negate that fact that any member of the public can access the data with a reasonable and insignificant barrier.
        diggan 9y ago
        Sure, the difference between getting a library card and signing up to a Google account is that usually the first has a user agreement which is about 10 lines long with human language and the second one has a user agreement which is 10+ pages with lawyer speak and probably includes that they are allowed to sell your data. How is it reasonable for a normal person to understand those kind of user agreements? And if that's your definition of public, isn't everything public, if you have the right amount of money, knows the right people and can get access to the right place?
        kalcode 9y ago
        You only pay a price because you are using someone else system. Which cost money. Public library are public, you also pay taxes on them. Because using it as a service still has a cost. Google give you 1TB on data to query each month. That is a lot of data, if you need more for free collect the data over time. But you still need a library card and if the facility gets overused they ask for a small tax to cover the operating costs. I fail to see how this isn't considered public. Public literally just means accessible to the general public. That all. A public event can still charge a fee for entry. As long as the fee doesn't create a barrier for most the public, you know accessible to the general public. Also if a library sold their data would it no longer be considered public? I believe it would still be considered public. User data isn't sold typically by name or email, byt activity. If a library compiled a list of books checked out and their frequency, the amount of people entering everyday etc that be the equivalent of most user data being sold. Very rare for a company to sell your actual personal data, when they do they disassociate your personal information with it. So if the above doesn't disqualify a library from being public, then neither would this dataset that is public. If you really disagree with that then you are just trying to be pedantic at that point.