6 ms·

> * They removed the cloudsearch syntax from the search API. Didn't know that. That sucks. Is the massive dataset of reddit posts still on BigTable?

ZoomStop 8y ago
Showing the error "Unable to find table: fh-bigquery:reddit_comments.2015_05" https://bigquery.cloud.google.com/table/fh-bigquery:reddit_comments.2015_05 https://bigquery.cloud.google.com/table/fh-bigquery:reddit_c...
- chatmasta 8y ago
  Wow. To be fair, lots of apps/sites scrape viral content from reddit. I can see the argument for why reddit would want to limit that access, but in reality it's a fool's errand in the same realm as Twitter restricting API access.
- minimaxir 8y ago
  I'm not seeing an error; you may have to display the `fh_bigquery` project first.
- throwawaymath 8y ago
  You can just get the canonical dataset directly at data.pushshift.io. That's the origin of the BigQuery-hosted version.
minimaxir 8y ago
The datasets are still available on BigQuery: https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit_posts?pli=1 https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit... https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit_comments?pli=1 https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit... They are not owned by Reddit.
- chatmasta 8y ago
  > They are not owned by Reddit. You sure about that? I would bet that if reddit requested google take down the data, google would comply. If not Reddit, then who owns it?
  - eco 8y ago
    The individual users own the content they create. The Reddit TOS gives Reddit a broad license to use it but the content itself is owned by the users.
    - chatmasta 8y ago
      So who owns the dataset aggregated from all users?
      - hathathat 8y ago
        More importantly, did the people running analysis on this dataset get explicit permission from the European post authors as required by the GDPR?
  - minimaxir 8y ago
    I mean the BQ datasets aren't managed by Reddit, they're sourced from pushshift, so Reddit wouldn't immediately take them down. Whether or not Reddit could file a DMCA complaint to force it down is up for debate.

ZoomStop 8y ago

Showing the error "Unable to find table: fh-bigquery:reddit_comments.2015_05" https://bigquery.cloud.google.com/table/fh-bigquery:reddit_comments.2015_05 https://bigquery.cloud.google.com/table/fh-bigquery:reddit_c...

chatmasta 8y ago
Wow. To be fair, lots of apps/sites scrape viral content from reddit. I can see the argument for why reddit would want to limit that access, but in reality it's a fool's errand in the same realm as Twitter restricting API access.
minimaxir 8y ago
I'm not seeing an error; you may have to display the `fh_bigquery` project first.
throwawaymath 8y ago
You can just get the canonical dataset directly at data.pushshift.io. That's the origin of the BigQuery-hosted version.

minimaxir 8y ago

The datasets are still available on BigQuery: https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit_posts?pli=1 https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit... https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit_comments?pli=1 https://bigquery.cloud.google.com/dataset/fh-bigquery:reddit... They are not owned by Reddit.

chatmasta 8y ago
> They are not owned by Reddit. You sure about that? I would bet that if reddit requested google take down the data, google would comply. If not Reddit, then who owns it?
- eco 8y ago
  The individual users own the content they create. The Reddit TOS gives Reddit a broad license to use it but the content itself is owned by the users.
  - chatmasta 8y ago
    So who owns the dataset aggregated from all users?
    - hathathat 8y ago
      More importantly, did the people running analysis on this dataset get explicit permission from the European post authors as required by the GDPR?
- minimaxir 8y ago
  I mean the BQ datasets aren't managed by Reddit, they're sourced from pushshift, so Reddit wouldn't immediately take them down. Whether or not Reddit could file a DMCA complaint to force it down is up for debate.