4 ms·
This is awesome and much needed. Just throwing it out there, but would you consider making a dump of the data you scraped that could be used by data scientists
by dave_sullivan 10y ago
This is awesome and much needed.
Just throwing it out there, but would you consider making a dump of the data you scraped that could be used by data scientists? Maybe as a torrent or something like that? Data about movies and what people say about them could form the basis of a lot of NLP projects.
What other big datasets are there for forum post text data? The reddit dataset most immediately comes to mind, and I've also seen a similar one for HN comments. Any others?
- mynewtb 10y agoArchiveTeam's web archives will be available to everyone without restrictions or profit as usual.
- JimSmith84 10y agoYeah that's definitely something I could do. Are there any proposed projects you know of that could benefit from this type of data?
- anilgulecha 10y agoEverything from simple sentiment analysis, to archive.org, to another mirror. I hope that does not discourage you from releasing the data. Edit: I see the other comment about archive team already collecting and releasing this data, for free in an open format. I think that will be a good first source as well.
- waynecolvin 10y agoI think the priority was to reenable discussion of new movies and TV shows, that would be useful moving forward. But maybe they could make an API.
- johnymontana 10y agoI'd love to explore how this data could be used to enhance recommender systems.
- endymi0n 10y agoAFAIK, Jinni is doing exactly that.