6 ms·

Hi Everyone, My name’s Jim, and I created MovieChat.org as an archive and replacement for IMDB’s message boards which are shutting down this week. For those of

Hi Everyone, My name’s Jim, and I created MovieChat.org as an archive and replacement for IMDB’s message boards which are shutting down this week. For those of you not familiar, the IMDb message boards allowed you to discuss any single movie or tv show with others (there was a separate forum for each movie/show). IMDb recently announced they were shutting down the message boards and its users were furious (there's a petition with close to 10k signatures here: https://www.ipetitions.com/petition/petition-to-keep-the-imdb-messageboards-going https://www.ipetitions.com/petition/petition-to-keep-the-imd...). I ventured out to create an archive (of all the existing posts) and replacement and hence MovieChat.org was born. Key Features of MovieChat.org: 1. Any movie/show on IMDB is also on MovieChat.org (over 4 million and counting) - we have separate boards for each movie/show, just like IMDB 2. I backed up most of the posts for IMDB’s top 10,000 movies/shows - most existing conversations on IMDB should also appear on MovieChat.org - we have over 3 million posts already (and I'm working non-stop to back up even more from IMDB)! Please visit http://MovieChat.org http://MovieChat.org, join or start a discussion, and let me know what you think. If you like it, please spread the word. If there’s anything I can improve, just email me (jim@moviechat.org) and I’ll get on it. Jim jim@moviechat.org

dave_sullivan 10y ago
This is awesome and much needed. Just throwing it out there, but would you consider making a dump of the data you scraped that could be used by data scientists? Maybe as a torrent or something like that? Data about movies and what people say about them could form the basis of a lot of NLP projects. What other big datasets are there for forum post text data? The reddit dataset most immediately comes to mind, and I've also seen a similar one for HN comments. Any others?
- mynewtb 10y ago
  ArchiveTeam's web archives will be available to everyone without restrictions or profit as usual.
  - amyjess 10y ago
    Well, thanks to your comment, I just found out ArchiveTeam exists. Thank you!
  - kenorb 10y ago
    Where you can download the boards?
- JimSmith84 10y ago
  Yeah that's definitely something I could do. Are there any proposed projects you know of that could benefit from this type of data?
  - anilgulecha 10y ago
    Everything from simple sentiment analysis, to archive.org, to another mirror. I hope that does not discourage you from releasing the data. Edit: I see the other comment about archive team already collecting and releasing this data, for free in an open format. I think that will be a good first source as well.
    - waynecolvin 10y ago
      I think the priority was to reenable discussion of new movies and TV shows, that would be useful moving forward. But maybe they could make an API.
  - johnymontana 10y ago
    I'd love to explore how this data could be used to enhance recommender systems.
    - endymi0n 10y ago
      AFAIK, Jinni is doing exactly that.
sergers 10y ago
Hi thanks for posting, looks like it has some potential. one item i didnt like is that if i search for a show, i get hits for each episode. would be good to have one entry per show, and then you can drill down into individual episodes. also would be good feature on ability to rate shows (as a registered user, but i guess IMDB is still offering that functionality)
- huac 10y ago
  So basically reimplement IMDB?
otoburb 10y ago
Thank you for this. I'm curious if IMDB considers their (soon to be former) forum posts under copyright, or if you already have a plan in place. One actually hopes that IMDB doesn't in fact have any claim on posts written by users that happened to be hosted on their site.
- JimSmith84 10y ago
  Yeah that's a good question. Before I became an engineer I practiced IP/copyright law. Also spoke (colloquially) with a few ppl at the company. I did my homework before starting this project and I'm confident we're in a good position :)
  - Guest98123 10y ago
    Could you explain why you think you're in a good position? From what I understand, the comments are owned by the users, and by posting them to IMDb, they allowed IMDb and their affiliates the right to use them. The users did not give you permission to use their data, and you're not associated with IMDb, and therefore do not inherit the rights. Lastly, IMDb specifically states that scraping or extracting data is forbidden without written consent.
    - icemelt8 10y ago
      I wish I could downvote you.
  - otoburb 10y ago
    >>Before I became an engineer I practiced IP/copyright law. I'm sure there's a story there since I would have thought the usual direction, even for a small aggregate number of eventual engineer-patent attorney pairings, would be reversed. Hopefully you can post an "About Us" on MovieChat.org at some point when you feel comfortable. Thanks!
- kristianp 10y ago
  From http://www.imdb.com/board/announcement http://www.imdb.com/board/announcement "...disable our message boards on February 3, 2017 but will leave them open for two additional weeks so that users will have ample time to archive any message board content they'd like to keep for personal use." Note the "personal use" above. Is a website like MovieChat.org considered personal use? From http://www.imdb.com/conditions http://www.imdb.com/conditions , section Copyright: "The compilation of all content included in or made available through any IMDb Service is the exclusive property of IMDb...".
deleted 10y ago
[deleted]
karim 10y ago
Hey Jim – thanks for doing that! I'm curious, how did you back up IMDB's message boards? Did you scrape the whole website or were you able to find a data dump?
- 5_minutes 10y ago
  I wanted to ask this too. Did you scrape it or did it via some api? Anyway, cool project, wish you all the best with it.
  - Markoff 10y ago
    illegally scraped,IMDb doesn't have API for this and automated scraping OP done is against their TOS, not mentioning without consent of users, for instance stole 14 years worth of my posts
    - fwn 10y ago
      He stole them? You mean they are gone now? Because the closest I can imagine happening is an unlicensed reproduction of your content, your credentials included. ... Like any archive service would do it.
    - thebluecat 10y ago
      I take it then, when you posted these 14 years of comments, you didn't want them available to the public to, I don't know, read?! Thank goodness IMDB deleted them all before we happened across them by accident & violated your privacy.
cpeterso 10y ago
Interesting project? The IMDb message boards were a good way to get some additional insight after watching a movie. Or at least give me a good laugh at the trolls. :) Who owns this post data? IMDb? The users? Do you have any plans or ideas for existing IMDb users to somehow claim their username on MovieChat, linking to their archived posts?
- quirkafleeg 10y ago
  > The IMDb message boards were a good way to get some additional insight after watching a movie. Or at least give me a good laugh at the trolls. :) Totally agree. I often also went there before watching a TV show or movie to get an idea what it was about or if I'd like it (though wary of spoiler potential), whereas I almost never read the reviews there, which I find are often far more useless than the message board comments. After they're gone IMDB will be just a soulless movie listing site, with no reason to visit other than to check a mark out of ten, and no reason to stick around afterwards. :/
nso 10y ago
I made a userscript to be a drop-in replacement for the current discussion-teaser on a movies page on imdb, but it gets the discussions from moviechat.org (and links to them directly). For it to work I would need for Jim to enable Access-Control-Allow-Origin from imdb.com (and ideally make a json/xml version of the thread list since I'm regexp'ing the html at this point). Maybe it would help persuade him if we showed that there was some interest for this. https://imma.gr/66880x4ca8b https://imma.gr/66880x4ca8b
trippycheez 10y ago
Thank you, THANK YOU, THANK YOU!!!! I just heard about the IMDb message board deletion and was feeling deeply sad about it. I love cinema, and their message board (despite its flaws) was a valuable resource. It was the only place where you could go to participate in in-depth discussions of even the most obscure films. I used it for 14 years and learned so much from other users. As far as I'm concerned you are a SAINT for preserving this bit of film and internet history. Major gratitude to you for your caring and putting the new site together. I honestly want to cry, I am so relieved.
Rand0mPers0n 10y ago
Great work man... Seriously, thank you.

dave_sullivan 10y ago

This is awesome and much needed. Just throwing it out there, but would you consider making a dump of the data you scraped that could be used by data scientists? Maybe as a torrent or something like that? Data about movies and what people say about them could form the basis of a lot of NLP projects. What other big datasets are there for forum post text data? The reddit dataset most immediately comes to mind, and I've also seen a similar one for HN comments. Any others?

mynewtb 10y ago
ArchiveTeam's web archives will be available to everyone without restrictions or profit as usual.
- amyjess 10y ago
  Well, thanks to your comment, I just found out ArchiveTeam exists. Thank you!
- kenorb 10y ago
  Where you can download the boards?
JimSmith84 10y ago
Yeah that's definitely something I could do. Are there any proposed projects you know of that could benefit from this type of data?
- anilgulecha 10y ago
  Everything from simple sentiment analysis, to archive.org, to another mirror. I hope that does not discourage you from releasing the data. Edit: I see the other comment about archive team already collecting and releasing this data, for free in an open format. I think that will be a good first source as well.
  - waynecolvin 10y ago
    I think the priority was to reenable discussion of new movies and TV shows, that would be useful moving forward. But maybe they could make an API.
- johnymontana 10y ago
  I'd love to explore how this data could be used to enhance recommender systems.
  - endymi0n 10y ago
    AFAIK, Jinni is doing exactly that.

sergers 10y ago

Hi thanks for posting, looks like it has some potential. one item i didnt like is that if i search for a show, i get hits for each episode. would be good to have one entry per show, and then you can drill down into individual episodes. also would be good feature on ability to rate shows (as a registered user, but i guess IMDB is still offering that functionality)

huac 10y ago
So basically reimplement IMDB?

otoburb 10y ago

Thank you for this. I'm curious if IMDB considers their (soon to be former) forum posts under copyright, or if you already have a plan in place. One actually hopes that IMDB doesn't in fact have any claim on posts written by users that happened to be hosted on their site.

JimSmith84 10y ago
Yeah that's a good question. Before I became an engineer I practiced IP/copyright law. Also spoke (colloquially) with a few ppl at the company. I did my homework before starting this project and I'm confident we're in a good position :)
- Guest98123 10y ago
  Could you explain why you think you're in a good position? From what I understand, the comments are owned by the users, and by posting them to IMDb, they allowed IMDb and their affiliates the right to use them. The users did not give you permission to use their data, and you're not associated with IMDb, and therefore do not inherit the rights. Lastly, IMDb specifically states that scraping or extracting data is forbidden without written consent.
  - icemelt8 10y ago
    I wish I could downvote you.
- otoburb 10y ago
  >>Before I became an engineer I practiced IP/copyright law. I'm sure there's a story there since I would have thought the usual direction, even for a small aggregate number of eventual engineer-patent attorney pairings, would be reversed. Hopefully you can post an "About Us" on MovieChat.org at some point when you feel comfortable. Thanks!
kristianp 10y ago
From http://www.imdb.com/board/announcement http://www.imdb.com/board/announcement "...disable our message boards on February 3, 2017 but will leave them open for two additional weeks so that users will have ample time to archive any message board content they'd like to keep for personal use." Note the "personal use" above. Is a website like MovieChat.org considered personal use? From http://www.imdb.com/conditions http://www.imdb.com/conditions , section Copyright: "The compilation of all content included in or made available through any IMDb Service is the exclusive property of IMDb...".

deleted 10y ago

[deleted]

karim 10y ago

Hey Jim – thanks for doing that! I'm curious, how did you back up IMDB's message boards? Did you scrape the whole website or were you able to find a data dump?

5_minutes 10y ago
I wanted to ask this too. Did you scrape it or did it via some api? Anyway, cool project, wish you all the best with it.
- Markoff 10y ago
  illegally scraped,IMDb doesn't have API for this and automated scraping OP done is against their TOS, not mentioning without consent of users, for instance stole 14 years worth of my posts
  - fwn 10y ago
    He stole them? You mean they are gone now? Because the closest I can imagine happening is an unlicensed reproduction of your content, your credentials included. ... Like any archive service would do it.
  - thebluecat 10y ago
    I take it then, when you posted these 14 years of comments, you didn't want them available to the public to, I don't know, read?! Thank goodness IMDB deleted them all before we happened across them by accident & violated your privacy.

cpeterso 10y ago

Interesting project? The IMDb message boards were a good way to get some additional insight after watching a movie. Or at least give me a good laugh at the trolls. :) Who owns this post data? IMDb? The users? Do you have any plans or ideas for existing IMDb users to somehow claim their username on MovieChat, linking to their archived posts?

quirkafleeg 10y ago
> The IMDb message boards were a good way to get some additional insight after watching a movie. Or at least give me a good laugh at the trolls. :) Totally agree. I often also went there before watching a TV show or movie to get an idea what it was about or if I'd like it (though wary of spoiler potential), whereas I almost never read the reviews there, which I find are often far more useless than the message board comments. After they're gone IMDB will be just a soulless movie listing site, with no reason to visit other than to check a mark out of ten, and no reason to stick around afterwards. :/

nso 10y ago

I made a userscript to be a drop-in replacement for the current discussion-teaser on a movies page on imdb, but it gets the discussions from moviechat.org (and links to them directly). For it to work I would need for Jim to enable Access-Control-Allow-Origin from imdb.com (and ideally make a json/xml version of the thread list since I'm regexp'ing the html at this point). Maybe it would help persuade him if we showed that there was some interest for this. https://imma.gr/66880x4ca8b https://imma.gr/66880x4ca8b

trippycheez 10y ago

Thank you, THANK YOU, THANK YOU!!!! I just heard about the IMDb message board deletion and was feeling deeply sad about it. I love cinema, and their message board (despite its flaws) was a valuable resource. It was the only place where you could go to participate in in-depth discussions of even the most obscure films. I used it for 14 years and learned so much from other users. As far as I'm concerned you are a SAINT for preserving this bit of film and internet history. Major gratitude to you for your caring and putting the new site together. I honestly want to cry, I am so relieved.

Rand0mPers0n 10y ago

Great work man... Seriously, thank you.