9 ms·
Show HN: MovieChat.org – Archive and Replacement for IMDb Message Boards
- JimSmith84 10y agoHi Everyone, My name’s Jim, and I created MovieChat.org as an archive and replacement for IMDB’s message boards which are shutting down this week. For those of you not familiar, the IMDb message boards allowed you to discuss any single movie or tv show with others (there was a separate forum for each movie/show). IMDb recently announced they were shutting down the message boards and its users were furious (there's a petition with close to 10k signatures here: https://www.ipetitions.com/petition/petition-to-keep-the-imdb-messageboards-going https://www.ipetitions.com/petition/petition-to-keep-the-imd...). I ventured out to create an archive (of all the existing posts) and replacement and hence MovieChat.org was born. Key Features of MovieChat.org: 1. Any movie/show on IMDB is also on MovieChat.org (over 4 million and counting) - we have separate boards for each movie/show, just like IMDB 2. I backed up most of the posts for IMDB’s top 10,000 movies/shows - most existing conversations on IMDB should also appear on MovieChat.org - we have over 3 million posts already (and I'm working non-stop to back up even more from IMDB)! Please visit http://MovieChat.org http://MovieChat.org, join or start a discussion, and let me know what you think. If you like it, please spread the word. If there’s anything I can improve, just email me (jim@moviechat.org) and I’ll get on it. Jim jim@moviechat.org
- dave_sullivan 10y agoThis is awesome and much needed. Just throwing it out there, but would you consider making a dump of the data you scraped that could be used by data scientists? Maybe as a torrent or something like that? Data about movies and what people say about them could form the basis of a lot of NLP projects. What other big datasets are there for forum post text data? The reddit dataset most immediately comes to mind, and I've also seen a similar one for HN comments. Any others?
- mynewtb 10y agoArchiveTeam's web archives will be available to everyone without restrictions or profit as usual.
- JimSmith84 10y agoYeah that's definitely something I could do. Are there any proposed projects you know of that could benefit from this type of data?
- anilgulecha 10y agoEverything from simple sentiment analysis, to archive.org, to another mirror. I hope that does not discourage you from releasing the data. Edit: I see the other comment about archive team already collecting and releasing this data, for free in an open format. I think that will be a good first source as well.
- waynecolvin 10y agoI think the priority was to reenable discussion of new movies and TV shows, that would be useful moving forward. But maybe they could make an API.
- johnymontana 10y agoI'd love to explore how this data could be used to enhance recommender systems.
- endymi0n 10y agoAFAIK, Jinni is doing exactly that.
- sergers 10y agoHi thanks for posting, looks like it has some potential. one item i didnt like is that if i search for a show, i get hits for each episode. would be good to have one entry per show, and then you can drill down into individual episodes. also would be good feature on ability to rate shows (as a registered user, but i guess IMDB is still offering that functionality)
- huac 10y agoSo basically reimplement IMDB?
- otoburb 10y agoThank you for this. I'm curious if IMDB considers their (soon to be former) forum posts under copyright, or if you already have a plan in place. One actually hopes that IMDB doesn't in fact have any claim on posts written by users that happened to be hosted on their site.
- JimSmith84 10y agoYeah that's a good question. Before I became an engineer I practiced IP/copyright law. Also spoke (colloquially) with a few ppl at the company. I did my homework before starting this project and I'm confident we're in a good position :)
- Guest98123 10y agoCould you explain why you think you're in a good position? From what I understand, the comments are owned by the users, and by posting them to IMDb, they allowed IMDb and their affiliates the right to use them. The users did not give you permission to use their data, and you're not associated with IMDb, and therefore do not inherit the rights. Lastly, IMDb specifically states that scraping or extracting data is forbidden without written consent.
- icemelt8 10y agoI wish I could downvote you.
- otoburb 10y ago>>Before I became an engineer I practiced IP/copyright law. I'm sure there's a story there since I would have thought the usual direction, even for a small aggregate number of eventual engineer-patent attorney pairings, would be reversed. Hopefully you can post an "About Us" on MovieChat.org at some point when you feel comfortable. Thanks!
- kristianp 10y agoFrom http://www.imdb.com/board/announcement http://www.imdb.com/board/announcement "...disable our message boards on February 3, 2017 but will leave them open for two additional weeks so that users will have ample time to archive any message board content they'd like to keep for personal use." Note the "personal use" above. Is a website like MovieChat.org considered personal use? From http://www.imdb.com/conditions http://www.imdb.com/conditions , section Copyright: "The compilation of all content included in or made available through any IMDb Service is the exclusive property of IMDb...".
- deleted 10y ago[deleted]
- karim 10y agoHey Jim – thanks for doing that! I'm curious, how did you back up IMDB's message boards? Did you scrape the whole website or were you able to find a data dump?
- 5_minutes 10y agoI wanted to ask this too. Did you scrape it or did it via some api? Anyway, cool project, wish you all the best with it.
- Markoff 10y agoillegally scraped,IMDb doesn't have API for this and automated scraping OP done is against their TOS, not mentioning without consent of users, for instance stole 14 years worth of my posts
- fwn 10y agoHe stole them? You mean they are gone now? Because the closest I can imagine happening is an unlicensed reproduction of your content, your credentials included. ... Like any archive service would do it.
- thebluecat 10y agoI take it then, when you posted these 14 years of comments, you didn't want them available to the public to, I don't know, read?! Thank goodness IMDB deleted them all before we happened across them by accident & violated your privacy.
- cpeterso 10y agoInteresting project? The IMDb message boards were a good way to get some additional insight after watching a movie. Or at least give me a good laugh at the trolls. :) Who owns this post data? IMDb? The users? Do you have any plans or ideas for existing IMDb users to somehow claim their username on MovieChat, linking to their archived posts?
- quirkafleeg 10y ago> The IMDb message boards were a good way to get some additional insight after watching a movie. Or at least give me a good laugh at the trolls. :) Totally agree. I often also went there before watching a TV show or movie to get an idea what it was about or if I'd like it (though wary of spoiler potential), whereas I almost never read the reviews there, which I find are often far more useless than the message board comments. After they're gone IMDB will be just a soulless movie listing site, with no reason to visit other than to check a mark out of ten, and no reason to stick around afterwards. :/
- nso 10y agoI made a userscript to be a drop-in replacement for the current discussion-teaser on a movies page on imdb, but it gets the discussions from moviechat.org (and links to them directly). For it to work I would need for Jim to enable Access-Control-Allow-Origin from imdb.com (and ideally make a json/xml version of the thread list since I'm regexp'ing the html at this point). Maybe it would help persuade him if we showed that there was some interest for this. https://imma.gr/66880x4ca8b https://imma.gr/66880x4ca8b
- trippycheez 10y agoThank you, THANK YOU, THANK YOU!!!! I just heard about the IMDb message board deletion and was feeling deeply sad about it. I love cinema, and their message board (despite its flaws) was a valuable resource. It was the only place where you could go to participate in in-depth discussions of even the most obscure films. I used it for 14 years and learned so much from other users. As far as I'm concerned you are a SAINT for preserving this bit of film and internet history. Major gratitude to you for your caring and putting the new site together. I honestly want to cry, I am so relieved.
- Rand0mPers0n 10y agoGreat work man... Seriously, thank you.
- supervill 10y agoHoly hell that's a ton of data you scraped
- CM30 10y agoIt's a good start. I mean, you've got the data from the original IMDB boards all laid out and accessible, so that's better than most alternatives at the moment. However, I'm curious about exactly how you're going to promote this site. The people currently on IMDB likely don't know much about it, and it's unlikely the administration there will redirect people over when they view the pages for a movie or TV show on their site. What's the plan to get the users whose data you scraped to come to this new domain and participate again?
- Markoff 10y agohe doesn't have movie database with ratings and watchlists, so i see no reason moving there when i can have all of this plus discussions at TMDb (yes, they are at the moment quite empty because nobody illegally copied posts from IMDb there, including mine)
- Markoff 10y agowhy would i participate on dumb backup of database, done without my consent (OP stole 14 years of my posts), when I can go to proper movie database with discussions, imported ratings and watchlist from IMDb, called TMDb (themoviedb.org)
- dewey 10y agoPlease add https, now with Let's encrypt there's really no excuse to not have it.
- carussell 10y agoLet's Encrypt can still be pretty convoluted and is not headache-free, even using the officially anointed tools. Despite your comment and the (well-intentioned) plans from browser vendors to do what they can to squash unencrypted HTTP, failure to use HTTPS, even with Let's Encrypt, is still a totally forgivable sin today.
- dewey 10y agoI disagree, by just running https://caddyserver.com/ https://caddyserver.com/ you would automatically have a https site up in seconds and it can be used as a reverse proxy too. With nginx: - `certbot certonly` - press `2` - type in your domain name - press return, done Add a few lines to your nginx config, done. ``` ssl_certificate /etc/letsencrypt/live/<yourdomain>/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/<yourdomain>/privkey.pem; ssl on; ```
- pmarreck 10y agoyeah but that's a webserver almost no one has heard of and may not even be in a position to switch to
- dewey 10y agoI'd say it's pretty well known if you have anything to do with Go but there's a simple tutorial for most web servers. If you are able to build a site like that, editing some web server configs by following a tutorial shouldn't be a problem. In any case, I like the project but I think that in 2017 it's irresponsible to do logins and user registration over http - even if it's just a weekend project.
- carussell 10y ago
- punjabisingh 10y agoGreat effort. I see some broken images here: http://www.moviechat.org/movies/search?name=westworld http://www.moviechat.org/movies/search?name=westworld I'm surprised there isn't enough interest in people to recognize tMDB as relevant. Especially since iMDB is ignoring users by taking down the forums.
- Markoff 10y agoagreed on second part that TMDb is the only real alternative, it has information about each movie and TV show, discussion for each of them and you can even import there your IMDb ratings and watchlist, plus is run since 2008 and edited by its users plus developer is very responsive and provide API free of charge
- WayneBro 10y agoThanks for making this. Some suggestions: - How about linking back to the IMDB page for each movie? - It would be great if you displayed various ratings from IMDB, Rotten Tomatoes and elsewhere kind of like JustWatch.com does (but with links to those ratings pages). - Actually, it would be really cool if you linked to a JustWatch search too so the user can easily see if the movie is streaming anywhere that they subscribe to. Another crazy thought - a merger with JustWatch would be awesome too (but I really wish JustWatch would also link back to IMDB, Rotten Tomatoes, etc).
- JimSmith84 10y agoYeah I was actually thinking of doing this, just been super busy trying to archive everything from IMDB before the boards shut down. Do you know the founders at JustWatch?
- endymi0n 10y agoHey Jim, one of the JustWatch founders here - we'd love to have linkouts as well. Unfortunately, interests with IMDB aren't super aligned at the moment and they're in business much, much longer than we are. Let's say they keep a pretty close guard on their data, as they've spent the last dozen years collecting it. But rest assured we're working heavily behind the scenes to make our site & UX better and better every day with free and open movie data sources - but always trying to stay safe on the legal side.
- chinathrow 10y agoNice - but have you consent of the comment authors to duplicate the content? Normally, UGC (user generated content) remains under copyright of the users.
- Markoff 10y agono, he doesn't and he illegally copied 14 years of my posts which i don't like especially since I moved to TMDb which is proper movie database where you can also import your IMDb ratings and watchlist and each movie there has own discussions
- thebluecat 10y agoWow, you really are a cry baby aren't you?!
- dirkg 10y agoThank you!! To my knowledge this is the only instance someone has managed to scrape the Imdb boards, very curious how you managed that. I really hope you get a sponsor/donations. This needs to be promoted to other sites/social media and I will do my part. I do have one request - can we do this for actors as well?
- mynewtb 10y agohttp://tracker.archiveteam.org/imdb/ http://tracker.archiveteam.org/imdb/ You can even easily join in if you want.
- Jaruzel 10y agoSomewhere buried on IMDB is the ability to download the IMDB database of actors/films/directors etc. I did it once, but discovered it's in such mess (schema wise) that it would be easier just scraping the actual IMDB pages I needed[1]. --- [1] To populate my Kodi .nfo files in a way that I was happy with. I know scraping is probably against IMDB t&cs, but as this was for my own use, I took the risk.
- Markoff 10y agopromoting illegally copied content of users without their consent? for what's it worth, he copied 14 years of my posts and I don't like it
- techer 10y agoTake a deep breath.
- dirkg 10y agoImdb also 'copied' all your posts, right? How is this any different?
- Markoff 10y agono, I published my comments on IMDb website under their ToS, so I am fine with them doing whatever they want on their website, I don't like it but it's their right, meanwhile this guy has no rights to this content
- blitzo 10y agoWhat most user's annoyed the most is years of profile status now gone. If you could somehow managed to get profile verification in sync that would do wonder.
- Markoff 10y agothank you for helping further fragmentation of IMDb alternatives by creating dumb forum like tons of other generic forums when there is REAL alternative movie database with discussions for each movie called The Movie database themoviedb.org run since 2008 and edited by its own users like Wikipedia it's nice you created searchable archive of IMDb discussions though guys at Archive are doing same anyway, not posing as alternative and I am not feeling very happy sometime took 14 years of my IMDb posts and put them on his website without my agreement
- Jaruzel 10y agoYou are fully within your legal right (based on most countries data protection laws - I do not know where you are), to reach out to MovieChat.org and request formally your data be deleted. If it bothers you that much, I fully recommend you do so. However, surely rule #1 of the internet is, once you've posted it, it's out there forever. Are you going to go around to every cache server, and everyone's browser cache and demand that your comments be deleted from there as well ?
- Markoff 10y agoshould not it be the other way? first he should have consent to publish that data and not that he illegally copy data and publish them and i must perform some action when he did some illegal? as much as i hate IMDb shutting down boards i hope they will close this illegal website OTOH i have no problem with recognized Archive.org instead of this solo thief
- fnj 10y agoPlease stop your trolling. Nothing is being "stolen".
- smylyface 10y agoFuckoff Markoff.Give it a rest already. It is OBVIOUS that you are affiliated with the site you keep whining about. You are worse than a spoiled toddler who doesn't want to share his toys. This guy is just trying to do something nice for everyone. People like you are the reason IMBD killed its message board. Go troll somewhere else.
- TazeTSchnitzel 10y agoMy university's internet blocks the site as “Pornography”. Was the domain name previously used for some other use, perhaps?
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- dini0519 10y agoJim-first thank you. But I accidentally signed up under my whole email when I meant to sign up only under the first part as a user name. Can you help? Dini
- trippycheez 10y agoThank you, THANK YOU, THANK YOU!!!! I just heard about the IMDb message board deletion and was feeling deeply sad about it. I love cinema, and their message board (despite its flaws) was a valuable resource. It was the only place where you could go to participate in in-depth discussions of even the most obscure films. I used it for 14 years and learned so much from other users. As far as I'm concerned you are a SAINT for preserving this bit of film and internet history. Major gratitude to you for your caring and putting the new site together. I honestly want to cry, I am so relieved.
- trippycheez 10y agoThank you, THANK YOU, THANK YOU!!!! I just heard about the IMDb message board deletion and was feeling deeply sad about it. I love cinema, and their message board (despite its flaws) was a valuable resource. It was the only place where you could go to participate in in-depth discussions of even the most obscure films. I used it for 14 years and learned so much from other users. As far as I'm concerned you are a SAINT for preserving this bit of film and internet history. Major gratitude to you for your caring and putting the new site together. I honestly want to cry, I am so relieved.
- Steve2k17 10y agotheimdbforum.com
- akadini519 10y agoSecond request-Help! I was active on IMDb under dini0519 but am now having trouble signing on under that name because I had previously signed up under my whole email not realizing that would be the username. Not really wanting my complete email floating around I tried to shorten my user name to the previous dini0519 and even just "dini" but nothing is working. I really want to be part of this community esp concerning the Twilight Zone boards, The Golden Girls, and Big Brother. Please help.